Realtime Compositor: Implement dilate erode node

This patch implements the dilate/erode node for the realtime compositor.

Differential Revision: https://developer.blender.org/D15790

Reviewed By: Clement Foucault

9 files changed, 320 insertions, 0 deletions

diff --git a/source/blender/gpu/shaders/compositor/compositor_morphological_distance.glsl b/source/blender/gpu/shaders/compositor/compositor_morphological_distance.glsl
new file mode 100644
index 00000000000..09f896b7a9d
--- /dev/null
+++ b/source/blender/gpu/shaders/compositor/compositor_morphological_distance.glsl
@@ -0,0 +1,24 @@
+#pragma BLENDER_REQUIRE(common_math_lib.glsl)
+#pragma BLENDER_REQUIRE(gpu_shader_compositor_texture_utilities.glsl)
+
+void main()
+{
+  ivec2 texel = ivec2(gl_GlobalInvocationID.xy);
+
+  /* Find the minimum/maximum value in the circular window of the given radius around the pixel. By
+   * circular window, we mean that pixels in the window whose distance to the center of window is
+   * larger than the given radius are skipped and not considered. Consequently, the dilation or
+   * erosion that take place produces round results as opposed to squarish ones. This is
+   * essentially a morphological operator with a circular structuring element. The LIMIT value
+   * should be FLT_MAX if OPERATOR is min and FLT_MIN if OPERATOR is max. */
+  float value = LIMIT;
+  for (int y = -radius; y <= radius; y++) {
+    for (int x = -radius; x <= radius; x++) {
+      if (x * x + y * y <= radius * radius) {
+        value = OPERATOR(value, texture_load(input_tx, texel + ivec2(x, y), vec4(LIMIT)).x);
+      }
+    }
+  }
+
+  imageStore(output_img, texel, vec4(value));
+}
diff --git a/source/blender/gpu/shaders/compositor/compositor_morphological_distance_feather.glsl b/source/blender/gpu/shaders/compositor/compositor_morphological_distance_feather.glsl
new file mode 100644
index 00000000000..8034f4a3ebd
--- /dev/null
+++ b/source/blender/gpu/shaders/compositor/compositor_morphological_distance_feather.glsl
@@ -0,0 +1,101 @@
+#pragma BLENDER_REQUIRE(gpu_shader_compositor_texture_utilities.glsl)
+
+/* The Morphological Distance Feather operation is a linear combination between the result of two
+ * operations. The first operation is a Gaussian blur with a radius equivalent to the dilate/erode
+ * distance, which is straightforward and implemented as a separable filter similar to the blur
+ * operation.
+ *
+ * The second operation is an approximation of a morphological inverse distance operation evaluated
+ * at a distance falloff function. The result of a morphological inverse distance operation is a
+ * narrow band distance field that starts at its maximum value at boundaries where a difference in
+ * values took place and linearly deceases until it reaches zero in the span of a number of pixels
+ * equivalent to the erode/dilate distance. Additionally, instead of linearly decreasing, the user
+ * may choose a different falloff which is evaluated at the computed distance. For dilation, the
+ * distance field decreases outwards, and for erosion, the distance field decreased inwards.
+ *
+ * The reason why the result of a Gaussian blur is mixed in with the distance field is because the
+ * distance field is merely approximated and not accurately computed, the defects of which is more
+ * apparent away from boundaries and especially at corners where the distance field should take a
+ * circular shape. That's why the Gaussian blur is mostly mixed only further from boundaries.
+ *
+ * The morphological inverse distance operation is approximated using a separable implementation
+ * and intertwined with the Gaussian blur implementation as follows. A search window of a radius
+ * equivalent to the dilate/erode distance is applied on the image to find either the minimum or
+ * maximum pixel value multiplied by its corresponding falloff value in the window. For dilation,
+ * we try to find the maximum, and for erosion, we try to find the minimum. Additionally, we also
+ * save the falloff value where the minimum or maximum was found. The found value will be that of
+ * the narrow band distance field and the saved falloff value will be used as the mixing factor
+ * with the Gaussian blur.
+ *
+ * To make sense of the aforementioned algorithm, assume we are dilating a binary image by 5 pixels
+ * whose half has a value of 1 and the other half has a value of zero. Consider the following:
+ *
+ * - A pixel of value 1 already has the maximum possible value, so its value will remain unchanged
+ *   regardless of its position.
+ * - A pixel of value 0 that is right at the boundary of the 1's region will have a maximum value
+ *   of around 0.8 depending on the falloff. That's because the search window intersects the 1's
+ *   region, which when multiplied by the falloff gives the first value of the falloff, which is
+ *   larger than the initially zero value computed at the center of the search window.
+ * - A pixel of value 0 that is 3 pixels away from the boundary will have a maximum value of around
+ *   0.4 depending on the falloff. That's because the search window intersects the 1's region,
+ *   which when multiplied by the falloff gives the third value of the falloff, which is larger
+ *   than the initially zero value computed at the center of the search window.
+ * - Finally, a pixel of value 0 that is 6 pixels away from the boundary will have a maximum value
+ *   of 0, because the search window doesn't intersects the 1's region and only spans zero values.
+ *
+ * The previous example demonstrates how the distance field naturally arises, and the same goes for
+ * the erode case, except the minimum value is computed instead.
+ */
+void main()
+{
+  ivec2 texel = ivec2(gl_GlobalInvocationID.xy);
+
+  /* A value for accumulating the blur result. */
+  float accumulated_value = 0.0;
+
+  /* Compute the contribution of the center pixel to the blur result. */
+  float center_value = texture_load(input_tx, texel).x;
+  accumulated_value += center_value * texture_load(weights_tx, 0).x;
+
+  /* Start with the center value as the maximum/minimum distance and reassign to the true maximum
+   * or minimum in the search loop below. Additionally, the center falloff is always 1.0, so start
+   * with that.  */
+  float limit_distance = center_value;
+  float limit_distance_falloff = 1.0;
+
+  /* Compute the contributions of the pixels to the right and left, noting that the weights and
+   * falloffs textures only store the weights and falloffs for the positive half, but since the
+   * they are both symmetric, the same weights and falloffs are used for the negative half and we
+   * compute both of their contributions. */
+  for (int i = 1; i < texture_size(weights_tx); i++) {
+    float weight = texture_load(weights_tx, i).x;
+    float falloff = texture_load(falloffs_tx, i).x;
+
+    /* Loop for two iterations, where s takes the value of -1 and 1, which is used as the sign
+     * needed to evaluated the positive and negative sides as explain above. */
+    for (int s = -1; s < 2; s += 2) {
+      /* Compute the contribution of the pixel to the blur result. */
+      float value = texture_load(input_tx, texel + ivec2(s * i, 0)).x;
+      accumulated_value += value * weight;
+
+      /* The distance is computed such that its highest value is the pixel value itself, so
+       * multiply the distance falloff by the pixel value. */
+      float falloff_distance = value * falloff;
+
+      /* Find either the maximum or the minimum for the dilate and erode cases respectively. */
+      if (COMPARE(falloff_distance, limit_distance)) {
+        limit_distance = falloff_distance;
+        limit_distance_falloff = falloff;
+      }
+    }
+  }
+
+  /* Mix between the limit distance and the blurred accumulated value such that the limit distance
+   * is used for pixels closer to the boundary and the blurred value is used for pixels away from
+   * the boundary. */
+  float value = mix(accumulated_value, limit_distance, limit_distance_falloff);
+
+  /* Write the value using the transposed texel. See the execute_distance_feather_horizontal_pass
+   * method for more information on the rational behind this. */
+  imageStore(output_img, texel.yx, vec4(value));
+}
diff --git a/source/blender/gpu/shaders/compositor/compositor_morphological_distance_threshold.glsl b/source/blender/gpu/shaders/compositor/compositor_morphological_distance_threshold.glsl
new file mode 100644
index 00000000000..5931c4f0271
--- /dev/null
+++ b/source/blender/gpu/shaders/compositor/compositor_morphological_distance_threshold.glsl
@@ -0,0 +1,88 @@
+#pragma BLENDER_REQUIRE(gpu_shader_compositor_texture_utilities.glsl)
+
+/* The Morphological Distance Threshold operation is effectively three consecutive operations
+ * implemented as a single operation. The three operations are as follows:
+ *
+ * .-----------.   .--------------.   .----------------.
+ * | Threshold |-->| Dilate/Erode |-->| Distance Inset |
+ * '-----------'   '--------------'   '----------------'
+ *
+ * The threshold operation just converts the input into a binary image, where the pixel is 1 if it
+ * is larger than 0.5 and 0 otherwise. Pixels that are 1 in the output of the threshold operation
+ * are said to be masked. The dilate/erode operation is a dilate or erode morphological operation
+ * with a circular structuring element depending on the sign of the distance, where it is a dilate
+ * operation if the distance is positive and an erode operation otherwise. This is equivalent to
+ * the Morphological Distance operation, see its implementation for more information. Finally, the
+ * distance inset is an operation that converts the binary image into a narrow band distance field.
+ * That is, pixels that are unmasked will remain 0, while pixels that are masked will start from
+ * zero at the boundary of the masked region and linearly increase until reaching 1 in the span of
+ * a number pixels given by the inset value.
+ *
+ * As a performance optimization, the dilate/erode operation is omitted and its effective result is
+ * achieved by slightly adjusting the distance inset operation. The base distance inset operation
+ * works by computing the signed distance from the current center pixel to the nearest pixel with a
+ * different value. Since our image is a binary image, that means that if the pixel is masked, we
+ * compute the signed distance to the nearest unmasked pixel, and if the pixel unmasked, we compute
+ * the signed distance to the nearest masked pixel. The distance is positive if the pixel is masked
+ * and negative otherwise. The distance is then normalized by dividing by the given inset value and
+ * clamped to the [0, 1] range. Since distances larger than the inset value are eventually clamped,
+ * the distance search window is limited to a radius equivalent to the inset value.
+ *
+ * To archive the effective result of the omitted dilate/erode operation, we adjust the distance
+ * inset operation as follows. First, we increase the radius of the distance search window by the
+ * radius of the dilate/erode operation. Then we adjust the resulting narrow band signed distance
+ * field as follows.
+ *
+ * For the erode case, we merely subtract the erode distance, which makes the outermost erode
+ * distance number of pixels zero due to clamping, consequently achieving the result of the erode,
+ * while retaining the needed inset because we increased the distance search window by the same
+ * amount we subtracted.
+ *
+ * Similarly, for the dilate case, we add the dilate distance, which makes the dilate distance
+ * number of pixels just outside of the masked region positive and part of the narrow band distance
+ * field, consequently achieving the result of the dilate, while at the same time, the innermost
+ * dilate distance number of pixels become 1 due to clamping, retaining the needed inset because we
+ * increased the distance search window by the same amount we added.
+ *
+ * Since the erode/dilate distance is already signed appropriately as described before, we just add
+ * it in both cases. */
+void main()
+{
+  ivec2 texel = ivec2(gl_GlobalInvocationID.xy);
+
+  /* Apply a threshold operation on the center pixel, where the threshold is currently hard-coded
+   * at 0.5. The pixels with values larger than the threshold are said to be masked. */
+  bool is_center_masked = texture_load(input_tx, texel).x > 0.5;
+
+  /* Since the distance search window will access pixels outside of the bounds of the image, we use
+   * a texture loader with a fallback value. And since we don't want those values to affect the
+   * result, the fallback value is chosen such that the inner condition fails, which is when the
+   * sampled pixel and the center pixel are the same, so choose a fallback that will be considered
+   * masked if the center pixel is masked and unmasked otherwise.  */
+  vec4 fallback = vec4(is_center_masked ? 1.0 : 0.0);
+
+  /* Since the distance search window is limited to the given radius, the maximum possible squared
+   * distance to the center is double the squared radius. */
+  int minimum_squared_distance = radius * radius * 2;
+
+  /* Find the squared distance to the nearest different pixel in the search window of the given
+   * radius. */
+  for (int y = -radius; y <= radius; y++) {
+    for (int x = -radius; x <= radius; x++) {
+      bool is_sample_masked = texture_load(input_tx, texel + ivec2(x, y), fallback).x > 0.5;
+      if (is_center_masked != is_sample_masked) {
+        minimum_squared_distance = min(minimum_squared_distance, x * x + y * y);
+      }
+    }
+  }
+
+  /* Compute the actual distance from the squared distance and assign it an appropriate sign
+   * depending on whether it lies in a masked region or not. */
+  float signed_minimum_distance = sqrt(minimum_squared_distance) * (is_center_masked ? 1.0 : -1.0);
+
+  /* Add the erode/dilate distance and divide by the inset amount as described in the discussion,
+   * then clamp to the [0, 1] range. */
+  float value = clamp((signed_minimum_distance + distance) / inset, 0.0, 1.0);
+
+  imageStore(output_img, texel, vec4(value));
+}
diff --git a/source/blender/gpu/shaders/compositor/compositor_morphological_step.glsl b/source/blender/gpu/shaders/compositor/compositor_morphological_step.glsl
new file mode 100644
index 00000000000..6992bc2afa5
--- /dev/null
+++ b/source/blender/gpu/shaders/compositor/compositor_morphological_step.glsl
@@ -0,0 +1,19 @@
+#pragma BLENDER_REQUIRE(common_math_lib.glsl)
+#pragma BLENDER_REQUIRE(gpu_shader_compositor_texture_utilities.glsl)
+
+void main()
+{
+  ivec2 texel = ivec2(gl_GlobalInvocationID.xy);
+
+  /* Find the minimum/maximum value in the window of the given radius around the pixel. This is
+   * essentially a morphological operator with a square structuring element. The LIMIT value should
+   * be FLT_MAX if OPERATOR is min and FLT_MIN if OPERATOR is max. */
+  float value = LIMIT;
+  for (int i = -radius; i <= radius; i++) {
+    value = OPERATOR(value, texture_load(input_tx, texel + ivec2(i, 0), vec4(LIMIT)).x);
+  }
+
+  /* Write the value using the transposed texel. See the execute_step_horizontal_pass method for
+   * more information on the rational behind this. */
+  imageStore(output_img, texel.yx, vec4(value));
+}
diff --git a/source/blender/gpu/shaders/compositor/infos/compositor_morphological_distance_feather_info.hh b/source/blender/gpu/shaders/compositor/infos/compositor_morphological_distance_feather_info.hh
new file mode 100644
index 00000000000..9f17f60129d
--- /dev/null
+++ b/source/blender/gpu/shaders/compositor/infos/compositor_morphological_distance_feather_info.hh
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "gpu_shader_create_info.hh"
+
+GPU_SHADER_CREATE_INFO(compositor_morphological_distance_feather_shared)
+    .local_group_size(16, 16)
+    .sampler(0, ImageType::FLOAT_2D, "input_tx")
+    .sampler(1, ImageType::FLOAT_1D, "weights_tx")
+    .sampler(2, ImageType::FLOAT_1D, "falloffs_tx")
+    .image(0, GPU_RGBA16F, Qualifier::WRITE, ImageType::FLOAT_2D, "output_img")
+    .compute_source("compositor_morphological_distance_feather.glsl");
+
+GPU_SHADER_CREATE_INFO(compositor_morphological_distance_feather_dilate)
+    .additional_info("compositor_morphological_distance_feather_shared")
+    .define("COMPARE(x, y)", "x > y")
+    .do_static_compilation(true);
+
+GPU_SHADER_CREATE_INFO(compositor_morphological_distance_feather_erode)
+    .additional_info("compositor_morphological_distance_feather_shared")
+    .define("COMPARE(x, y)", "x < y")
+    .do_static_compilation(true);
diff --git a/source/blender/gpu/shaders/compositor/infos/compositor_morphological_distance_info.hh b/source/blender/gpu/shaders/compositor/infos/compositor_morphological_distance_info.hh
new file mode 100644
index 00000000000..fc960e119e5
--- /dev/null
+++ b/source/blender/gpu/shaders/compositor/infos/compositor_morphological_distance_info.hh
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "gpu_shader_create_info.hh"
+
+GPU_SHADER_CREATE_INFO(compositor_morphological_distance_shared)
+    .local_group_size(16, 16)
+    .push_constant(Type::INT, "radius")
+    .sampler(0, ImageType::FLOAT_2D, "input_tx")
+    .image(0, GPU_R16F, Qualifier::WRITE, ImageType::FLOAT_2D, "output_img")
+    .compute_source("compositor_morphological_distance.glsl");
+
+GPU_SHADER_CREATE_INFO(compositor_morphological_distance_dilate)
+    .additional_info("compositor_morphological_distance_shared")
+    .define("OPERATOR(a, b)", "max(a, b)")
+    .define("LIMIT", "FLT_MIN")
+    .do_static_compilation(true);
+
+GPU_SHADER_CREATE_INFO(compositor_morphological_distance_erode)
+    .additional_info("compositor_morphological_distance_shared")
+    .define("OPERATOR(a, b)", "min(a, b)")
+    .define("LIMIT", "FLT_MAX")
+    .do_static_compilation(true);
diff --git a/source/blender/gpu/shaders/compositor/infos/compositor_morphological_distance_threshold_info.hh b/source/blender/gpu/shaders/compositor/infos/compositor_morphological_distance_threshold_info.hh
new file mode 100644
index 00000000000..b1d64f61b80
--- /dev/null
+++ b/source/blender/gpu/shaders/compositor/infos/compositor_morphological_distance_threshold_info.hh
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "gpu_shader_create_info.hh"
+
+GPU_SHADER_CREATE_INFO(compositor_morphological_distance_threshold)
+    .local_group_size(16, 16)
+    .push_constant(Type::INT, "radius")
+    .push_constant(Type::INT, "distance")
+    .push_constant(Type::FLOAT, "inset")
+    .sampler(0, ImageType::FLOAT_2D, "input_tx")
+    .image(0, GPU_R16F, Qualifier::WRITE, ImageType::FLOAT_2D, "output_img")
+    .compute_source("compositor_morphological_distance_threshold.glsl")
+    .do_static_compilation(true);
diff --git a/source/blender/gpu/shaders/compositor/infos/compositor_morphological_step_info.hh b/source/blender/gpu/shaders/compositor/infos/compositor_morphological_step_info.hh
new file mode 100644
index 00000000000..e97ffd9feea
--- /dev/null
+++ b/source/blender/gpu/shaders/compositor/infos/compositor_morphological_step_info.hh
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "gpu_shader_create_info.hh"
+
+GPU_SHADER_CREATE_INFO(compositor_morphological_step_shared)
+    .local_group_size(16, 16)
+    .push_constant(Type::INT, "radius")
+    .sampler(0, ImageType::FLOAT_2D, "input_tx")
+    .image(0, GPU_R16F, Qualifier::WRITE, ImageType::FLOAT_2D, "output_img")
+    .compute_source("compositor_morphological_step.glsl");
+
+GPU_SHADER_CREATE_INFO(compositor_morphological_step_dilate)
+    .additional_info("compositor_morphological_step_shared")
+    .define("OPERATOR(a, b)", "max(a, b)")
+    .define("LIMIT", "FLT_MIN")
+    .do_static_compilation(true);
+
+GPU_SHADER_CREATE_INFO(compositor_morphological_step_erode)
+    .additional_info("compositor_morphological_step_shared")
+    .define("OPERATOR(a, b)", "min(a, b)")
+    .define("LIMIT", "FLT_MAX")
+    .do_static_compilation(true);
diff --git a/source/blender/gpu/shaders/compositor/library/gpu_shader_compositor_texture_utilities.glsl b/source/blender/gpu/shaders/compositor/library/gpu_shader_compositor_texture_utilities.glsl
index 00e9a391097..128fc6aeaf5 100644
--- a/source/blender/gpu/shaders/compositor/library/gpu_shader_compositor_texture_utilities.glsl
+++ b/source/blender/gpu/shaders/compositor/library/gpu_shader_compositor_texture_utilities.glsl
@@ -23,3 +23,13 @@ vec4 texture_load(sampler2D sampler, ivec2 texel)
   const ivec2 texture_bounds = texture_size(sampler) - ivec2(1);
   return texelFetch(sampler, clamp(texel, ivec2(0), texture_bounds), 0);
 }
+
+/* A shorthand for 2D texelFetch with zero LOD and a fallback value for out-of-bound access. */
+vec4 texture_load(sampler2D sampler, ivec2 texel, vec4 fallback)
+{
+  const ivec2 texture_bounds = texture_size(sampler) - ivec2(1);
+  if (any(lessThan(texel, ivec2(0))) || any(greaterThan(texel, texture_bounds))) {
+    return fallback;
+  }
+  return texelFetch(sampler, texel, 0);
+}