Face Sets: Add relax support to Mesh Filter and Draw Face Sets

This enables a relax operation that works only on face sets boundaries,
which smooths the jagged edges that are produced when painting or
expanding face sets by sliding the topology without affecting the shape
of the mesh. This has many uses in hard surface sculpting for things
like sculpting panels or smoothing surfaces. It can also help when
working with remeshed topology as it makes the face sets looks better
and more organized if needed.

The operation is implemented as an Shift smooth in the Draw Face sets
tool, similar to the Slide/Relax tool.

The same operation is also available in the mesh filter to smooth all
the face sets boundaries uniformly or to smooth the face set under the
cursor with the Use Face Sets option.

Reviewed By: brecht

Differential Revision: https://developer.blender.org/D7034

1 files changed, 15 insertions, 0 deletions

diff --git a/source/blender/editors/sculpt_paint/sculpt_intern.h b/source/blender/editors/sculpt_paint/sculpt_intern.h
index 090d1f33d74..7f90f86edd4 100644
--- a/source/blender/editors/sculpt_paint/sculpt_intern.h
+++ b/source/blender/editors/sculpt_paint/sculpt_intern.h
@@ -260,6 +260,13 @@ void SCULPT_multiplane_scrape_preview_draw(const uint gpuattr,
                                            const float outline_col[3],
                                            const float outline_alpha);
 
+/* Slide/Relax */
+void SCULPT_relax_vertex(struct SculptSession *ss,
+                         struct PBVHVertexIter *vd,
+                         float factor,
+                         bool filter_boundary_face_sets,
+                         float *r_final_pos);
+
 /* Sculpt Visibility API */
 void SCULPT_visibility_sync_all_face_sets_to_vertices(struct SculptSession *ss);
 void SCULPT_visibility_sync_all_vertex_to_face_sets(struct SculptSession *ss);
@@ -528,6 +535,10 @@ typedef struct StrokeCache {
   float location[3];
   float last_location[3];
 
+  /* Used for alternating between deformation in brushes that need to apply different ones to
+   * achieve certain effects. */
+  int iteration_count;
+
   /* Original pixel radius with the pressure curve applied for dyntopo detail size */
   float dyntopo_pixel_radius;
 
@@ -635,6 +646,10 @@ typedef struct FilterCache {
   bool enabled_axis[3];
   int random_seed;
 
+  /* Used for alternating between filter operations in filters that need to apply different ones to
+   * achieve certain effects. */
+  int iteration_count;
+
   /* unmasked nodes */
   PBVHNode **nodes;
   int totnode;