Sculpt: Normal-based automasking modes

Two new normal-based automasking modes.

The first mode, "brush", compares vertex normals with the initial
normal at the beginning of the brush stroke.

The second, "view", compares vertex normals with the view normal.
If "occlusion" is on then rays will be shot from each vertex to test
if it is occluded by other geometry (note: this can be very slow).\
Only geometry inside the sculpt mesh is considered.

Each mode has an associated angular limit and a falloff.

Reviewed by: Julien Kaspar and Jeroen Bakker
Differential Revision: https://developer.blender.org/D15297
Ref D15297

1 files changed, 8 insertions, 1 deletions

diff --git a/source/blender/editors/sculpt_paint/sculpt_ops.c b/source/blender/editors/sculpt_paint/sculpt_ops.c
index e56d967808f..64a55168977 100644
--- a/source/blender/editors/sculpt_paint/sculpt_ops.c
+++ b/source/blender/editors/sculpt_paint/sculpt_ops.c
@@ -1033,8 +1033,15 @@ static void sculpt_bake_cavity_exec_task_cb(void *__restrict userdata,
 
   SCULPT_undo_push_node(tdata->ob, node, SCULPT_UNDO_MASK);
 
+  AutomaskingNodeData automask_data;
+  SCULPT_automasking_node_begin(
+      tdata->ob, ss, ss->cache->automasking, &automask_data, node);
+
   BKE_pbvh_vertex_iter_begin (ss->pbvh, node, vd, PBVH_ITER_UNIQUE) {
-    float automask = SCULPT_automasking_factor_get(tdata->automasking, ss, vd.vertex);
+    SCULPT_automasking_node_update(ss, &automask_data, &vd);
+
+    float automask = SCULPT_automasking_factor_get(
+        tdata->automasking, ss, vd.vertex, &automask_data);
     float mask;
 
     switch (mode) {