Fix T72721: Add visibility flags updates to the PBVH

Currently, there its a function that sets manually the fully_hidden flag
of the nodes from the visibility operators in paint_hide.c. The undo
code was not updating the flag, so the visibility state of the nodes was
incorrect after preforming undo operations. This sometimes was drawing
fully hidden nodes with empty buffers, causing artifacts in the
geometry.

I added a function to mark nodes which visibility state changed (similar
as we are updating the mask flags and the nodes bounding boxes). This
way, the tools, operators and undo code don't have to update the
visibility flags, making everything much simpler to understand and
maintain.

I did not remove the flag update code from the current visibility
operators in this patch, but after reimplementing them (and all the new
ones) in the new visibility system, all visibility updates should be
done using this method and the BKE_pbvh_node_fully_hidden_set function
should be removed.

Reviewed By: jbakker

Maniphest Tasks: T72721

Differential Revision: https://developer.blender.org/D6767

1 files changed, 5 insertions, 0 deletions

diff --git a/source/blender/editors/sculpt_paint/sculpt_undo.c b/source/blender/editors/sculpt_paint/sculpt_undo.c
index cd34e2c6f99..bc4ed0bf7bf 100644
--- a/source/blender/editors/sculpt_paint/sculpt_undo.c
+++ b/source/blender/editors/sculpt_paint/sculpt_undo.c
@@ -80,6 +80,7 @@ static void update_cb(PBVHNode *node, void *rebuild)
   BKE_pbvh_node_mark_update_mask(node);
   if (*((bool *)rebuild)) {
     BKE_pbvh_node_mark_rebuild_draw(node);
+    BKE_pbvh_node_mark_visibility_update(node);
   }
   BKE_pbvh_node_fully_hidden_set(node, 0);
 }
@@ -641,6 +642,10 @@ static void sculpt_undo_restore_list(bContext *C, Depsgraph *depsgraph, ListBase
       tag_update |= true;
     }
 
+    if (update_visibility) {
+      BKE_pbvh_update_visibility(ss->pbvh);
+    }
+
     if (BKE_pbvh_type(ss->pbvh) == PBVH_FACES && update_visibility) {
       Mesh *mesh = ob->data;
       BKE_mesh_flush_hidden_from_verts(mesh);