Fix T83504: Cancel trim operators when there is no geometry

The boolean solver crashes when there is no geometry in the mesh. Also,
using the trimming tools without a valid intersection in the PBVH will
make the orientation and position functionality of the trimming shape
not work, so this is the safer solution.

Reviewed By: mont29

Maniphest Tasks: T83504

Differential Revision: https://developer.blender.org/D9777

1 files changed, 10 insertions, 0 deletions

diff --git a/source/blender/editors/sculpt_paint/paint_mask.c b/source/blender/editors/sculpt_paint/paint_mask.c
index 17690757fa5..92c78a674f0 100644
--- a/source/blender/editors/sculpt_paint/paint_mask.c
+++ b/source/blender/editors/sculpt_paint/paint_mask.c
@@ -1552,6 +1552,11 @@ static int sculpt_trim_gesture_box_exec(bContext *C, wmOperator *op)
     return OPERATOR_CANCELLED;
   }
 
+  if (ss->totvert == 0) {
+    /* No geometry to trim or to detect a valid position for the trimming shape. */
+    return OPERATOR_CANCELLED;
+  }
+
   SculptGestureContext *sgcontext = sculpt_gesture_init_from_box(C, op);
   if (!sgcontext) {
     return OPERATOR_CANCELLED;
@@ -1589,6 +1594,11 @@ static int sculpt_trim_gesture_lasso_exec(bContext *C, wmOperator *op)
     return OPERATOR_CANCELLED;
   }
 
+  if (ss->totvert == 0) {
+    /* No geometry to trim or to detect a valid position for the trimming shape. */
+    return OPERATOR_CANCELLED;
+  }
+
   SculptGestureContext *sgcontext = sculpt_gesture_init_from_lasso(C, op);
   if (!sgcontext) {
     return OPERATOR_CANCELLED;