Fix T82736, Exact Boolean fail with repeated subtraction of same object.

Two problems were fixed. One, the code for dissolving vertices
left a face around if dissolving a vertex would leave less than
three vertices. Instead, the face should be deleted.
Two, with transformations like "rotate 180 degrees", this should
be no problem with exact, but the current transformation matrix
has very small non-zero entries where it shouldn't. Cleaning the
transformation matrix makes it more likely that user expectations
about coplanar faces will be fulfilled.

1 files changed, 7 insertions, 1 deletions

diff --git a/source/blender/blenlib/BLI_mesh_intersect.hh b/source/blender/blenlib/BLI_mesh_intersect.hh
index 877363b998a..ddda3edf2ff 100644
--- a/source/blender/blenlib/BLI_mesh_intersect.hh
+++ b/source/blender/blenlib/BLI_mesh_intersect.hh
@@ -327,8 +327,14 @@ class IMesh {
    * Replace face at given index with one that elides the
    * vertices at the positions in face_pos_erase that are true.
    * Use arena to allocate the new face in.
+   * This may end up setting the face at f_index to NULL.
+   * Return true if that is so, else return false.
+   * The caller may want to use remove_null_faces if any face
+   * was removed, to avoid the need to check for null faces later.
    */
-  void erase_face_positions(int f_index, Span<bool> face_pos_erase, IMeshArena *arena);
+  bool erase_face_positions(int f_index, Span<bool> face_pos_erase, IMeshArena *arena);
+
+  void remove_null_faces();
 };
 
 std::ostream &operator<<(std::ostream &os, const IMesh &mesh);