fix [#33106] Decimate modifier/collapse give bad result

FLT_EPSILON was too small to use when checking if the edge collapse result is an error.

add invert_m3_m3_ex(), invert_m3_ex() functiosn which take an epsilon to check the determinant, saves calculating it twice per edge collapse.

1 files changed, 3 insertions, 0 deletions

diff --git a/source/blender/blenlib/BLI_math_matrix.h b/source/blender/blenlib/BLI_math_matrix.h
index 9e34631d460..0783a7981ea 100644
--- a/source/blender/blenlib/BLI_math_matrix.h
+++ b/source/blender/blenlib/BLI_math_matrix.h
@@ -98,6 +98,9 @@ void mul_m3_fl(float R[3][3], float f);
 void mul_m4_fl(float R[4][4], float f);
 void mul_mat3_m4_fl(float R[4][4], float f);
 
+int invert_m3_ex(float m[3][3], const float epsilon);
+int invert_m3_m3_ex(float m1[3][3], float m2[3][3], const float epsilon);
+
 int invert_m3(float R[3][3]);
 int invert_m3_m3(float R[3][3], float A[3][3]);
 int invert_m4(float R[4][4]);