Fix [#31807] Ngon triangulation error

Notes:
*This implements a quite simple algorithm, which simply checks angles (actually, absolute cosines) of created tri and remaining face (which may be a tri, quad, or more NGon), so that both are "best" (ie avoid as much as possible too much narrow/wide corners), and also checks the new edge is OK (i.e. does not goes "out" of original face).
*Incidently, it fixes a typo in that bm_face_goodline() func!
*It's quite performant (a bit quicker than previous code, as far as I have tested it) and prevent creation of completely flat triangles as much as possible, but it's far from being a "best" solution (as it is still a "progressive" one)!
*It also introduces a new math func (in BLI_math_vector.h), cos_v3v3v3, which computes cosine (ie dot product of normalized vectors) and is roughly a quicker replacement for angle_v3v3v3, when real angles are not needed.

1 files changed, 1 insertions, 0 deletions

diff --git a/source/blender/blenlib/BLI_math_vector.h b/source/blender/blenlib/BLI_math_vector.h
index be492fb6fdd..8499a7f219c 100644
--- a/source/blender/blenlib/BLI_math_vector.h
+++ b/source/blender/blenlib/BLI_math_vector.h
@@ -191,6 +191,7 @@ float angle_v2v2v2(const float a[2], const float b[2], const float c[2]);
 float angle_normalized_v2v2(const float a[2], const float b[2]);
 float angle_v3v3(const float a[3], const float b[3]);
 float angle_v3v3v3(const float a[3], const float b[3], const float c[3]);
+float cos_v3v3v3(const float p1[3], const float p2[3], const float p3[3]);
 float angle_normalized_v3v3(const float v1[3], const float v2[3]);
 float angle_on_axis_v3v3v3_v3(const float v1[3], const float v2[3], const float v3[3], const float axis[3]);
 void angle_tri_v3(float angles[3], const float v1[3], const float v2[3], const float v3[3]);