Fix T78113: Random explosions of cloth with self collision

The problem is caused by a lack of prediction in the `isect_line_segment_tri_v3`
that incorrectly confirms some intersections of coplanar segments to the triangle.

The solution is to use another algorithm to detect intersections.

This also resulted in a slight improvement in the performance:
- 1min 17sec to 1min 6sec in my test file

Differential Revision: https://developer.blender.org/D8500

1 files changed, 2 insertions, 2 deletions

diff --git a/source/blender/python/mathutils/mathutils_bvhtree.c b/source/blender/python/mathutils/mathutils_bvhtree.c
index 5a0dc7d6a5e..6a007502d3e 100644
--- a/source/blender/python/mathutils/mathutils_bvhtree.c
+++ b/source/blender/python/mathutils/mathutils_bvhtree.c
@@ -549,8 +549,8 @@ static bool py_bvhtree_overlap_cb(void *userdata, int index_a, int index_b, int
     }
   }
 
-  return (isect_tri_tri_epsilon_v3(
-              UNPACK3(tri_a_co), UNPACK3(tri_b_co), ix_pair[0], ix_pair[1], data->epsilon) &&
+  return (isect_tri_tri_v3(
+              UNPACK3(tri_a_co), UNPACK3(tri_b_co), ix_pair[0], ix_pair[1]) &&
           ((verts_shared == 0) || (len_squared_v3v3(ix_pair[0], ix_pair[1]) > data->epsilon)));
 }