Mesh Edit: preserve Custom Normal vectors in topology operators.

Custom Loop Normals are normally encoded relative to the default
normals, similar to normal maps, allowing them to naturally follow
mesh deformations. Changes to mesh topology however often result
in nonsensical effects that are not desired.

The Remove Doubles operation especially (now known as Merge By
Distance) is intended as a purely topological operation, and
definitely should not change the vector of the custom normals.

This patch implements that behavior by converting the relative
encoding into an absolute vector layer for the duration of the
operation. It also modifies other Merge types in this way for
consistency, the Rip operator as their inverse counterpart;
and also Delete, Dissolve, Connect Path and Knife operators
as other examples more related to topology than shape.

On the technical side, this ports mesh_normals_loop_custom_set
to BMesh, and then uses a temporary Custom Data layer to store
the normals as vectors for the duration of the above mentioned
operations. When the normals are converted back to custom data,
the caller can choose whether to mark edges as sharp to preserve
distinct normals, or just average them instead. All but Remove
Doubles choose to average for now.

Differential Revision: https://developer.blender.org/D4994

1 files changed, 3 insertions, 0 deletions

diff --git a/source/blender/bmesh/intern/bmesh_mesh.h b/source/blender/bmesh/intern/bmesh_mesh.h
index fa542ba5f12..4ba0d948499 100644
--- a/source/blender/bmesh/intern/bmesh_mesh.h
+++ b/source/blender/bmesh/intern/bmesh_mesh.h
@@ -73,6 +73,9 @@ struct BMLoopNorEditDataArray *BM_loop_normal_editdata_array_init(BMesh *bm,
                                                                   const bool do_all_loops_of_vert);
 void BM_loop_normal_editdata_array_free(struct BMLoopNorEditDataArray *lnors_ed_arr);
 
+bool BM_custom_loop_normals_to_vector_layer(struct BMesh *bm);
+void BM_custom_loop_normals_from_vector_layer(struct BMesh *bm, bool add_sharp_edges);
+
 void BM_edges_sharp_from_angle_set(BMesh *bm, const float split_angle);
 
 void bmesh_edit_begin(BMesh *bm, const BMOpTypeFlag type_flag);