Refactor: Move normals out of MVert, lazy calculation

As described in T91186, this commit moves mesh vertex normals into a
contiguous array of float vectors in a custom data layer, how face
normals are currently stored.

The main interface is documented in `BKE_mesh.h`. Vertex and face
normals are now calculated on-demand and cached, retrieved with an
"ensure" function. Since the logical state of a mesh is now "has
normals when necessary", they can be retrieved from a `const` mesh.

The goal is to use on-demand calculation for all derived data, but
leave room for eager calculation for performance purposes (modifier
evaluation is threaded, but viewport data generation is not).

**Benefits**
This moves us closer to a SoA approach rather than the current AoS
paradigm. Accessing a contiguous `float3` is much more efficient than
retrieving data from a larger struct. The memory requirements for
accessing only normals or vertex locations are smaller, and at the
cost of more memory usage for just normals, they now don't have to
be converted between float and short, which also simplifies code

In the future, the remaining items can be removed from `MVert`,
leaving only `float3`, which has similar benefits (see T93602).

Removing the combination of derived and original data makes it
conceptually simpler to only calculate normals when necessary.
This is especially important now that we have more opportunities
for temporary meshes in geometry nodes.

**Performance**
In addition to the theoretical future performance improvements by
making `MVert == float3`, I've done some basic performance testing
on this patch directly. The data is fairly rough, but it gives an idea
about where things stand generally.
 - Mesh line primitive 4m Verts: 1.16x faster (36 -> 31 ms),
   showing that accessing just `MVert` is now more efficient.
 - Spring Splash Screen: 1.03-1.06 -> 1.06-1.11 FPS, a very slight
   change that at least shows there is no regression.
 - Sprite Fright Snail Smoosh: 3.30-3.40 -> 3.42-3.50 FPS, a small
   but observable speedup.
 - Set Position Node with Scaled Normal: 1.36x faster (53 -> 39 ms),
   shows that using normals in geometry nodes is faster.
 - Normal Calculation 1.6m Vert Cube: 1.19x faster (25 -> 21 ms),
   shows that calculating normals is slightly faster now.
 - File Size of 1.6m Vert Cube: 1.03x smaller (214.7 -> 208.4 MB),
   Normals are not saved in files, which can help with large meshes.

As for memory usage, it may be slightly more in some cases, but
I didn't observe any difference in the production files I tested.

**Tests**
Some modifiers and cycles test results need to be updated with this
commit, for two reasons:
 - The subdivision surface modifier is not responsible for calculating
   normals anymore. In master, the modifier creates different normals
   than the result of the `Mesh` normal calculation, so this is a bug
   fix.
 - There are small differences in the results of some modifiers that
   use normals because they are not converted to and from `short`
   anymore.

**Future improvements**
 - Remove `ModifierTypeInfo::dependsOnNormals`. Code in each modifier
   already retrieves normals if they are needed anyway.
 - Copy normals as part of a better CoW system for attributes.
 - Make more areas use lazy instead of eager normal calculation.
 - Remove `BKE_mesh_normals_tag_dirty` in more places since that is
   now the default state of a new mesh.
 - Possibly apply a similar change to derived face corner normals.

Differential Revision: https://developer.blender.org/D12770

1 files changed, 19 insertions, 100 deletions

diff --git a/source/blender/blenkernel/intern/geometry_component_mesh.cc b/source/blender/blenkernel/intern/geometry_component_mesh.cc
index 88a58220b23..11b350ef2be 100644
--- a/source/blender/blenkernel/intern/geometry_component_mesh.cc
+++ b/source/blender/blenkernel/intern/geometry_component_mesh.cc
@@ -128,111 +128,45 @@ void MeshComponent::ensure_owns_direct_data()
 
 namespace blender::bke {
 
-static VArray<float3> mesh_face_normals(const Mesh &mesh,
-                                        const Span<MVert> verts,
-                                        const Span<MPoly> polys,
-                                        const Span<MLoop> loops,
-                                        const IndexMask mask)
-{
-  /* Use existing normals to avoid unnecessarily recalculating them, if possible. */
-  if (!(mesh.runtime.cd_dirty_poly & CD_MASK_NORMAL) &&
-      CustomData_has_layer(&mesh.pdata, CD_NORMAL)) {
-    const void *data = CustomData_get_layer(&mesh.pdata, CD_NORMAL);
-
-    return VArray<float3>::ForSpan({(const float3 *)data, polys.size()});
-  }
-
-  auto normal_fn = [verts, polys, loops](const int i) -> float3 {
-    float3 normal;
-    const MPoly &poly = polys[i];
-    BKE_mesh_calc_poly_normal(&poly, &loops[poly.loopstart], verts.data(), normal);
-    return normal;
-  };
-
-  return VArray<float3>::ForFunc(mask.min_array_size(), normal_fn);
-}
-
-static VArray<float3> mesh_vertex_normals(const Mesh &mesh,
-                                          const Span<MVert> verts,
-                                          const Span<MPoly> polys,
-                                          const Span<MLoop> loops,
-                                          const IndexMask mask)
-{
-  /* Use existing normals to avoid unnecessarily recalculating them, if possible. */
-  if (!(mesh.runtime.cd_dirty_vert & CD_MASK_NORMAL) &&
-      CustomData_has_layer(&mesh.vdata, CD_NORMAL)) {
-    const void *data = CustomData_get_layer(&mesh.pdata, CD_NORMAL);
-
-    return VArray<float3>::ForSpan({(const float3 *)data, mesh.totvert});
-  }
-
-  /* If the normals are dirty, they must be recalculated for the output of this node's field
-   * source. Ideally vertex normals could be calculated lazily on a const mesh, but that's not
-   * possible at the moment, so we take ownership of the results. Sadly we must also create a copy
-   * of MVert to use the mesh normals API. This can be improved by adding mutex-protected lazy
-   * calculation of normals on meshes.
-   *
-   * Use mask.min_array_size() to avoid calculating a final chunk of data if possible. */
-  Array<MVert> temp_verts(verts);
-  Array<float3> normals(verts.size()); /* Use full size for accumulation from faces. */
-  BKE_mesh_calc_normals_poly_and_vertex(temp_verts.data(),
-                                        mask.min_array_size(),
-                                        loops.data(),
-                                        loops.size(),
-                                        polys.data(),
-                                        polys.size(),
-                                        nullptr,
-                                        (float(*)[3])normals.data());
-
-  return VArray<float3>::ForContainer(std::move(normals));
-}
-
 VArray<float3> mesh_normals_varray(const MeshComponent &mesh_component,
                                    const Mesh &mesh,
                                    const IndexMask mask,
                                    const AttributeDomain domain)
 {
-  Span<MVert> verts{mesh.mvert, mesh.totvert};
-  Span<MEdge> edges{mesh.medge, mesh.totedge};
-  Span<MPoly> polys{mesh.mpoly, mesh.totpoly};
-  Span<MLoop> loops{mesh.mloop, mesh.totloop};
-
   switch (domain) {
     case ATTR_DOMAIN_FACE: {
-      return mesh_face_normals(mesh, verts, polys, loops, mask);
+      return VArray<float3>::ForSpan(
+          {(float3 *)BKE_mesh_poly_normals_ensure(&mesh), mesh.totpoly});
     }
     case ATTR_DOMAIN_POINT: {
-      return mesh_vertex_normals(mesh, verts, polys, loops, mask);
+      return VArray<float3>::ForSpan(
+          {(float3 *)BKE_mesh_vertex_normals_ensure(&mesh), mesh.totvert});
     }
     case ATTR_DOMAIN_EDGE: {
       /* In this case, start with vertex normals and convert to the edge domain, since the
-       * conversion from edges to vertices is very simple. Use the full mask since the edges
-       * might use the vertex normal from any index. */
-      GVArray vert_normals = mesh_vertex_normals(
-          mesh, verts, polys, loops, IndexRange(verts.size()));
-      Span<float3> vert_normals_span = vert_normals.get_internal_span().typed<float3>();
+       * conversion from edges to vertices is very simple. Use "manual" domain interpolation
+       * instead of the GeometryComponent API to avoid calculating unnecessary values and to
+       * allow normalizing the result more simply. */
+      Span<float3> vert_normals{(float3 *)BKE_mesh_vertex_normals_ensure(&mesh), mesh.totvert};
       Array<float3> edge_normals(mask.min_array_size());
-
-      /* Use "manual" domain interpolation instead of the GeometryComponent API to avoid
-       * calculating unnecessary values and to allow normalizing the result much more simply. */
+      Span<MEdge> edges{mesh.medge, mesh.totedge};
       for (const int i : mask) {
         const MEdge &edge = edges[i];
         edge_normals[i] = math::normalize(
-            math::interpolate(vert_normals_span[edge.v1], vert_normals_span[edge.v2], 0.5f));
+            math::interpolate(vert_normals[edge.v1], vert_normals[edge.v2], 0.5f));
       }
 
       return VArray<float3>::ForContainer(std::move(edge_normals));
     }
     case ATTR_DOMAIN_CORNER: {
       /* The normals on corners are just the mesh's face normals, so start with the face normal
-       * array and copy the face normal for each of its corners. */
-      VArray<float3> face_normals = mesh_face_normals(
-          mesh, verts, polys, loops, IndexRange(polys.size()));
-
-      /* In this case using the mesh component's generic domain interpolation is fine, the data
-       * will still be normalized, since the face normal is just copied to every corner. */
-      return mesh_component.attribute_try_adapt_domain<float3>(
-          std::move(face_normals), ATTR_DOMAIN_FACE, ATTR_DOMAIN_CORNER);
+       * array and copy the face normal for each of its corners. In this case using the mesh
+       * component's generic domain interpolation is fine, the data will still be normalized,
+       * since the face normal is just copied to every corner. */
+      return mesh_component.attribute_try_adapt_domain(
+          VArray<float3>::ForSpan({(float3 *)BKE_mesh_poly_normals_ensure(&mesh), mesh.totpoly}),
+          ATTR_DOMAIN_FACE,
+          ATTR_DOMAIN_CORNER);
     }
     default:
       return {};
@@ -1276,25 +1210,10 @@ class NormalAttributeProvider final : public BuiltinAttributeProvider {
   {
     const MeshComponent &mesh_component = static_cast<const MeshComponent &>(component);
     const Mesh *mesh = mesh_component.get_for_read();
-    if (mesh == nullptr) {
+    if (mesh == nullptr || mesh->totpoly == 0) {
       return {};
     }
-
-    /* Use existing normals if possible. */
-    if (!(mesh->runtime.cd_dirty_poly & CD_MASK_NORMAL) &&
-        CustomData_has_layer(&mesh->pdata, CD_NORMAL)) {
-      const void *data = CustomData_get_layer(&mesh->pdata, CD_NORMAL);
-
-      return VArray<float3>::ForSpan(Span<float3>((const float3 *)data, mesh->totpoly));
-    }
-
-    Array<float3> normals(mesh->totpoly);
-    for (const int i : IndexRange(mesh->totpoly)) {
-      const MPoly *poly = &mesh->mpoly[i];
-      BKE_mesh_calc_poly_normal(poly, &mesh->mloop[poly->loopstart], mesh->mvert, normals[i]);
-    }
-
-    return VArray<float3>::ForContainer(std::move(normals));
+    return VArray<float3>::ForSpan({(float3 *)BKE_mesh_poly_normals_ensure(mesh), mesh->totpoly});
   }
 
   WriteAttributeLookup try_get_for_write(GeometryComponent &UNUSED(component)) const final