Geometry Nodes: use virtual arrays in internal attribute api

A virtual array is a data structure that is similar to a normal array
in that its elements can be accessed by an index. However, a virtual
array does not have to be a contiguous array internally. Instead, its
elements can be layed out arbitrarily while element access happens
through a virtual function call. However, the virtual array data
structures are designed so that the virtual function call can be avoided
in cases where it could become a bottleneck.

Most commonly, a virtual array is backed by an actual array/span or
is a single value internally, that is the same for every index.
Besides those, there are many more specialized virtual arrays like the
ones that provides vertex positions based on the `MVert` struct or
vertex group weights.

Not all attributes used by geometry nodes are stored in simple contiguous
arrays. To provide uniform access to all kinds of attributes, the attribute
API has to provide virtual array functionality that hides the implementation
details of attributes.

Before this refactor, the attribute API provided its own virtual array
implementation as part of the `ReadAttribute` and `WriteAttribute` types.
That resulted in unnecessary code duplication with the virtual array system.
Even worse, it bound many algorithms used by geometry nodes to the specifics
of the attribute API, even though they could also use different data sources
(such as data from sockets, default values, later results of expressions, ...).

This refactor removes the `ReadAttribute` and `WriteAttribute` types and
replaces them with `GVArray` and `GVMutableArray` respectively. The `GV`
stands for "generic virtual". The "generic" means that the data type contained
in those virtual arrays is only known at run-time. There are the corresponding
statically typed types `VArray<T>` and `VMutableArray<T>` as well.

No regressions are expected from this refactor. It does come with one
improvement for users. The attribute API can convert the data type
on write now. This is especially useful when writing to builtin attributes
like `material_index` with e.g. the Attribute Math node (which usually
just writes to float attributes, while `material_index` is an integer attribute).

Differential Revision: https://developer.blender.org/D10994

1 files changed, 74 insertions, 108 deletions

diff --git a/source/blender/blenkernel/BKE_geometry_set.hh b/source/blender/blenkernel/BKE_geometry_set.hh
index d94b2e7902b..f5041d2bf5f 100644
--- a/source/blender/blenkernel/BKE_geometry_set.hh
+++ b/source/blender/blenkernel/BKE_geometry_set.hh
@@ -56,60 +56,6 @@ class ComponentAttributeProviders;
 class GeometryComponent;
 
 /**
- * An #OutputAttributePtr wraps a #WriteAttributePtr that might not be stored in its final
- * destination yet. Therefore, once the attribute has been filled with data, the #save method has
- * to be called, to store the attribute where it belongs (possibly by replacing an existing
- * attribute with the same name).
- *
- * This is useful for example in the Attribute Color Ramp node, when the same attribute name is
- * used as input and output. Typically the input is a float attribute, and the output is a color.
- * Those two attributes cannot exist at the same time, due to a name collision. To handle this
- * situation well, first the output colors have to be computed before the input floats are deleted.
- * Therefore, the outputs have to be written to a temporary buffer that replaces the existing
- * attribute once all computations are done.
- */
-class OutputAttributePtr {
- private:
-  blender::bke::WriteAttributePtr attribute_;
-
- public:
-  OutputAttributePtr() = default;
-  OutputAttributePtr(blender::bke::WriteAttributePtr attribute);
-  OutputAttributePtr(GeometryComponent &component,
-                     AttributeDomain domain,
-                     std::string name,
-                     CustomDataType data_type);
-
-  ~OutputAttributePtr();
-
-  /* Returns false, when this wrapper is empty. */
-  operator bool() const
-  {
-    return static_cast<bool>(attribute_);
-  }
-
-  /* Get a reference to the underlying #WriteAttribute. */
-  blender::bke::WriteAttribute &get()
-  {
-    BLI_assert(attribute_);
-    return *attribute_;
-  }
-
-  blender::bke::WriteAttribute &operator*()
-  {
-    return *attribute_;
-  }
-
-  blender::bke::WriteAttribute *operator->()
-  {
-    return attribute_.get();
-  }
-
-  void save();
-  void apply_span_and_save();
-};
-
-/**
  * Contains information about an attribute in a geometry component.
  * More information can be added in the future. E.g. whether the attribute is builtin and how it is
  * stored (uv map, vertex group, ...).
@@ -161,21 +107,25 @@ class GeometryComponent {
   /* Can only be used with supported domain types. */
   virtual int attribute_domain_size(const AttributeDomain domain) const;
 
+  bool attribute_is_builtin(const blender::StringRef attribute_name) const;
+
   /* Get read-only access to the highest priority attribute with the given name.
    * Returns null if the attribute does not exist. */
-  blender::bke::ReadAttributePtr attribute_try_get_for_read(
+  blender::bke::ReadAttributeLookup attribute_try_get_for_read(
       const blender::StringRef attribute_name) const;
 
   /* Get read and write access to the highest priority attribute with the given name.
    * Returns null if the attribute does not exist. */
-  blender::bke::WriteAttributePtr attribute_try_get_for_write(
+  blender::bke::WriteAttributeLookup attribute_try_get_for_write(
       const blender::StringRef attribute_name);
 
   /* Get a read-only attribute for the domain based on the given attribute. This can be used to
    * interpolate from one domain to another.
    * Returns null if the interpolation is not implemented. */
-  virtual blender::bke::ReadAttributePtr attribute_try_adapt_domain(
-      blender::bke::ReadAttributePtr attribute, const AttributeDomain new_domain) const;
+  virtual std::unique_ptr<blender::fn::GVArray> attribute_try_adapt_domain(
+      std::unique_ptr<blender::fn::GVArray> varray,
+      const AttributeDomain from_domain,
+      const AttributeDomain to_domain) const;
 
   /* Returns true when the attribute has been deleted. */
   bool attribute_try_delete(const blender::StringRef attribute_name);
@@ -185,80 +135,94 @@ class GeometryComponent {
                             const AttributeDomain domain,
                             const CustomDataType data_type);
 
+  /* Try to create the builtin attribute with the given name. No data type or domain has to be
+   * provided, because those are fixed for builtin attributes. */
+  bool attribute_try_create_builtin(const blender::StringRef attribute_name);
+
   blender::Set<std::string> attribute_names() const;
   bool attribute_foreach(const AttributeForeachCallback callback) const;
 
   virtual bool is_empty() const;
 
-  /* Get a read-only attribute for the given domain and data type.
-   * Returns null when it does not exist. */
-  blender::bke::ReadAttributePtr attribute_try_get_for_read(
+  /* Get a virtual array to read the data of an attribute on the given domain and data type.
+   * Returns null when the attribute does not exist or cannot be converted to the requested domain
+   * and data type. */
+  std::unique_ptr<blender::fn::GVArray> attribute_try_get_for_read(
       const blender::StringRef attribute_name,
       const AttributeDomain domain,
       const CustomDataType data_type) const;
 
-  /* Get a read-only attribute interpolated to the input domain, leaving the data type unchanged.
-   * Returns null when the attribute does not exist. */
-  blender::bke::ReadAttributePtr attribute_try_get_for_read(
+  /* Get a virtual array to read the data of an attribute on the given domain. The data type is
+   * left unchanged. Returns null when the attribute does not exist or cannot be adapted to the
+   * requested domain. */
+  std::unique_ptr<blender::fn::GVArray> attribute_try_get_for_read(
       const blender::StringRef attribute_name, const AttributeDomain domain) const;
 
-  /* Get a read-only attribute for the given domain and data type.
-   * Returns a constant attribute based on the default value if the attribute does not exist.
-   * Never returns null. */
-  blender::bke::ReadAttributePtr attribute_get_for_read(const blender::StringRef attribute_name,
-                                                        const AttributeDomain domain,
-                                                        const CustomDataType data_type,
-                                                        const void *default_value) const;
-
-  /* Get a typed read-only attribute for the given domain and type. */
-  template<typename T>
-  blender::bke::TypedReadAttribute<T> attribute_get_for_read(
+  /* Get a virtual array to read the data of an attribute. If that is not possible, the returned
+   * virtual array will contain a default value. This never returns null. */
+  std::unique_ptr<blender::fn::GVArray> attribute_get_for_read(
       const blender::StringRef attribute_name,
       const AttributeDomain domain,
-      const T &default_value) const
+      const CustomDataType data_type,
+      const void *default_value = nullptr) const;
+
+  /* Should be used instead of the method above when the requested data type is known at compile
+   * time for better type safety. */
+  template<typename T>
+  blender::fn::GVArray_Typed<T> attribute_get_for_read(const blender::StringRef attribute_name,
+                                                       const AttributeDomain domain,
+                                                       const T &default_value) const
   {
     const blender::fn::CPPType &cpp_type = blender::fn::CPPType::get<T>();
     const CustomDataType type = blender::bke::cpp_type_to_custom_data_type(cpp_type);
-    return this->attribute_get_for_read(attribute_name, domain, type, &default_value);
+    std::unique_ptr varray = this->attribute_get_for_read(
+        attribute_name, domain, type, &default_value);
+    return blender::fn::GVArray_Typed<T>(std::move(varray));
   }
 
-  /* Get a read-only dummy attribute that always returns the same value. */
-  blender::bke::ReadAttributePtr attribute_get_constant_for_read(const AttributeDomain domain,
-                                                                 const CustomDataType data_type,
-                                                                 const void *value) const;
+  /**
+   * Returns an "output attribute", which is essentially a mutable virtual array with some commonly
+   * used convience features. The returned output attribute might be empty if requested attribute
+   * cannot exist on the geometry.
+   *
+   * The included convenience features are:
+   * - Implicit type conversion when writing to builtin attributes.
+   * - If the attribute name exists already, but has a different type/domain, a temporary attribute
+   *   is created that will overwrite the existing attribute in the end.
+   */
+  blender::bke::OutputAttribute attribute_try_get_for_output(
+      const blender::StringRef attribute_name,
+      const AttributeDomain domain,
+      const CustomDataType data_type,
+      const void *default_value = nullptr);
 
-  /* Create a read-only dummy attribute that always returns the same value.
-   * The given value is converted to the correct type if necessary. */
-  blender::bke::ReadAttributePtr attribute_get_constant_for_read_converted(
+  /* Same as attribute_try_get_for_output, but should be used when the original values in the
+   * attributes are not read, i.e. the attribute is used only for output. Since values are not read
+   * from this attribute, no default value is necessary. */
+  blender::bke::OutputAttribute attribute_try_get_for_output_only(
+      const blender::StringRef attribute_name,
       const AttributeDomain domain,
-      const CustomDataType in_data_type,
-      const CustomDataType out_data_type,
-      const void *value) const;
+      const CustomDataType data_type);
 
-  /* Get a read-only dummy attribute that always returns the same value. */
+  /* Statically typed method corresponding to the equally named generic one. */
   template<typename T>
-  blender::bke::TypedReadAttribute<T> attribute_get_constant_for_read(const AttributeDomain domain,
-                                                                      const T &value) const
+  blender::bke::OutputAttribute_Typed<T> attribute_try_get_for_output(
+      const blender::StringRef attribute_name, const AttributeDomain domain, const T default_value)
   {
     const blender::fn::CPPType &cpp_type = blender::fn::CPPType::get<T>();
-    const CustomDataType type = blender::bke::cpp_type_to_custom_data_type(cpp_type);
-    return this->attribute_get_constant_for_read(domain, type, &value);
+    const CustomDataType data_type = blender::bke::cpp_type_to_custom_data_type(cpp_type);
+    return this->attribute_try_get_for_output(attribute_name, domain, data_type, &default_value);
   }
 
-  /**
-   * If an attribute with the given params exist, it is returned.
-   * If no attribute with the given name exists, create it and
-   * fill it with the default value if it is provided.
-   * If an attribute with the given name but different domain or type exists, a temporary attribute
-   * is created that has to be saved after the output has been computed. This avoids deleting
-   * another attribute, before a computation is finished.
-   *
-   * This might return no attribute when the attribute cannot exist on the component.
-   */
-  OutputAttributePtr attribute_try_get_for_output(const blender::StringRef attribute_name,
-                                                  const AttributeDomain domain,
-                                                  const CustomDataType data_type,
-                                                  const void *default_value = nullptr);
+  /* Statically typed method corresponding to the equally named generic one. */
+  template<typename T>
+  blender::bke::OutputAttribute_Typed<T> attribute_try_get_for_output_only(
+      const blender::StringRef attribute_name, const AttributeDomain domain)
+  {
+    const blender::fn::CPPType &cpp_type = blender::fn::CPPType::get<T>();
+    const CustomDataType data_type = blender::bke::cpp_type_to_custom_data_type(cpp_type);
+    return this->attribute_try_get_for_output_only(attribute_name, domain, data_type);
+  }
 
  private:
   virtual const blender::bke::ComponentAttributeProviders *get_attribute_providers() const;
@@ -377,8 +341,10 @@ class MeshComponent : public GeometryComponent {
   Mesh *get_for_write();
 
   int attribute_domain_size(const AttributeDomain domain) const final;
-  blender::bke::ReadAttributePtr attribute_try_adapt_domain(
-      blender::bke::ReadAttributePtr attribute, const AttributeDomain new_domain) const final;
+  std::unique_ptr<blender::fn::GVArray> attribute_try_adapt_domain(
+      std::unique_ptr<blender::fn::GVArray> varray,
+      const AttributeDomain from_domain,
+      const AttributeDomain to_domain) const final;
 
   bool is_empty() const final;