Nodes: refactor node tree update handling

Goals of this refactor:
* More unified approach to updating everything that needs to be updated
  after a change in a node tree.
* The updates should happen in the correct order and quadratic or worse
  algorithms should be avoided.
* Improve detection of changes to the output to avoid tagging the depsgraph
  when it's not necessary.
* Move towards a more declarative style of defining nodes by having a
  more centralized update procedure.

The refactor consists of two main parts:
* Node tree tagging and update refactor.
  * Generally, when changes are done to a node tree, it is tagged dirty
    until a global update function is called that updates everything in
    the correct order.
  * The tagging is more fine-grained compared to before, to allow for more
    precise depsgraph update tagging.
* Depsgraph changes.
  * The shading specific depsgraph node for node trees as been removed.
  * Instead, there is a new `NTREE_OUTPUT` depsgrap node, which is only
    tagged when the output of the node tree changed (e.g. the Group Output
    or Material Output node).
  * The copy-on-write relation from node trees to the data block they are
    embedded in is now non-flushing. This avoids e.g. triggering a material
    update after the shader node tree changed in unrelated ways. Instead
    the material has a flushing relation to the new `NTREE_OUTPUT` node now.
  * The depsgraph no longer reports data block changes through to cycles
    through `Depsgraph.updates` when only the node tree changed in ways
    that do not affect the output.

Avoiding unnecessary updates seems to work well for geometry nodes and cycles.
The situation is a bit worse when there are drivers on the node tree, but that
could potentially be improved separately in the future.

Avoiding updates in eevee and the compositor is more tricky, but also less urgent.
* Eevee updates are triggered by calling `DRW_notify_view_update` in
  `ED_render_view3d_update` indirectly from `DEG_editors_update`.
* Compositor updates are triggered by `ED_node_composite_job` in `node_area_refresh`.
  This is triggered by calling `ED_area_tag_refresh` in `node_area_listener`.

Removing updates always has the risk of breaking some dependency that no
one was aware of. It's not unlikely that this will happen here as well. Adding
back missing updates should be quite a bit easier than getting rid of
unnecessary updates though.

Differential Revision: https://developer.blender.org/D13246

1 files changed, 109 insertions, 0 deletions

diff --git a/source/blender/blenkernel/BKE_node_tree_update.h b/source/blender/blenkernel/BKE_node_tree_update.h
new file mode 100644
index 00000000000..ebaa56c89c9
--- /dev/null
+++ b/source/blender/blenkernel/BKE_node_tree_update.h
@@ -0,0 +1,109 @@
+/*
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+ */
+
+#pragma once
+
+/** \file
+ * \ingroup bke
+ */
+
+struct bNode;
+struct bNodeSocket;
+struct bNodeTree;
+struct bNodeLink;
+struct Main;
+struct ID;
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+ * Tag tree as changed without providing any more information about what has changed exactly.
+ * The update process has to assume that everything may have changed.
+ *
+ * Using one of the methods below to tag the tree after changes is preffered when possible.
+ */
+void BKE_ntree_update_tag_all(struct bNodeTree *ntree);
+
+/**
+ * More specialized tag functions that may result in a more efficient update.
+ */
+
+void BKE_ntree_update_tag_node_property(struct bNodeTree *ntree, struct bNode *node);
+void BKE_ntree_update_tag_node_new(struct bNodeTree *ntree, struct bNode *node);
+void BKE_ntree_update_tag_node_removed(struct bNodeTree *ntree);
+void BKE_ntree_update_tag_node_internal_link(struct bNodeTree *ntree, struct bNode *node);
+
+void BKE_ntree_update_tag_socket_property(struct bNodeTree *ntree, struct bNodeSocket *socket);
+void BKE_ntree_update_tag_socket_new(struct bNodeTree *ntree, struct bNodeSocket *socket);
+void BKE_ntree_update_tag_socket_type(struct bNodeTree *ntree, struct bNodeSocket *socket);
+void BKE_ntree_update_tag_socket_availability(struct bNodeTree *ntree, struct bNodeSocket *socket);
+void BKE_ntree_update_tag_socket_removed(struct bNodeTree *ntree);
+
+void BKE_ntree_update_tag_link_changed(struct bNodeTree *ntree);
+void BKE_ntree_update_tag_link_removed(struct bNodeTree *ntree);
+void BKE_ntree_update_tag_link_added(struct bNodeTree *ntree, struct bNodeLink *link);
+void BKE_ntree_update_tag_link_mute(struct bNodeTree *ntree, struct bNodeLink *link);
+
+/** Used after file loading when run-time data on the tree has not been initialized yet. */
+void BKE_ntree_update_tag_missing_runtime_data(struct bNodeTree *ntree);
+/** Used when the interface sockets/values have changed. */
+void BKE_ntree_update_tag_interface(struct bNodeTree *ntree);
+/** Used when an id data block changed that might be used by nodes that need to be updated. */
+void BKE_ntree_update_tag_id_changed(struct Main *bmain, struct ID *id);
+
+typedef struct NodeTreeUpdateExtraParams {
+  /**
+   * Data passed into the callbacks.
+   */
+  void *user_data;
+
+  /**
+   * Called for every tree that has been changed during the update. This can be used to send
+   * notifiers to trigger redraws or depsgraph updates.
+   */
+  void (*tree_changed_fn)(struct ID *, struct bNodeTree *, void *user_data);
+
+  /**
+   * Called for every tree whose output value may have changed based on the provided update tags.
+   * This can be used to tag the depsgraph if necessary.
+   */
+  void (*tree_output_changed_fn)(struct ID *, struct bNodeTree *, void *user_data);
+} NodeTreeUpdateExtraParams;
+
+/**
+ * Updates #bmain based on changes to node trees.
+ */
+void BKE_ntree_update_main(struct Main *bmain, struct NodeTreeUpdateExtraParams *params);
+
+/**
+ * Same as #BKE_ntree_update_main, but will first only look at the provided tree and only looks
+ * at #bmain when something relevant for other data-blocks changed. This avoids scanning #bmain in
+ * many cases.
+ *
+ * If #bmain is null, only the provided tree is updated. This should only be used in very rare
+ * cases because it may result it incorrectly synced data in DNA.
+ *
+ * If #tree is null, this is the same as calling #BKE_ntree_update_main.
+ */
+void BKE_ntree_update_main_tree(struct Main *bmain,
+                                struct bNodeTree *ntree,
+                                struct NodeTreeUpdateExtraParams *params);
+
+#ifdef __cplusplus
+}
+#endif