Nodes: refactor node tree update handling

Goals of this refactor:
* More unified approach to updating everything that needs to be updated
  after a change in a node tree.
* The updates should happen in the correct order and quadratic or worse
  algorithms should be avoided.
* Improve detection of changes to the output to avoid tagging the depsgraph
  when it's not necessary.
* Move towards a more declarative style of defining nodes by having a
  more centralized update procedure.

The refactor consists of two main parts:
* Node tree tagging and update refactor.
  * Generally, when changes are done to a node tree, it is tagged dirty
    until a global update function is called that updates everything in
    the correct order.
  * The tagging is more fine-grained compared to before, to allow for more
    precise depsgraph update tagging.
* Depsgraph changes.
  * The shading specific depsgraph node for node trees as been removed.
  * Instead, there is a new `NTREE_OUTPUT` depsgrap node, which is only
    tagged when the output of the node tree changed (e.g. the Group Output
    or Material Output node).
  * The copy-on-write relation from node trees to the data block they are
    embedded in is now non-flushing. This avoids e.g. triggering a material
    update after the shader node tree changed in unrelated ways. Instead
    the material has a flushing relation to the new `NTREE_OUTPUT` node now.
  * The depsgraph no longer reports data block changes through to cycles
    through `Depsgraph.updates` when only the node tree changed in ways
    that do not affect the output.

Avoiding unnecessary updates seems to work well for geometry nodes and cycles.
The situation is a bit worse when there are drivers on the node tree, but that
could potentially be improved separately in the future.

Avoiding updates in eevee and the compositor is more tricky, but also less urgent.
* Eevee updates are triggered by calling `DRW_notify_view_update` in
  `ED_render_view3d_update` indirectly from `DEG_editors_update`.
* Compositor updates are triggered by `ED_node_composite_job` in `node_area_refresh`.
  This is triggered by calling `ED_area_tag_refresh` in `node_area_listener`.

Removing updates always has the risk of breaking some dependency that no
one was aware of. It's not unlikely that this will happen here as well. Adding
back missing updates should be quite a bit easier than getting rid of
unnecessary updates though.

Differential Revision: https://developer.blender.org/D13246

1 files changed, 20 insertions, 25 deletions

diff --git a/source/blender/makesdna/DNA_node_types.h b/source/blender/makesdna/DNA_node_types.h
index f4a1efe84b8..5d51d8eb606 100644
--- a/source/blender/makesdna/DNA_node_types.h
+++ b/source/blender/makesdna/DNA_node_types.h
@@ -159,7 +159,7 @@ typedef struct bNodeSocket {
    * restores pointer from matching own_index. */
   struct bNodeSocket *groupsock DNA_DEPRECATED;
 
-  /** A link pointer, set in ntreeUpdateTree. */
+  /** A link pointer, set in #BKE_ntree_update_main. */
   struct bNodeLink *link;
 
   /* XXX deprecated, socket input values are stored in default_value now.
@@ -172,6 +172,10 @@ typedef struct bNodeSocket {
    * data. It has to be updated when the node declaration changes.
    */
   const SocketDeclarationHandle *declaration;
+
+  /** #eNodeTreeChangedFlag. */
+  uint32_t changed_flag;
+  char _pad[4];
 } bNodeSocket;
 
 /** #bNodeSocket.type & #bNodeSocketType.type */
@@ -260,8 +264,9 @@ typedef struct bNode {
 
   /** Used as a boolean for execution. */
   uint8_t need_exec;
-
-  char _pad[1];
+  char _pad2[5];
+  /** #eNodeTreeChangedFlag. */
+  uint32_t changed_flag;
 
   /** Custom user-defined color. */
   float color[3];
@@ -400,10 +405,6 @@ typedef struct bNode {
 #define __NODE_ACTIVE_PREVIEW (1 << 18) /* deprecated */
 
 /* node->update */
-/* XXX NODE_UPDATE is a generic update flag. More fine-grained updates
- * might be used in the future, but currently all work the same way.
- */
-#define NODE_UPDATE 0xFFFF     /* generic update flag (includes all others) */
 #define NODE_UPDATE_ID 1       /* associated id data block has changed */
 #define NODE_UPDATE_OPERATOR 2 /* node update triggered from update operator */
 
@@ -511,8 +512,12 @@ typedef struct bNodeTree {
    */
   int cur_index;
   int flag;
-  /** Update flags. */
-  int update;
+  /**
+   * Keeps track of what changed in the node tree until the next update.
+   * Should not be changed directly, instead use the functions in `BKE_node_tree_update.h`.
+   * #eNodeTreeChangedFlag.
+   */
+  uint32_t changed_flag;
   /** Flag to prevent re-entrant update calls. */
   short is_updating;
   /** Generic temporary flag for recursion check (DFS/BFS). */
@@ -546,7 +551,11 @@ typedef struct bNodeTree {
    * in case multiple different editors are used and make context ambiguous.
    */
   bNodeInstanceKey active_viewer_key;
-  char _pad[4];
+  /**
+   * A hash of the topology of the node tree leading up to the outputs. This is used to determine
+   * of the node tree changed in a way that requires updating geometry nodes or shaders.
+   */
+  uint32_t output_topology_hash;
 
   /** Execution data.
    *
@@ -594,21 +603,7 @@ typedef struct bNodeTree {
 /* tree is localized copy, free when deleting node groups */
 /* #define NTREE_IS_LOCALIZED           (1 << 5) */
 
-/** #NodeTree.update */
-typedef enum eNodeTreeUpdate {
-  NTREE_UPDATE = 0xFFFF,             /* generic update flag (includes all others) */
-  NTREE_UPDATE_LINKS = (1 << 0),     /* links have been added or removed */
-  NTREE_UPDATE_NODES = (1 << 1),     /* nodes or sockets have been added or removed */
-  NTREE_UPDATE_GROUP_IN = (1 << 4),  /* group inputs have changed */
-  NTREE_UPDATE_GROUP_OUT = (1 << 5), /* group outputs have changed */
-  /* The field interface has changed. So e.g. an output that was always a field before is not
-   * anymore. This implies that the field type inferencing has to be done again. */
-  NTREE_UPDATE_FIELD_INFERENCING = (1 << 6),
-  /* group has changed (generic flag including all other group flags) */
-  NTREE_UPDATE_GROUP = (NTREE_UPDATE_GROUP_IN | NTREE_UPDATE_GROUP_OUT),
-} eNodeTreeUpdate;
-
-/** #NodeTree.execution_mode */
+/* tree->execution_mode */
 typedef enum eNodeTreeExecutionMode {
   NTREE_EXECUTION_MODE_TILED = 0,
   NTREE_EXECUTION_MODE_FULL_FRAME = 1,