DNA: optimize struct reconstruction by doing some preprocessing

When loading large files that are more than a couple weeks old
(such that DNA has changed in that time), a significant amount of
time is spent in `DNA_struct_reconstruct`. This function takes a struct
in the old layout and creates a struct in the new layout from it.

This was slow because it was computing the diff between the struct
layouts every time a struct is updated. Now the steps for the struct
reconstruction is computed only once per struct. This information is
then used to actually reconstruct all structs that changed.

I measured about 10-20% speedup when loading Spring files.
E.g. `10.6s -> 8.7s` for `06_055_A.anim.blend` in BKE_blendfile_read`.

This percentage varies a lot based on the number of blocks that have
to be reconstructed and how much DNA has changed since they have
been written. In none of my tests was the new code slower than the
old code.

Reviewers: campbellbarton

Differential Revision: https://developer.blender.org/D8959

1 files changed, 9 insertions, 3 deletions

diff --git a/source/blender/makesdna/DNA_genfile.h b/source/blender/makesdna/DNA_genfile.h
index 1cdaba81ffa..b09b6dcd8e4 100644
--- a/source/blender/makesdna/DNA_genfile.h
+++ b/source/blender/makesdna/DNA_genfile.h
@@ -78,6 +78,8 @@ enum eSDNA_StructCompare {
   /* Struct is different in some way
    * (needs to be copied/converted field by field) */
   SDNA_CMP_NOT_EQUAL = 2,
+  /* This is only used temporarily by #DNA_struct_get_compareflags. */
+  SDNA_CMP_UNKNOWN = 3,
 };
 
 struct SDNA *DNA_sdna_from_data(const void *data,
@@ -93,13 +95,17 @@ void DNA_sdna_current_init(void);
 const struct SDNA *DNA_sdna_current_get(void);
 void DNA_sdna_current_free(void);
 
+struct DNA_ReconstructInfo;
+struct DNA_ReconstructInfo *DNA_reconstruct_info_create(const struct SDNA *oldsdna,
+                                                        const struct SDNA *newsdna,
+                                                        const char *compare_flags);
+void DNA_reconstruct_info_free(struct DNA_ReconstructInfo *reconstruct_info);
+
 int DNA_struct_find_nr_ex(const struct SDNA *sdna, const char *str, unsigned int *index_last);
 int DNA_struct_find_nr(const struct SDNA *sdna, const char *str);
 void DNA_struct_switch_endian(const struct SDNA *oldsdna, int oldSDNAnr, char *data);
 const char *DNA_struct_get_compareflags(const struct SDNA *sdna, const struct SDNA *newsdna);
-void *DNA_struct_reconstruct(const struct SDNA *newsdna,
-                             const struct SDNA *oldsdna,
-                             const char *compflags,
+void *DNA_struct_reconstruct(const struct DNA_ReconstructInfo *reconstruct_info,
                              int oldSDNAnr,
                              int blocks,
                              const void *data);