readfile: reduce memory usage at load time

Delay loading all DATA sections of the blend file until they're needed.

Loading all data-blocks caused high peak memory usage especially with
libraries - since a lot of data may exist which isn't used directly.

In one test (spring project: 10_010_A.anim.blend),
peaked at ~12.5gig, dropping back to ~2.5gig once loaded.
With this change peaks memory usage reaches ~2.7gig while loading.

Besides this there are some minor gains from not having to read data
from the file-system and we can skip an alloc + memcpy reading data
written with the same version of Blender.

1 files changed, 1 insertions, 0 deletions

diff --git a/source/blender/blenloader/intern/readfile.h b/source/blender/blenloader/intern/readfile.h
index 91dfae4a71d..167d5511a0c 100644
--- a/source/blender/blenloader/intern/readfile.h
+++ b/source/blender/blenloader/intern/readfile.h
@@ -56,6 +56,7 @@ typedef struct FileData {
 	int buffersize;
 	int file_offset;
 	int (*read)(struct FileData *filedata, void *buffer, unsigned int size);
+	int (*seek)(struct FileData *filedata, int offset, int whence);
 
 	/** Variables needed for reading from memory / stream. */
 	const char *buffer;