Speed up the new OBJ exporter via bigger write buffer and parallelization.

This is a patch from Aras Pranckevicius, D13927. See that patch for full
details. On Windows, the many small fprintfs were taking up a large amount
of time and significant speedup comes from using snprintf into chained buffers,
and writing them all out later.
On both Windows and Linux, parallelizing the processing by Object can also lead
to a significant increase in speed.
The 3.0 splash screen scene exports 8 times faster than the current C++ exporter
on a Windows machine with 32 threads, and 5.8 times faster on a Linux machine
with 48 threads.

There is admittedly more memory usage for this, but it is still using 25 times
less memory than the old python exporter on the 3.0 splash screen scene, so
this seems an acceptable tradeoff. If use cases come up for exporting obj files
that exceed the memory size of users, a flag could be added to not parallelize
and write the buffers out every so often.

1 files changed, 33 insertions, 1 deletions

diff --git a/source/blender/io/wavefront_obj/tests/obj_exporter_tests.cc b/source/blender/io/wavefront_obj/tests/obj_exporter_tests.cc
index 1e2d509ef29..ac2ee8d566b 100644
--- a/source/blender/io/wavefront_obj/tests/obj_exporter_tests.cc
+++ b/source/blender/io/wavefront_obj/tests/obj_exporter_tests.cc
@@ -193,7 +193,7 @@ static std::string read_temp_file_in_string(const std::string &file_path)
   std::string res;
   size_t buffer_len;
   void *buffer = BLI_file_read_text_as_mem(file_path.c_str(), 0, &buffer_len);
-  if (buffer != NULL) {
+  if (buffer != nullptr) {
     res.assign((const char *)buffer, buffer_len);
     MEM_freeN(buffer);
   }
@@ -238,6 +238,38 @@ TEST(obj_exporter_writer, mtllib)
   BLI_delete(out_file_path.c_str(), false, false);
 }
 
+TEST(obj_exporter_writer, format_handler_buffer_chunking)
+{
+  /* Use a tiny buffer chunk size, so that the test below ends up creating several blocks. */
+  FormatHandler<eFileType::OBJ, 16, 8> h;
+  h.write<eOBJSyntaxElement::object_name>("abc");
+  h.write<eOBJSyntaxElement::object_name>("abcd");
+  h.write<eOBJSyntaxElement::object_name>("abcde");
+  h.write<eOBJSyntaxElement::object_name>("abcdef");
+  h.write<eOBJSyntaxElement::object_name>("012345678901234567890123456789abcd");
+  h.write<eOBJSyntaxElement::object_name>("123");
+  h.write<eOBJSyntaxElement::curve_element_begin>();
+  h.write<eOBJSyntaxElement::new_line>();
+  h.write<eOBJSyntaxElement::nurbs_parameter_begin>();
+  h.write<eOBJSyntaxElement::new_line>();
+
+  size_t got_blocks = h.get_block_count();
+  ASSERT_EQ(got_blocks, 7);
+
+  std::string got_string = h.get_as_string();
+  using namespace std::string_literals;
+  const char *expected = R"(o abc
+o abcd
+o abcde
+o abcdef
+o 012345678901234567890123456789abcd
+o 123
+curv 0.0 1.0
+parm 0.0
+)";
+  ASSERT_EQ(got_string, expected);
+}
+
 /* Return true if string #a and string #b are equal after their first newline. */
 static bool strings_equal_after_first_lines(const std::string &a, const std::string &b)
 {