OpenColorIO: upgrade to version 2.0.0

Ref T84819

Build System
============

This is an API breaking new version, and the updated code only builds with
OpenColorIO 2.0 and later. Adding backwards compatibility was too complicated.

* Tinyxml was replaced with Expat, adding a new dependency.
* Yaml-cpp is now built as a dependency on Unix, as was already done on Windows.
* Removed currently unused LCMS code.
* Pystring remains built as part of OCIO itself, since it has no good build system.
* Linux and macOS check for the OpenColorIO verison, and disable it if too old.

Ref D10270

Processors and Transforms
=========================

CPU processors now need to be created to do CPU processing. These are cached
internally, but the cache lookup is not fast enough to execute per pixel or
texture sample, so for performance these are now also exposed in the C API.

The C API for transforms will no longer be needed afer all changes, so remove
it to simplify the API and fallback implementation.

Ref D10271

Display Transforms
==================

Needs a bit more manual work constructing the transform. LegacyViewingPipeline
could also have been used, but isn't really any simpler and since it's legacy
we better not rely on it.

We moved more logic into the opencolorio module, to simplify the API. There is
no need to wrap a dozen functions just to be able to do this in C rather than C++.
It's also tightly coupled to the GPU shader logic, and so should be in the same
module.

Ref D10271

GPU Display Shader
==================

To avoid baking exposure and gamma into the GLSL shader and requiring slow
recompiles when tweaking, we manually apply them in the shader. This leads
to some logic duplicaton between the CPU and GPU display processor, but it
seems unavoidable.

Caching was also changed. Previously this was done both on the imbuf and
opencolorio module levels. Now it's all done in the opencolorio module by
simply matching color space names. We no longer use cacheIDs from OpenColorIO
since computing them is expensive, and they are unlikely to match now that
more is baked into the shader code.

Shaders can now use multiple 2D textures, 3D textures and uniforms, rather
than a single 3D texture. So allocating and binding those adds some code.

Color space conversions for blending with overlays is now hardcoded in the
shader. This was using harcoded numbers anyway, if this every becomes a
general OpenColorIO transform it can be changed, but for now there is no
point to add code complexity.

Ref D10273

CIE XYZ
=======

We need standard CIE XYZ values for rendering effects like blackbody emission.
The relation to the scene linear role is based on OpenColorIO configuration.

In OpenColorIO 2.0 configs roles can no longer have the same name as color
spaces, which means our XYZ role and colorspace in the configuration give an
error.

Instead use the new standard aces_interchange role, which relates scene linear
to a known scene referred color space. Compatibility with the old XYZ role is
preserved, if the configuration file has no conflicting names.

Also includes a non-functional change to the configuraton file to use an
XYZ-to-ACES matrix instead of REC709-to-ACES, makes debugging a little easier
since the matrix is the same one we have in the code now and that is also
found easily in the ACES specs.

Ref D10274

4 files changed, 104 insertions, 42 deletions

diff --git a/intern/cycles/render/CMakeLists.txt b/intern/cycles/render/CMakeLists.txt
index 389f913b145..c67919b375a 100644
--- a/intern/cycles/render/CMakeLists.txt
+++ b/intern/cycles/render/CMakeLists.txt
@@ -133,8 +133,11 @@ if(WITH_OPENCOLORIO)
     SYSTEM
     ${OPENCOLORIO_INCLUDE_DIRS}
   )
+  list(APPEND LIB
+    ${OPENCOLORIO_LIBRARIES}
+  )
   if(WIN32)
-    add_definitions(-DOpenColorIO_STATIC)
+    add_definitions(-DOpenColorIO_SKIP_IMPORTS)
   endif()
 endif()
 
diff --git a/intern/cycles/render/colorspace.cpp b/intern/cycles/render/colorspace.cpp
index 4c9e86ea278..4540793f78d 100644
--- a/intern/cycles/render/colorspace.cpp
+++ b/intern/cycles/render/colorspace.cpp
@@ -192,6 +192,7 @@ void ColorSpaceManager::is_builtin_colorspace(ustring colorspace,
     return;
   }
 
+  OCIO::ConstCPUProcessorRcPtr device_processor = processor->getDefaultCPUProcessor();
   is_scene_linear = true;
   is_srgb = true;
   for (int i = 0; i < 256; i++) {
@@ -201,10 +202,10 @@ void ColorSpaceManager::is_builtin_colorspace(ustring colorspace,
     float cG[3] = {0, v, 0};
     float cB[3] = {0, 0, v};
     float cW[3] = {v, v, v};
-    processor->applyRGB(cR);
-    processor->applyRGB(cG);
-    processor->applyRGB(cB);
-    processor->applyRGB(cW);
+    device_processor->applyRGB(cR);
+    device_processor->applyRGB(cG);
+    device_processor->applyRGB(cB);
+    device_processor->applyRGB(cW);
 
     /* Make sure that there is no channel crosstalk. */
     if (fabsf(cR[1]) > 1e-5f || fabsf(cR[2]) > 1e-5f || fabsf(cG[0]) > 1e-5f ||
@@ -267,6 +268,7 @@ inline void processor_apply_pixels(const OCIO::Processor *processor, T *pixels,
   /* TODO: implement faster version for when we know the conversion
    * is a simple matrix transform between linear spaces. In that case
    * un-premultiply is not needed. */
+  OCIO::ConstCPUProcessorRcPtr device_processor = processor->getDefaultCPUProcessor();
 
   /* Process large images in chunks to keep temporary memory requirement down. */
   const size_t chunk_size = std::min((size_t)(16 * 1024 * 1024), num_pixels);
@@ -289,7 +291,7 @@ inline void processor_apply_pixels(const OCIO::Processor *processor, T *pixels,
     }
 
     OCIO::PackedImageDesc desc((float *)float_pixels.data(), width, 1, 4);
-    processor->apply(desc);
+    device_processor->apply(desc);
 
     for (size_t i = 0; i < width; i++) {
       float4 value = float_pixels[i];
@@ -345,13 +347,14 @@ void ColorSpaceManager::to_scene_linear(ColorSpaceProcessor *processor_,
   const OCIO::Processor *processor = (const OCIO::Processor *)processor_;
 
   if (processor) {
+    OCIO::ConstCPUProcessorRcPtr device_processor = processor->getDefaultCPUProcessor();
     if (channels == 3) {
-      processor->applyRGB(pixel);
+      device_processor->applyRGB(pixel);
     }
     else if (channels == 4) {
       if (pixel[3] == 1.0f || pixel[3] == 0.0f) {
         /* Fast path for RGBA. */
-        processor->applyRGB(pixel);
+        device_processor->applyRGB(pixel);
       }
       else {
         /* Un-associate and associate alpha since color management should not
@@ -363,7 +366,7 @@ void ColorSpaceManager::to_scene_linear(ColorSpaceProcessor *processor_,
         pixel[1] *= inv_alpha;
         pixel[2] *= inv_alpha;
 
-        processor->applyRGB(pixel);
+        device_processor->applyRGB(pixel);
 
         pixel[0] *= alpha;
         pixel[1] *= alpha;
diff --git a/intern/cycles/render/shader.cpp b/intern/cycles/render/shader.cpp
index 332599be708..650587cb694 100644
--- a/intern/cycles/render/shader.cpp
+++ b/intern/cycles/render/shader.cpp
@@ -35,6 +35,7 @@
 #include "util/util_foreach.h"
 #include "util/util_murmurhash.h"
 #include "util/util_task.h"
+#include "util/util_transform.h"
 
 #ifdef WITH_OCIO
 #  include <OpenColorIO/OpenColorIO.h>
@@ -399,39 +400,7 @@ ShaderManager::ShaderManager()
   update_flags = UPDATE_ALL;
   beckmann_table_offset = TABLE_OFFSET_INVALID;
 
-  xyz_to_r = make_float3(3.2404542f, -1.5371385f, -0.4985314f);
-  xyz_to_g = make_float3(-0.9692660f, 1.8760108f, 0.0415560f);
-  xyz_to_b = make_float3(0.0556434f, -0.2040259f, 1.0572252f);
-  rgb_to_y = make_float3(0.2126729f, 0.7151522f, 0.0721750f);
-
-#ifdef WITH_OCIO
-  OCIO::ConstConfigRcPtr config = OCIO::GetCurrentConfig();
-  if (config) {
-    if (config->hasRole("XYZ") && config->hasRole("scene_linear")) {
-      OCIO::ConstProcessorRcPtr to_rgb_processor = config->getProcessor("XYZ", "scene_linear");
-      OCIO::ConstProcessorRcPtr to_xyz_processor = config->getProcessor("scene_linear", "XYZ");
-      if (to_rgb_processor && to_xyz_processor) {
-        float r[] = {1.0f, 0.0f, 0.0f};
-        float g[] = {0.0f, 1.0f, 0.0f};
-        float b[] = {0.0f, 0.0f, 1.0f};
-        to_xyz_processor->applyRGB(r);
-        to_xyz_processor->applyRGB(g);
-        to_xyz_processor->applyRGB(b);
-        rgb_to_y = make_float3(r[1], g[1], b[1]);
-
-        float x[] = {1.0f, 0.0f, 0.0f};
-        float y[] = {0.0f, 1.0f, 0.0f};
-        float z[] = {0.0f, 0.0f, 1.0f};
-        to_rgb_processor->applyRGB(x);
-        to_rgb_processor->applyRGB(y);
-        to_rgb_processor->applyRGB(z);
-        xyz_to_r = make_float3(x[0], y[0], z[0]);
-        xyz_to_g = make_float3(x[1], y[1], z[1]);
-        xyz_to_b = make_float3(x[2], y[2], z[2]);
-      }
-    }
-  }
-#endif
+  init_xyz_transforms();
 }
 
 ShaderManager::~ShaderManager()
@@ -829,4 +798,89 @@ bool ShaderManager::need_update() const
   return update_flags != UPDATE_NONE;
 }
 
+#ifdef WITH_OCIO
+static bool to_scene_linear_transform(OCIO::ConstConfigRcPtr &config,
+                                      const char *colorspace,
+                                      Transform &to_scene_linear)
+{
+  OCIO::ConstProcessorRcPtr processor;
+  try {
+    processor = config->getProcessor(OCIO::ROLE_SCENE_LINEAR, colorspace);
+  }
+  catch (OCIO::Exception &exception) {
+    return false;
+  }
+
+  if (!processor) {
+    return false;
+  }
+
+  OCIO::ConstCPUProcessorRcPtr device_processor = processor->getDefaultCPUProcessor();
+  if (!device_processor) {
+    return false;
+  }
+
+  to_scene_linear = transform_identity();
+  device_processor->applyRGB(&to_scene_linear.x.x);
+  device_processor->applyRGB(&to_scene_linear.y.x);
+  device_processor->applyRGB(&to_scene_linear.z.x);
+  to_scene_linear = transform_transposed_inverse(to_scene_linear);
+  return true;
+}
+#endif
+
+void ShaderManager::init_xyz_transforms()
+{
+  /* Default to ITU-BT.709 in case no appropriate transform found. */
+  xyz_to_r = make_float3(3.2404542f, -1.5371385f, -0.4985314f);
+  xyz_to_g = make_float3(-0.9692660f, 1.8760108f, 0.0415560f);
+  xyz_to_b = make_float3(0.0556434f, -0.2040259f, 1.0572252f);
+  rgb_to_y = make_float3(0.2126729f, 0.7151522f, 0.0721750f);
+
+#ifdef WITH_OCIO
+  /* Get from OpenColorO config if it has the required roles. */
+  OCIO::ConstConfigRcPtr config = OCIO::GetCurrentConfig();
+  if (!(config && config->hasRole(OCIO::ROLE_SCENE_LINEAR))) {
+    return;
+  }
+
+  Transform xyz_to_rgb;
+
+  if (config->hasRole("aces_interchange")) {
+    /* Standard OpenColorIO role, defined as ACES2065-1. */
+    const Transform xyz_to_aces = make_transform(1.0498110175f,
+                                                 0.0f,
+                                                 -0.0000974845f,
+                                                 0.0f,
+                                                 -0.4959030231f,
+                                                 1.3733130458f,
+                                                 0.0982400361f,
+                                                 0.0f,
+                                                 0.0f,
+                                                 0.0f,
+                                                 0.9912520182f,
+                                                 0.0f);
+    Transform aces_to_rgb;
+    if (!to_scene_linear_transform(config, "aces_interchange", aces_to_rgb)) {
+      return;
+    }
+
+    xyz_to_rgb = aces_to_rgb * xyz_to_aces;
+  }
+  else if (config->hasRole("XYZ")) {
+    /* Custom role used before the standard existed. */
+    if (!to_scene_linear_transform(config, "XYZ", xyz_to_rgb)) {
+      return;
+    }
+  }
+
+  xyz_to_r = float4_to_float3(xyz_to_rgb.x);
+  xyz_to_g = float4_to_float3(xyz_to_rgb.y);
+  xyz_to_b = float4_to_float3(xyz_to_rgb.z);
+
+  const Transform rgb_to_xyz = transform_inverse(xyz_to_rgb);
+  rgb_to_y = float4_to_float3(rgb_to_xyz.y);
+#endif
+}
+
 CCL_NAMESPACE_END
diff --git a/intern/cycles/render/shader.h b/intern/cycles/render/shader.h
index 4375ef9e978..f47d64f346c 100644
--- a/intern/cycles/render/shader.h
+++ b/intern/cycles/render/shader.h
@@ -221,6 +221,8 @@ class ShaderManager {
 
   bool need_update() const;
 
+  void init_xyz_transforms();
+
  protected:
   ShaderManager();