Cycles: Add Hydra render delegate

This patch adds a [Hydra](https://graphics.pixar.com/usd/release/glossary.html#usdglossary-hydra) render delegate to Cycles, allowing Cycles to be used for rendering in applications that provide a Hydra viewport (e.g. USDView or NVIDIA Omniverse Kit). The implementation was written from scratch against Cycles X, for integration into the Blender repository to make it possible to continue developing it in step with the rest of Cycles. For this purpose it follows the style of the rest of the Cycles code and can be built with a CMake option (`WITH_CYCLES_HYDRA_RENDER_DELEGATE=1`) similar to the existing standalone version of Cycles.

Supported features:
- CPU/CUDA/OptiX/HIP/Metal support
- Camera Settings
- Render Settings (automatically queried from Cycles via node type system)
- Basic AOVs (color, depth, normal, primId, instanceId)
- Lights (Disk, Distant, Dome, Rect, Sphere)
- Meshes
- Geom Subsets
- Subdivision Surfaces (using native Cycles support)
- Custom Primvars (converted to Cycles attributes)
- Cycles Materials (can be exported to USD using the [universal-scene-description branch of Blender](https://developer.blender.org/diffusion/B/history/universal-scene-description/))
- USD Preview Surface Materials
- Curves
- Point Clouds
- OpenVDB Volumes

Still missing features:
- Motion Blur
- Custom AOVs
- ...

Since Hydra render delegates need to be built against the exact USD version and other dependencies as the target application is using, this is intended to be built separate from Blender (`WITH_BLENDER=0` CMake option) and with support for library versions different from what Blender is using. As such the CMake build scripts for Windows had to be modified slightly, so that the Cycles Hydra render delegate can e.g. be built with MSVC 2017 again even though Blender requires MSVC 2019 now, and it's possible to specify custom paths to the USD SDK etc. The codebase supports building against the latest USD release 22.03 and all the way back to USD 20.08 (with some limitations).

This also includes an optimization for Hydra viewports that display the result using OpenGL, in which case the texture can be kept entirely on the GPU (see display_driver.cpp). Unfortunately this is a bit difficult since Hydra doesn't give any control over the OpenGL context created by an application, so the only way to make it available to Cycles (which is rendering on a separate thread) without disturbing the target application is to create a second OpenGL context that is sharing resources with the primary one. The USD SDK doesn't provide an API to do so, and can't use Blenders Ghost since the render delegate is built without Blender, so have to create it manualy. That is only implemented for Windows right now, so on Linux the optimization is disabled (see `HdCyclesDelegate::IsDisplaySupported()`).

---

**To build:**

1. [Set up a Blender build environment](https://wiki.blender.org/wiki/Building_Blender) as usual but download and apply this patch to the Git repository (Download Raw Diff on the right via `Save Link As` and then run `git apply patch.diff` with the downloaded file in your local repository after syncing to latest master branch).
2. Set these CMake variables:
```
WITH_BLENDER=0
WITH_CYCLES_HYDRA_RENDER_DELEGATE=1
USD_INCLUDE_DIRS=<path to your USD build>/include
USD_LIBRARY_DIR=<path to your USD build>/lib
USD_LIBRARY_PREFIX=<optional, if e.g. USD libs are called "usd_hd.lib", set to "usd_">
```
3. Continue following the usual Blender build instructions. After building the INSTALL target, the output directory contains the `hdCycles` shared library and associated resource files which can be loaded as a USD plugin.

**To execute:**

4. Copy `hdCycles.dll`/`hdCycles.a` and the `hdCycles` directory from the output directory to the USD plugin directory of the target application, or point a `PXR_PLUGINPATH_NAME` environment variable to the output directory.
5. Launch the target application, it should now automatically detect the Cycles Hydra render delegate.

Differential Revision: https://developer.blender.org/D14398

1 files changed, 297 insertions, 0 deletions

diff --git a/intern/cycles/hydra/camera.cpp b/intern/cycles/hydra/camera.cpp
new file mode 100644
index 00000000000..05f1c32d3c4
--- /dev/null
+++ b/intern/cycles/hydra/camera.cpp
@@ -0,0 +1,297 @@
+/* SPDX-License-Identifier: Apache-2.0
+ * Copyright 2022 NVIDIA Corporation
+ * Copyright 2022 Blender Foundation */
+
+#include "hydra/camera.h"
+#include "scene/camera.h"
+
+#include <pxr/base/gf/frustum.h>
+#include <pxr/imaging/hd/sceneDelegate.h>
+#include <pxr/usd/usdGeom/tokens.h>
+
+HDCYCLES_NAMESPACE_OPEN_SCOPE
+
+extern Transform convert_transform(const GfMatrix4d &matrix);
+
+HdCyclesCamera::HdCyclesCamera(const SdfPath &sprimId) : HdCamera(sprimId)
+{
+#if PXR_VERSION >= 2102
+  // Synchronize default values
+  _horizontalAperture = _data.GetHorizontalAperture() * GfCamera::APERTURE_UNIT;
+  _verticalAperture = _data.GetVerticalAperture() * GfCamera::APERTURE_UNIT;
+  _horizontalApertureOffset = _data.GetHorizontalApertureOffset() * GfCamera::APERTURE_UNIT;
+  _verticalApertureOffset = _data.GetVerticalApertureOffset() * GfCamera::APERTURE_UNIT;
+  _focalLength = _data.GetFocalLength() * GfCamera::FOCAL_LENGTH_UNIT;
+  _clippingRange = _data.GetClippingRange();
+  _fStop = _data.GetFStop();
+  _focusDistance = _data.GetFocusDistance();
+#endif
+}
+
+HdCyclesCamera::~HdCyclesCamera()
+{
+}
+
+HdDirtyBits HdCyclesCamera::GetInitialDirtyBitsMask() const
+{
+  return DirtyBits::AllDirty;
+}
+
+void HdCyclesCamera::Sync(HdSceneDelegate *sceneDelegate,
+                          HdRenderParam *renderParam,
+                          HdDirtyBits *dirtyBits)
+{
+  if (*dirtyBits == DirtyBits::Clean) {
+    return;
+  }
+
+  VtValue value;
+  const SdfPath &id = GetId();
+
+#if PXR_VERSION >= 2102
+  if (*dirtyBits & DirtyBits::DirtyTransform) {
+    sceneDelegate->SampleTransform(id, &_transformSamples);
+
+    for (size_t i = 0; i < _transformSamples.count; ++i) {
+      if (_transformSamples.times[i] == 0.0f) {
+        _transform = _transformSamples.values[i];
+        _data.SetTransform(_transform);
+        break;
+      }
+    }
+  }
+#else
+  if (*dirtyBits & DirtyBits::DirtyViewMatrix) {
+    sceneDelegate->SampleTransform(id, &_transformSamples);
+
+    value = sceneDelegate->GetCameraParamValue(id, HdCameraTokens->worldToViewMatrix);
+    if (!value.IsEmpty()) {
+      _worldToViewMatrix = value.Get<GfMatrix4d>();
+      _worldToViewInverseMatrix = _worldToViewMatrix.GetInverse();
+      _data.SetTransform(_worldToViewInverseMatrix);
+    }
+  }
+#endif
+
+  if (*dirtyBits & DirtyBits::DirtyProjMatrix) {
+    value = sceneDelegate->GetCameraParamValue(id, HdCameraTokens->projectionMatrix);
+    if (!value.IsEmpty()) {
+      _projectionMatrix = value.Get<GfMatrix4d>();
+      const float focalLength = _data.GetFocalLength();  // Get default focal length
+#if PXR_VERSION >= 2102
+      _data.SetFromViewAndProjectionMatrix(GetViewMatrix(), _projectionMatrix, focalLength);
+#else
+      if (_projectionMatrix[2][3] < -0.5) {
+        _data.SetProjection(GfCamera::Perspective);
+
+        const float horizontalAperture = (2.0 * focalLength) / _projectionMatrix[0][0];
+        _data.SetHorizontalAperture(horizontalAperture);
+        _data.SetHorizontalApertureOffset(0.5 * horizontalAperture * _projectionMatrix[2][0]);
+        const float verticalAperture = (2.0 * focalLength) / _projectionMatrix[1][1];
+        _data.SetVerticalAperture(verticalAperture);
+        _data.SetVerticalApertureOffset(0.5 * verticalAperture * _projectionMatrix[2][1]);
+
+        _data.SetClippingRange(
+            GfRange1f(_projectionMatrix[3][2] / (_projectionMatrix[2][2] - 1.0),
+                      _projectionMatrix[3][2] / (_projectionMatrix[2][2] + 1.0)));
+      }
+      else {
+        _data.SetProjection(GfCamera::Orthographic);
+
+        const float horizontalAperture = (2.0 / GfCamera::APERTURE_UNIT) / _projectionMatrix[0][0];
+        _data.SetHorizontalAperture(horizontalAperture);
+        _data.SetHorizontalApertureOffset(-0.5 * horizontalAperture * _projectionMatrix[3][0]);
+        const float verticalAperture = (2.0 / GfCamera::APERTURE_UNIT) / _projectionMatrix[1][1];
+        _data.SetVerticalAperture(verticalAperture);
+        _data.SetVerticalApertureOffset(-0.5 * verticalAperture * _projectionMatrix[3][1]);
+
+        const double nearMinusFarHalf = 1.0 / _projectionMatrix[2][2];
+        const double nearPlusFarHalf = nearMinusFarHalf * _projectionMatrix[3][2];
+        _data.SetClippingRange(
+            GfRange1f(nearPlusFarHalf + nearMinusFarHalf, nearPlusFarHalf - nearMinusFarHalf));
+      }
+#endif
+    }
+  }
+
+  if (*dirtyBits & DirtyBits::DirtyWindowPolicy) {
+    value = sceneDelegate->GetCameraParamValue(id, HdCameraTokens->windowPolicy);
+    if (!value.IsEmpty()) {
+      _windowPolicy = value.Get<CameraUtilConformWindowPolicy>();
+    }
+  }
+
+  if (*dirtyBits & DirtyBits::DirtyClipPlanes) {
+    value = sceneDelegate->GetCameraParamValue(id, HdCameraTokens->clipPlanes);
+    if (!value.IsEmpty()) {
+      _clipPlanes = value.Get<std::vector<GfVec4d>>();
+    }
+  }
+
+  if (*dirtyBits & DirtyBits::DirtyParams) {
+#if PXR_VERSION >= 2102
+    value = sceneDelegate->GetCameraParamValue(id, HdCameraTokens->projection);
+    if (!value.IsEmpty()) {
+      _projection = value.Get<Projection>();
+      _data.SetProjection(_projection != Orthographic ? GfCamera::Perspective :
+                                                        GfCamera::Orthographic);
+    }
+#else
+    value = sceneDelegate->GetCameraParamValue(id, UsdGeomTokens->projection);
+    if (!value.IsEmpty()) {
+      _data.SetProjection(value.Get<TfToken>() != UsdGeomTokens->orthographic ?
+                              GfCamera::Perspective :
+                              GfCamera::Orthographic);
+    }
+#endif
+
+    value = sceneDelegate->GetCameraParamValue(id, HdCameraTokens->horizontalAperture);
+    if (!value.IsEmpty()) {
+      const auto horizontalAperture = value.Get<float>();
+#if PXR_VERSION >= 2102
+      _horizontalAperture = horizontalAperture;
+#endif
+      _data.SetHorizontalAperture(horizontalAperture / GfCamera::APERTURE_UNIT);
+    }
+
+    value = sceneDelegate->GetCameraParamValue(id, HdCameraTokens->verticalAperture);
+    if (!value.IsEmpty()) {
+      const auto verticalAperture = value.Get<float>();
+#if PXR_VERSION >= 2102
+      _verticalAperture = verticalAperture;
+#endif
+      _data.SetVerticalAperture(verticalAperture / GfCamera::APERTURE_UNIT);
+    }
+
+    value = sceneDelegate->GetCameraParamValue(id, HdCameraTokens->horizontalApertureOffset);
+    if (!value.IsEmpty()) {
+      const auto horizontalApertureOffset = value.Get<float>();
+#if PXR_VERSION >= 2102
+      _horizontalApertureOffset = horizontalApertureOffset;
+#endif
+      _data.SetHorizontalApertureOffset(horizontalApertureOffset / GfCamera::APERTURE_UNIT);
+    }
+
+    value = sceneDelegate->GetCameraParamValue(id, HdCameraTokens->verticalApertureOffset);
+    if (!value.IsEmpty()) {
+      const auto verticalApertureOffset = value.Get<float>();
+#if PXR_VERSION >= 2102
+      _verticalApertureOffset = verticalApertureOffset;
+#endif
+      _data.SetVerticalApertureOffset(verticalApertureOffset / GfCamera::APERTURE_UNIT);
+    }
+
+    value = sceneDelegate->GetCameraParamValue(id, HdCameraTokens->focalLength);
+    if (!value.IsEmpty()) {
+      const auto focalLength = value.Get<float>();
+#if PXR_VERSION >= 2102
+      _focalLength = focalLength;
+#endif
+      _data.SetFocalLength(focalLength / GfCamera::FOCAL_LENGTH_UNIT);
+    }
+
+    value = sceneDelegate->GetCameraParamValue(id, HdCameraTokens->clippingRange);
+    if (!value.IsEmpty()) {
+      const auto clippingRange = value.Get<GfRange1f>();
+#if PXR_VERSION >= 2102
+      _clippingRange = clippingRange;
+#endif
+      _data.SetClippingRange(clippingRange);
+    }
+
+    value = sceneDelegate->GetCameraParamValue(id, HdCameraTokens->fStop);
+    if (!value.IsEmpty()) {
+      const auto fStop = value.Get<float>();
+#if PXR_VERSION >= 2102
+      _fStop = fStop;
+#endif
+      _data.SetFStop(fStop);
+    }
+
+    value = sceneDelegate->GetCameraParamValue(id, HdCameraTokens->focusDistance);
+    if (!value.IsEmpty()) {
+      const auto focusDistance = value.Get<float>();
+#if PXR_VERSION >= 2102
+      _focusDistance = focusDistance;
+#endif
+      _data.SetFocusDistance(focusDistance);
+    }
+  }
+
+  *dirtyBits = DirtyBits::Clean;
+}
+
+void HdCyclesCamera::Finalize(HdRenderParam *renderParam)
+{
+  HdCamera::Finalize(renderParam);
+}
+
+void HdCyclesCamera::ApplyCameraSettings(Camera *cam) const
+{
+  ApplyCameraSettings(_data, _windowPolicy, cam);
+
+  array<Transform> motion(_transformSamples.count);
+  for (size_t i = 0; i < _transformSamples.count; ++i)
+    motion[i] = convert_transform(_transformSamples.values[i]) *
+                transform_scale(1.0f, 1.0f, -1.0f);
+  cam->set_motion(motion);
+}
+
+void HdCyclesCamera::ApplyCameraSettings(const GfCamera &dataUnconformedWindow,
+                                         CameraUtilConformWindowPolicy windowPolicy,
+                                         Camera *cam)
+{
+  const float width = cam->get_full_width();
+  const float height = cam->get_full_height();
+
+  auto data = dataUnconformedWindow;
+  CameraUtilConformWindow(&data, windowPolicy, width / height);
+
+  static_assert(GfCamera::Perspective == CAMERA_PERSPECTIVE &&
+                GfCamera::Orthographic == CAMERA_ORTHOGRAPHIC);
+  cam->set_camera_type(static_cast<CameraType>(data.GetProjection()));
+
+  auto viewplane = data.GetFrustum().GetWindow();
+  auto focalLength = 1.0f;
+  if (data.GetProjection() == GfCamera::Perspective) {
+    viewplane *= 2.0 / viewplane.GetSize()[1];  // Normalize viewplane
+    focalLength = data.GetFocalLength() * 1e-3f;
+
+    cam->set_fov(GfDegreesToRadians(data.GetFieldOfView(GfCamera::FOVVertical)));
+  }
+
+  cam->set_sensorwidth(data.GetHorizontalAperture() * GfCamera::APERTURE_UNIT);
+  cam->set_sensorheight(data.GetVerticalAperture() * GfCamera::APERTURE_UNIT);
+
+  cam->set_nearclip(data.GetClippingRange().GetMin());
+  cam->set_farclip(data.GetClippingRange().GetMax());
+
+  cam->set_viewplane_left(viewplane.GetMin()[0]);
+  cam->set_viewplane_right(viewplane.GetMax()[0]);
+  cam->set_viewplane_bottom(viewplane.GetMin()[1]);
+  cam->set_viewplane_top(viewplane.GetMax()[1]);
+
+  if (data.GetFStop() != 0.0f) {
+    cam->set_focaldistance(data.GetFocusDistance());
+    cam->set_aperturesize(focalLength / (2.0f * data.GetFStop()));
+  }
+
+  cam->set_matrix(convert_transform(data.GetTransform()) * transform_scale(1.0f, 1.0f, -1.0f));
+}
+
+void HdCyclesCamera::ApplyCameraSettings(const GfMatrix4d &worldToViewMatrix,
+                                         const GfMatrix4d &projectionMatrix,
+                                         const std::vector<GfVec4d> &clipPlanes,
+                                         Camera *cam)
+{
+#if PXR_VERSION >= 2102
+  GfCamera data;
+  data.SetFromViewAndProjectionMatrix(worldToViewMatrix, projectionMatrix);
+
+  ApplyCameraSettings(data, CameraUtilFit, cam);
+#else
+  TF_CODING_ERROR("Not implemented");
+#endif
+}
+
+HDCYCLES_NAMESPACE_CLOSE_SCOPE