DRWManager: New implementation.

This is a new implementation of the draw manager using modern
rendering practices and GPU driven culling.

This only ports features that are not considered deprecated or to be
removed.

The old DRW API is kept working along side this new one, and does not
interfeer with it. However this needed some more hacking inside the
draw_view_lib.glsl. At least the create info are well separated.

The reviewer might start by looking at `draw_pass_test.cc` to see the
API in usage.

Important files are `draw_pass.hh`, `draw_command.hh`,
`draw_command_shared.hh`.

In a nutshell (for a developper used to old DRW API):
- `DRWShadingGroups` are replaced by `Pass<T>::Sub`.
- Contrary to DRWShadingGroups, all commands recorded inside a pass or
   sub-pass (even binds / push_constant / uniforms) will be executed in order.
- All memory is managed per object (except for Sub-Pass which are managed
   by their parent pass) and not from draw manager pools. So passes "can"
   potentially be recorded once and submitted multiple time (but this is
   not really encouraged for now). The only implicit link is between resource
   lifetime and `ResourceHandles`
- Sub passes can be any level deep.
- IMPORTANT: All state propagate from sub pass to subpass. There is no
   state stack concept anymore. Ensure the correct render state is set before
   drawing anything using `Pass::state_set()`.
- The drawcalls now needs a `ResourceHandle` instead of an `Object *`.
   This is to remove any implicit dependency between `Pass` and `Manager`.
   This was a huge problem in old implementation since the manager did not
   know what to pull from the object. Now it is explicitly requested by the
   engine.
- The pases need to be submitted to a `draw::Manager` instance which can
   be retrieved using `DRW_manager_get()` (for now).

Internally:
- All object data are stored in contiguous storage buffers. Removing a lot
   of complexity in the pass submission.
- Draw calls are sorted and visibility tested on GPU. Making more modern
   culling and better instancing usage possible in the future.
- Unit Tests have been added for regression testing and avoid most API
   breakage.
- `draw::View` now contains culling data for all objects in the scene
   allowing caching for multiple views.
- Bounding box and sphere final setup is moved to GPU.
- Some global resources locations have been hardcoded to reduce complexity.

What is missing:
- ~~Workaround for lack of gl_BaseInstanceARB.~~ Done
- ~~Object Uniform Attributes.~~ Done (Not in this patch)
- Workaround for hardware supporting a maximum of 8 SSBO.

Reviewed By: jbakker

Differential Revision: https://developer.blender.org/D15817

1 files changed, 192 insertions, 0 deletions

diff --git a/source/blender/draw/intern/draw_manager.hh b/source/blender/draw/intern/draw_manager.hh
new file mode 100644
index 00000000000..5f110b8bb6b
--- /dev/null
+++ b/source/blender/draw/intern/draw_manager.hh
@@ -0,0 +1,192 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later
+ * Copyright 2022 Blender Foundation. */
+
+#pragma once
+
+/** \file
+ * \ingroup draw
+ *
+ * `draw::Manager` is the interface between scene data and viewport engines.
+ *
+ * It holds per component data (`ObjectInfo`, `ObjectMatrices`, ...) indexed per `ResourceHandle`.
+ *
+ * \note It is currently work in progress and should replace the old global draw manager.
+ */
+
+#include "BLI_sys_types.h"
+
+#include "draw_resource.hh"
+#include "draw_view.hh"
+
+#include <string>
+
+namespace blender::draw {
+
+/* Forward declarations. */
+
+namespace detail {
+template<typename T> class Pass;
+}  // namespace detail
+
+namespace command {
+class DrawCommandBuf;
+class DrawMultiBuf;
+}  // namespace command
+
+using PassSimple = detail::Pass<command::DrawCommandBuf>;
+using PassMain = detail::Pass<command::DrawMultiBuf>;
+class PassSortable;
+
+class Manager {
+  using ObjectMatricesBuf = StorageArrayBuffer<ObjectMatrices, 128>;
+  using ObjectBoundsBuf = StorageArrayBuffer<ObjectBounds, 128>;
+  using ObjectInfosBuf = StorageArrayBuffer<ObjectInfos, 128>;
+
+ public:
+  struct SubmitDebugOutput {
+    /** Indexed by resource id. */
+    Span<uint32_t> visibility;
+    /** Indexed by drawn instance. */
+    Span<uint32_t> resource_id;
+  };
+
+  struct DataDebugOutput {
+    /** Indexed by resource id. */
+    Span<ObjectMatrices> matrices;
+    /** Indexed by resource id. */
+    Span<ObjectBounds> bounds;
+    /** Indexed by resource id. */
+    Span<ObjectInfos> infos;
+  };
+
+  /**
+   * Buffers containing all object data. Referenced by resource index.
+   * Exposed as public members for shader access after sync.
+   */
+  ObjectMatricesBuf matrix_buf;
+  ObjectBoundsBuf bounds_buf;
+  ObjectInfosBuf infos_buf;
+
+  /** List of textures coming from Image data-blocks. They need to be refcounted in order to avoid
+   * beeing freed in another thread. */
+  Vector<GPUTexture *> acquired_textures;
+
+ private:
+  uint resource_len_ = 0;
+  Object *object = nullptr;
+
+  Object *object_active = nullptr;
+
+ public:
+  Manager(){};
+  ~Manager();
+
+  /**
+   * Create a new resource handle for the given object. Can be called multiple time with the
+   * same object **successively** without duplicating the data.
+   */
+  ResourceHandle resource_handle(const ObjectRef ref);
+  /**
+   * Get resource id for a loose matrix. The draw-calls for this resource handle won't be culled
+   * and there won't be any associated object info / bounds. Assumes correct handedness / winding.
+   */
+  ResourceHandle resource_handle(const float4x4 &model_matrix);
+  /**
+   * Get resource id for a loose matrix with bounds. The draw-calls for this resource handle will
+   * be culled bute there won't be any associated object info / bounds. Assumes correct handedness
+   * / winding.
+   */
+  ResourceHandle resource_handle(const float4x4 &model_matrix,
+                                 const float3 &bounds_center,
+                                 const float3 &bounds_half_extent);
+
+  /**
+   * Populate additional per resource data on demand.
+   */
+  void extract_object_attributes(ResourceHandle handle,
+                                 Object &object,
+                                 Span<GPUMaterial *> materials);
+
+  /**
+   * Submit a pass for drawing. All resource reference will be dereferenced and commands will be
+   * sent to GPU.
+   */
+  void submit(PassSimple &pass, View &view);
+  void submit(PassMain &pass, View &view);
+  void submit(PassSortable &pass, View &view);
+  /**
+   * Variant without any view. Must not contain any shader using `draw_view` create info.
+   */
+  void submit(PassSimple &pass);
+
+  /**
+   * Submit a pass for drawing but read back all data buffers for inspection.
+   */
+  SubmitDebugOutput submit_debug(PassSimple &pass, View &view);
+  SubmitDebugOutput submit_debug(PassMain &pass, View &view);
+
+  /**
+   * Check data buffers of the draw manager. Only to be used after end_sync().
+   */
+  DataDebugOutput data_debug();
+
+  /**
+   * Will acquire the texture using ref counting and release it after drawing. To be used for
+   * texture coming from blender Image.
+   */
+  void acquire_texture(GPUTexture *texture)
+  {
+    GPU_texture_ref(texture);
+    acquired_textures.append(texture);
+  }
+
+  /** TODO(fclem): The following should become private at some point. */
+  void begin_sync();
+  void end_sync();
+
+  void debug_bind();
+};
+
+inline ResourceHandle Manager::resource_handle(const ObjectRef ref)
+{
+  bool is_active_object = (ref.dupli_object ? ref.dupli_parent : ref.object) == object_active;
+  matrix_buf.get_or_resize(resource_len_).sync(*ref.object);
+  bounds_buf.get_or_resize(resource_len_).sync(*ref.object);
+  infos_buf.get_or_resize(resource_len_).sync(ref, is_active_object);
+  return ResourceHandle(resource_len_++, (ref.object->transflag & OB_NEG_SCALE) != 0);
+}
+
+inline ResourceHandle Manager::resource_handle(const float4x4 &model_matrix)
+{
+  matrix_buf.get_or_resize(resource_len_).sync(model_matrix);
+  bounds_buf.get_or_resize(resource_len_).sync();
+  infos_buf.get_or_resize(resource_len_).sync();
+  return ResourceHandle(resource_len_++, false);
+}
+
+inline ResourceHandle Manager::resource_handle(const float4x4 &model_matrix,
+                                               const float3 &bounds_center,
+                                               const float3 &bounds_half_extent)
+{
+  matrix_buf.get_or_resize(resource_len_).sync(model_matrix);
+  bounds_buf.get_or_resize(resource_len_).sync(bounds_center, bounds_half_extent);
+  infos_buf.get_or_resize(resource_len_).sync();
+  return ResourceHandle(resource_len_++, false);
+}
+
+inline void Manager::extract_object_attributes(ResourceHandle handle,
+                                               Object &object,
+                                               Span<GPUMaterial *> materials)
+{
+  /* TODO */
+  (void)handle;
+  (void)object;
+  (void)materials;
+}
+
+}  // namespace blender::draw
+
+/* TODO(@fclem): This is for testing. The manager should be passed to the engine through the
+ * callbacks. */
+blender::draw::Manager *DRW_manager_get();
+blender::draw::ObjectRef DRW_object_ref_get(Object *object);