diff options
author | Julian Eisel <julian@blender.org> | 2020-03-17 22:20:55 +0300 |
---|---|---|
committer | Julian Eisel <julian@blender.org> | 2020-03-17 23:42:44 +0300 |
commit | dc2df8307f41888bab722f75fa9e73adecf86b72 (patch) | |
tree | f83c54f43e27a07e9cb9fed306d79b08864f29f2 /intern/ghost | |
parent | 406bfd43040a5526702b51f88f1491cb61aecedb (diff) |
VR: Initial Virtual Reality support - Milestone 1, Scene Inspection
NOTE: While most of the milestone 1 goals are there, a few smaller features and
improvements are still to be done.
Big picture of this milestone: Initial, OpenXR-based virtual reality support
for users and foundation for advanced use cases.
Maniphest Task: https://developer.blender.org/T71347
The tasks contains more information about this milestone.
To be clear: This is not a feature rich VR implementation, it's focused on the
initial scene inspection use case. We intentionally focused on that, further
features like controller support are part of the next milestone.
- How to use?
Instructions on how to use this are here:
https://wiki.blender.org/wiki/User:Severin/GSoC-2019/How_to_Test
These will be updated and moved to a more official place (likely the manual) soon.
Currently Windows Mixed Reality and Oculus devices are usable. Valve/HTC
headsets don't support the OpenXR standard yet and hence, do not work with this
implementation.
---------------
This is the C-side implementation of the features added for initial VR
support as per milestone 1. A "VR Scene Inspection" Add-on will be
committed separately, to expose the VR functionality in the UI. It also
adds some further features for milestone 1, namely a landmarking system
(stored view locations in the VR space)
Main additions/features:
* Support for rendering viewports to an HMD, with good performance.
* Option to sync the VR view perspective with a fully interactive,
regular 3D View (VR-Mirror).
* Option to disable positional tracking. Keeps the current position (calculated
based on the VR eye center pose) when enabled while a VR session is running.
* Some regular viewport settings for the VR view
* RNA/Python-API to query and set VR session state information.
* WM-XR: Layer tying Ghost-XR to the Blender specific APIs/data
* wmSurface API: drawable, non-window container (manages Ghost-OpenGL and GPU
context)
* DNA/RNA for management of VR session settings
* `--debug-xr` and `--debug-xr-time` commandline options
* Utility batch & config file for using the Oculus runtime on Windows.
* Most VR data is runtime only. The exception is user settings which are saved
to files (`XrSessionSettings`).
* VR support can be disabled through the `WITH_XR_OPENXR` compiler flag.
For architecture and code documentation, see
https://wiki.blender.org/wiki/Source/Interface/XR.
---------------
A few thank you's:
* A huge shoutout to Ray Molenkamp for his help during the project - it would
have not been that successful without him!
* Sebastian Koenig and Simeon Conzendorf for testing and feedback!
* The reviewers, especially Brecht Van Lommel!
* Dalai Felinto for pushing and managing me to get this done ;)
* The OpenXR working group for providing an open standard. I think we're the
first bigger application to adopt OpenXR. Congratulations to them and
ourselves :)
This project started as a Google Summer of Code 2019 project - "Core Support of
Virtual Reality Headsets through OpenXR" (see
https://wiki.blender.org/wiki/User:Severin/GSoC-2019/).
Some further information, including ideas for further improvements can be found
in the final GSoC report:
https://wiki.blender.org/wiki/User:Severin/GSoC-2019/Final_Report
Differential Revisions: D6193, D7098
Reviewed by: Brecht Van Lommel, Jeroen Bakker
Diffstat (limited to 'intern/ghost')
-rw-r--r-- | intern/ghost/CMakeLists.txt | 3 | ||||
-rw-r--r-- | intern/ghost/GHOST_C-api.h | 12 | ||||
-rw-r--r-- | intern/ghost/GHOST_IContext.h | 9 | ||||
-rw-r--r-- | intern/ghost/GHOST_Types.h | 31 | ||||
-rw-r--r-- | intern/ghost/intern/GHOST_C-api.cpp | 14 | ||||
-rw-r--r-- | intern/ghost/intern/GHOST_Context.h | 8 | ||||
-rw-r--r-- | intern/ghost/intern/GHOST_IXrGraphicsBinding.h | 2 | ||||
-rw-r--r-- | intern/ghost/intern/GHOST_XrContext.cpp | 4 | ||||
-rw-r--r-- | intern/ghost/intern/GHOST_XrContext.h | 3 | ||||
-rw-r--r-- | intern/ghost/intern/GHOST_XrGraphicsBinding.cpp | 2 | ||||
-rw-r--r-- | intern/ghost/intern/GHOST_XrSession.cpp | 43 | ||||
-rw-r--r-- | intern/ghost/intern/GHOST_XrSession.h | 3 |
12 files changed, 111 insertions, 23 deletions
diff --git a/intern/ghost/CMakeLists.txt b/intern/ghost/CMakeLists.txt index 07d98475c00..611da8d6a44 100644 --- a/intern/ghost/CMakeLists.txt +++ b/intern/ghost/CMakeLists.txt @@ -377,6 +377,9 @@ if(WITH_XR_OPENXR) list(APPEND INC_SYS ${XR_OPENXR_SDK_INCLUDE_DIR} ) + list(APPEND LIB + ${XR_OPENXR_SDK_LIBRARIES} + ) set(XR_PLATFORM_DEFINES -DXR_USE_GRAPHICS_API_OPENGL) diff --git a/intern/ghost/GHOST_C-api.h b/intern/ghost/GHOST_C-api.h index aafe374a93b..aba5b5f733b 100644 --- a/intern/ghost/GHOST_C-api.h +++ b/intern/ghost/GHOST_C-api.h @@ -759,6 +759,18 @@ extern GHOST_TSuccess GHOST_ReleaseOpenGLContext(GHOST_ContextHandle contexthand /** * Get the OpenGL framebuffer handle that serves as a default framebuffer. */ +extern unsigned int GHOST_GetContextDefaultOpenGLFramebuffer(GHOST_ContextHandle contexthandle); + +/** + * Returns whether a context is rendered upside down compared to OpenGL. This only needs to be + * called if there's a non-OpenGL context, which is really the exception. + * So generally, this does not need to be called. + */ +extern int GHOST_isUpsideDownContext(GHOST_ContextHandle contexthandle); + +/** + * Get the OpenGL framebuffer handle that serves as a default framebuffer. + */ extern unsigned int GHOST_GetDefaultOpenGLFramebuffer(GHOST_WindowHandle windwHandle); /** diff --git a/intern/ghost/GHOST_IContext.h b/intern/ghost/GHOST_IContext.h index a341e18ca0a..33422b8e351 100644 --- a/intern/ghost/GHOST_IContext.h +++ b/intern/ghost/GHOST_IContext.h @@ -56,6 +56,15 @@ class GHOST_IContext { */ virtual GHOST_TSuccess releaseDrawingContext() = 0; + virtual unsigned int getDefaultFramebuffer() = 0; + + virtual GHOST_TSuccess swapBuffers() = 0; + + /** + * Returns if the window is rendered upside down compared to OpenGL. + */ + virtual bool isUpsideDown() const = 0; + #ifdef WITH_CXX_GUARDEDALLOC MEM_CXX_CLASS_ALLOC_FUNCS("GHOST:GHOST_IContext") #endif diff --git a/intern/ghost/GHOST_Types.h b/intern/ghost/GHOST_Types.h index adda782b96d..70c4d3ef00c 100644 --- a/intern/ghost/GHOST_Types.h +++ b/intern/ghost/GHOST_Types.h @@ -598,6 +598,8 @@ typedef void (*GHOST_TimerProcPtr)(struct GHOST_TimerTaskHandle__ *task, GHOST_T #ifdef WITH_XR_OPENXR +struct GHOST_XrError; +struct GHOST_XrDrawViewInfo; /** * The XR view (i.e. the OpenXR runtime) may require a different graphics library than OpenGL. An * offscreen texture of the viewport will then be drawn into using OpenGL, but the final texture @@ -605,7 +607,7 @@ typedef void (*GHOST_TimerProcPtr)(struct GHOST_TimerTaskHandle__ *task, GHOST_T * * This enum defines the possible graphics bindings to attempt to enable. */ -typedef enum { +typedef enum GHOST_TXrGraphicsBinding { GHOST_kXrGraphicsUnknown = 0, GHOST_kXrGraphicsOpenGL, # ifdef WIN32 @@ -614,6 +616,16 @@ typedef enum { /* For later */ // GHOST_kXrGraphicsVulkan, } GHOST_TXrGraphicsBinding; + +typedef void (*GHOST_XrErrorHandlerFn)(const struct GHOST_XrError *); + +typedef void (*GHOST_XrSessionExitFn)(void *customdata); + +typedef void *(*GHOST_XrGraphicsContextBindFn)(enum GHOST_TXrGraphicsBinding graphics_lib); +typedef void (*GHOST_XrGraphicsContextUnbindFn)(enum GHOST_TXrGraphicsBinding graphics_lib, + GHOST_ContextHandle graphics_context); +typedef void (*GHOST_XrDrawViewFn)(const struct GHOST_XrDrawViewInfo *draw_view, void *customdata); + /* An array of GHOST_TXrGraphicsBinding items defining the candidate bindings to use. The first * available candidate will be chosen, so order defines priority. */ typedef const GHOST_TXrGraphicsBinding *GHOST_XrGraphicsBindingCandidates; @@ -638,13 +650,17 @@ typedef struct { typedef struct { GHOST_XrPose base_pose; + + GHOST_XrSessionExitFn exit_fn; + void *exit_customdata; } GHOST_XrSessionBeginInfo; -typedef struct { +typedef struct GHOST_XrDrawViewInfo { int ofsx, ofsy; int width, height; - GHOST_XrPose pose; + GHOST_XrPose eye_pose; + GHOST_XrPose local_pose; struct { float angle_left, angle_right; @@ -655,19 +671,12 @@ typedef struct { char expects_srgb_buffer; } GHOST_XrDrawViewInfo; -typedef struct { +typedef struct GHOST_XrError { const char *user_message; void *customdata; } GHOST_XrError; -typedef void (*GHOST_XrErrorHandlerFn)(const GHOST_XrError *); - -typedef void *(*GHOST_XrGraphicsContextBindFn)(GHOST_TXrGraphicsBinding graphics_lib); -typedef void (*GHOST_XrGraphicsContextUnbindFn)(GHOST_TXrGraphicsBinding graphics_lib, - void *graphics_context); -typedef void (*GHOST_XrDrawViewFn)(const GHOST_XrDrawViewInfo *draw_view, void *customdata); - #endif #endif // __GHOST_TYPES_H__ diff --git a/intern/ghost/intern/GHOST_C-api.cpp b/intern/ghost/intern/GHOST_C-api.cpp index 60d20474a5a..d43a2637ad3 100644 --- a/intern/ghost/intern/GHOST_C-api.cpp +++ b/intern/ghost/intern/GHOST_C-api.cpp @@ -709,6 +709,20 @@ GHOST_TSuccess GHOST_ReleaseOpenGLContext(GHOST_ContextHandle contexthandle) return context->releaseDrawingContext(); } +unsigned int GHOST_GetContextDefaultOpenGLFramebuffer(GHOST_ContextHandle contexthandle) +{ + GHOST_IContext *context = (GHOST_IContext *)contexthandle; + + return context->getDefaultFramebuffer(); +} + +int GHOST_isUpsideDownContext(GHOST_ContextHandle contexthandle) +{ + GHOST_IContext *context = (GHOST_IContext *)contexthandle; + + return context->isUpsideDown(); +} + unsigned int GHOST_GetDefaultOpenGLFramebuffer(GHOST_WindowHandle windowhandle) { GHOST_IWindow *window = (GHOST_IWindow *)windowhandle; diff --git a/intern/ghost/intern/GHOST_Context.h b/intern/ghost/intern/GHOST_Context.h index bbf6d6a510d..0bd6f63d07e 100644 --- a/intern/ghost/intern/GHOST_Context.h +++ b/intern/ghost/intern/GHOST_Context.h @@ -120,6 +120,14 @@ class GHOST_Context : public GHOST_IContext { } /** + * Returns if the window is rendered upside down compared to OpenGL. + */ + inline bool isUpsideDown() const + { + return false; + } + + /** * Gets the OpenGL framebuffer associated with the OpenGL context * \return The ID of an OpenGL framebuffer object. */ diff --git a/intern/ghost/intern/GHOST_IXrGraphicsBinding.h b/intern/ghost/intern/GHOST_IXrGraphicsBinding.h index 19fe00cdad5..25281d3d0ba 100644 --- a/intern/ghost/intern/GHOST_IXrGraphicsBinding.h +++ b/intern/ghost/intern/GHOST_IXrGraphicsBinding.h @@ -41,6 +41,8 @@ class GHOST_IXrGraphicsBinding { #endif } oxr_binding; + virtual ~GHOST_IXrGraphicsBinding() = default; + /** * Does __not__ require this object is initialized (can be called prior to * #initFromGhostContext). It's actually meant to be called first. diff --git a/intern/ghost/intern/GHOST_XrContext.cpp b/intern/ghost/intern/GHOST_XrContext.cpp index 410837e9805..d7b83114c85 100644 --- a/intern/ghost/intern/GHOST_XrContext.cpp +++ b/intern/ghost/intern/GHOST_XrContext.cpp @@ -454,10 +454,12 @@ GHOST_TXrGraphicsBinding GHOST_XrContext::determineGraphicsBindingTypeToEnable( void GHOST_XrContext::startSession(const GHOST_XrSessionBeginInfo *begin_info) { + m_custom_funcs.session_exit_fn = begin_info->exit_fn; + m_custom_funcs.session_exit_customdata = begin_info->exit_customdata; + if (m_session == nullptr) { m_session = std::unique_ptr<GHOST_XrSession>(new GHOST_XrSession(this)); } - m_session->start(begin_info); } diff --git a/intern/ghost/intern/GHOST_XrContext.h b/intern/ghost/intern/GHOST_XrContext.h index b361fb5caf8..5140ad7ea27 100644 --- a/intern/ghost/intern/GHOST_XrContext.h +++ b/intern/ghost/intern/GHOST_XrContext.h @@ -33,6 +33,9 @@ struct GHOST_XrCustomFuncs { /** Function to release (possibly free) a graphics context. */ GHOST_XrGraphicsContextUnbindFn gpu_ctx_unbind_fn = nullptr; + GHOST_XrSessionExitFn session_exit_fn = nullptr; + void *session_exit_customdata = nullptr; + /** Custom per-view draw function for Blender side drawing. */ GHOST_XrDrawViewFn draw_view_fn = nullptr; }; diff --git a/intern/ghost/intern/GHOST_XrGraphicsBinding.cpp b/intern/ghost/intern/GHOST_XrGraphicsBinding.cpp index ddc757b8f8a..f094b0744a2 100644 --- a/intern/ghost/intern/GHOST_XrGraphicsBinding.cpp +++ b/intern/ghost/intern/GHOST_XrGraphicsBinding.cpp @@ -116,6 +116,8 @@ class GHOST_XrGraphicsBindingOpenGL : public GHOST_IXrGraphicsBinding { oxr_binding.glx.glxDrawable = ctx_glx->m_window; oxr_binding.glx.glxContext = ctx_glx->m_context; oxr_binding.glx.visualid = visual_info->visualid; + + XFree(visual_info); #elif defined(WIN32) GHOST_ContextWGL *ctx_wgl = static_cast<GHOST_ContextWGL *>(ghost_ctx); diff --git a/intern/ghost/intern/GHOST_XrSession.cpp b/intern/ghost/intern/GHOST_XrSession.cpp index 1e2b8c0bc9d..a85bde3cab6 100644 --- a/intern/ghost/intern/GHOST_XrSession.cpp +++ b/intern/ghost/intern/GHOST_XrSession.cpp @@ -43,6 +43,7 @@ struct OpenXRSessionData { /* Only stereo rendering supported now. */ const XrViewConfigurationType view_type = XR_VIEW_CONFIGURATION_TYPE_PRIMARY_STEREO; XrSpace reference_space; + XrSpace view_space; std::vector<XrView> views; std::vector<std::unique_ptr<GHOST_XrSwapchain>> swapchains; }; @@ -81,6 +82,8 @@ GHOST_XrSession::~GHOST_XrSession() m_oxr->session = XR_NULL_HANDLE; m_oxr->session_state = XR_SESSION_STATE_UNKNOWN; + + m_context->getCustomFuncs().session_exit_fn(m_context->getCustomFuncs().session_exit_customdata); } /** @@ -107,7 +110,7 @@ void GHOST_XrSession::initSystem() * * \{ */ -static void create_reference_space(OpenXRSessionData *oxr, const GHOST_XrPose *base_pose) +static void create_reference_spaces(OpenXRSessionData *oxr, const GHOST_XrPose *base_pose) { XrReferenceSpaceCreateInfo create_info = {XR_TYPE_REFERENCE_SPACE_CREATE_INFO}; create_info.poseInReferenceSpace.orientation.w = 1.0f; @@ -138,6 +141,10 @@ static void create_reference_space(OpenXRSessionData *oxr, const GHOST_XrPose *b CHECK_XR(xrCreateReferenceSpace(oxr->session, &create_info, &oxr->reference_space), "Failed to create reference space."); + + create_info.referenceSpaceType = XR_REFERENCE_SPACE_TYPE_VIEW; + CHECK_XR(xrCreateReferenceSpace(oxr->session, &create_info, &oxr->view_space), + "Failed to create view reference space."); } void GHOST_XrSession::start(const GHOST_XrSessionBeginInfo *begin_info) @@ -184,7 +191,7 @@ void GHOST_XrSession::start(const GHOST_XrSessionBeginInfo *begin_info) "detailed error information to the command line."); prepareDrawing(); - create_reference_space(m_oxr.get(), &begin_info->base_pose); + create_reference_spaces(m_oxr.get(), &begin_info->base_pose); } void GHOST_XrSession::requestEnd() @@ -343,16 +350,22 @@ void GHOST_XrSession::draw(void *draw_customdata) endFrameDrawing(&layers); } +static void copy_openxr_pose_to_ghost_pose(const XrPosef &oxr_pose, GHOST_XrPose &r_ghost_pose) +{ + /* Set and convert to Blender coodinate space. */ + r_ghost_pose.position[0] = oxr_pose.position.x; + r_ghost_pose.position[1] = oxr_pose.position.y; + r_ghost_pose.position[2] = oxr_pose.position.z; + r_ghost_pose.orientation_quat[0] = oxr_pose.orientation.w; + r_ghost_pose.orientation_quat[1] = oxr_pose.orientation.x; + r_ghost_pose.orientation_quat[2] = oxr_pose.orientation.y; + r_ghost_pose.orientation_quat[3] = oxr_pose.orientation.z; +} + static void ghost_xr_draw_view_info_from_view(const XrView &view, GHOST_XrDrawViewInfo &r_info) { /* Set and convert to Blender coodinate space. */ - r_info.pose.position[0] = view.pose.position.x; - r_info.pose.position[1] = view.pose.position.y; - r_info.pose.position[2] = view.pose.position.z; - r_info.pose.orientation_quat[0] = view.pose.orientation.w; - r_info.pose.orientation_quat[1] = view.pose.orientation.x; - r_info.pose.orientation_quat[2] = view.pose.orientation.y; - r_info.pose.orientation_quat[3] = view.pose.orientation.z; + copy_openxr_pose_to_ghost_pose(view.pose, r_info.eye_pose); r_info.fov.angle_left = view.fov.angleLeft; r_info.fov.angle_right = view.fov.angleRight; @@ -370,6 +383,7 @@ static bool ghost_xr_draw_view_expects_srgb_buffer(const GHOST_XrContext *contex void GHOST_XrSession::drawView(GHOST_XrSwapchain &swapchain, XrCompositionLayerProjectionView &r_proj_layer_view, + XrSpaceLocation &view_location, XrView &view, void *draw_customdata) { @@ -386,6 +400,7 @@ void GHOST_XrSession::drawView(GHOST_XrSwapchain &swapchain, draw_view_info.ofsy = r_proj_layer_view.subImage.imageRect.offset.y; draw_view_info.width = r_proj_layer_view.subImage.imageRect.extent.width; draw_view_info.height = r_proj_layer_view.subImage.imageRect.extent.height; + copy_openxr_pose_to_ghost_pose(view_location.pose, draw_view_info.local_pose); ghost_xr_draw_view_info_from_view(view, draw_view_info); /* Draw! */ @@ -401,6 +416,7 @@ XrCompositionLayerProjection GHOST_XrSession::drawLayer( XrViewLocateInfo viewloc_info = {XR_TYPE_VIEW_LOCATE_INFO}; XrViewState view_state = {XR_TYPE_VIEW_STATE}; XrCompositionLayerProjection layer = {XR_TYPE_COMPOSITION_LAYER_PROJECTION}; + XrSpaceLocation view_location{XR_TYPE_SPACE_LOCATION}; uint32_t view_count; viewloc_info.viewConfigurationType = m_oxr->view_type; @@ -416,11 +432,17 @@ XrCompositionLayerProjection GHOST_XrSession::drawLayer( "Failed to query frame view and projection state."); assert(m_oxr->swapchains.size() == view_count); + CHECK_XR( + xrLocateSpace( + m_oxr->view_space, m_oxr->reference_space, viewloc_info.displayTime, &view_location), + "Failed to query frame view space"); + r_proj_layer_views.resize(view_count); for (uint32_t view_idx = 0; view_idx < view_count; view_idx++) { drawView(*m_oxr->swapchains[view_idx], r_proj_layer_views[view_idx], + view_location, m_oxr->views[view_idx], draw_customdata); } @@ -479,7 +501,8 @@ void GHOST_XrSession::unbindGraphicsContext() { const GHOST_XrCustomFuncs &custom_funcs = m_context->getCustomFuncs(); if (custom_funcs.gpu_ctx_unbind_fn) { - custom_funcs.gpu_ctx_unbind_fn(m_context->getGraphicsBindingType(), m_gpu_ctx); + custom_funcs.gpu_ctx_unbind_fn(m_context->getGraphicsBindingType(), + (GHOST_ContextHandle)m_gpu_ctx); } m_gpu_ctx = nullptr; } diff --git a/intern/ghost/intern/GHOST_XrSession.h b/intern/ghost/intern/GHOST_XrSession.h index 3340385c1b6..ef2da61df3d 100644 --- a/intern/ghost/intern/GHOST_XrSession.h +++ b/intern/ghost/intern/GHOST_XrSession.h @@ -25,9 +25,9 @@ #include <memory> class GHOST_XrContext; +class GHOST_XrSwapchain; struct OpenXRSessionData; struct GHOST_XrDrawInfo; -struct GHOST_XrSwapchain; class GHOST_XrSession { public: @@ -74,6 +74,7 @@ class GHOST_XrSession { std::vector<XrCompositionLayerProjectionView> &r_proj_layer_views, void *draw_customdata); void drawView(GHOST_XrSwapchain &swapchain, XrCompositionLayerProjectionView &r_proj_layer_view, + XrSpaceLocation &view_location, XrView &view, void *draw_customdata); void beginFrameDrawing(); |