Awesome
VSG OpenXR Integration
VR/XR integration layer for VulkanSceneGraph via OpenXR.
Status
Generally things are a work in progress, but functionality is suitable for simple scenes:
- vsgvr works with the main VR platforms, and the codebase is somewhat portable
- Basic rendering and presentation is functional, presented to application through an OpenXRViewer class, similar to the desktop vsg::Viewer.
- Pose bindings are available, to allow models to be bound to controller positions or similar.
- The basic
example_vr.cppapplication is similar in usage to the desktop vsg Viewer application, and some test models are provided.
Supported Hardware
In theory, any OpenXR compatible hardware or systems are supported. Some slightly different behaviour has been noted between OpenXR runtimes however.
The following configurations have been tested:
- Rendering to a virtual HMD in Monado, on Linux
- Rendering to SteamVR (HTC Vive), on Linux, and Windows
- Rendering to SteamVR + Oculus Link (Quest 2) on Windows
- Rendering to Oculus OpenXR runtime + Oculus Link (Quest 2) on Windows
- Rendering to Oculus Quest 2, native build for Android
- Rendering to the Monado runtime, on an Android phone (Moto G Pro, Android 12)
The following should work:
- All other desktop OpenXR runtimes, which use the standard OpenXR loader
| Feature | Status |
|---|---|
| Linux Build | Done |
| Windows Build | Done |
| Android Build (Oculus loader) | Done |
| Android Build (Generic loader) | Done |
| Code quality / API | Messy, but in roughly the right structure for now |
| OpenXR Rendering | Working, could do with a cleanup and better vsg integration at some point |
| OpenXR Input | Working |
| Controller tracking | Working |
| Controller models in scene | Working |
| HMD tracking | Working |
| Desktop view | Working |
OpenXR Spec and Extensions
As a minimum OpenXR requires the following OpenXR extensions:
- XR_KHR_vulkan_enable
- On Android: XR_KHR_android_create_instance
In addition the Vulkan runtime must satisfy the requirements of OpenXR. These requirements will vary but generally relate to shared memory extensions and similar.
As features are implemented vsgvr will use more extensions if available, to support common features such as additional composition layers. If specific extensions are not present, or not requested by the application these features will be inoperable.
Coordinate Spaces
In OpenXR various coordinate spaces are available, and follow a Y-up convention: x-right, y-up, z-out (backwards from user's perspective).
While it may be modified, VulkanSceneGraph by default follows a Z-up convention: x-right, y-forward, z-up.
vsgvr exposes both of these in some way, but by default follows the vsg Z-up convention:
- Any API exposing vsg::mat4, vsg::vec3, vsg::quat, etc - Z-up
- Any API exposing OpenXR types such as XrPosef - Y-up
Most classes use OpenXR types internally, but expose both vsg and OpenXR types in their interfaces. Where possible the vsg types should be used within applications.
This results in a logical mapping to the scene graph for world-based coordinate spaces (STAGE, LOCAL, VIEW).
Action spaces are mapped using an ActionPoseBinding, which presents a vsg matrix for its transform - Any action related spaces are positioned within the OpenXR Session's space, which results in the transform being mapped to the overall 'world' space of the scenne graph automatically. Advanced users may query the action space directly and re-locate against a different XrSpace if desired.
Setup
If you have a VR headset, simply build vsgvr against the appropriate OpenXR loader, and run applications.
If you don't have a VR headset there's a couple of options outlined below
SteamVR Mock HMD
SteamVR can be configured to simulate a HMD - This allows testing of rendering, though doesn't provide controller or headset tracking.
See here - Force a null driver to allow basic display output: https://developer.valvesoftware.com/wiki/SteamVR/steamvr.vrsettings
Monado
Monado can run on any Linux box pretty much, just install through your package manager and run the following.
This should display a preview of the headset, along with a few controls to simulate headset motion.
export QWERTY_ENABLE=1
export OXR_DEBUG_GUI=1
export XRT_COMPOSITOR_FORCE_XCB=1
rm /tmp/monado_comp_ipc
monado-service
Android Phone / Tablet (Monado)
Monado can also run on Android devices, providing a basic HMD-like display. I haven't tested this approach extensively, but it should be usable on most phones / tablets.
For more information see examples/android-monado.
VSGVR Compilation
Required:
- cmake > 3.14
- vulkan sdk
- VulkanSceneGraph
- The OpenXR loader - From a variety of sources
- (default) OPENXR_GENERIC - The generic OpenXR loader, included as a git submodule at deps/openxr
- OPENXR_SYSTEM - The generic OpenXR loader, from system packages
- OPENXR_OCULUS_MOBILE - The Oculus mobile SDK, available from https://developer.oculus.com/downloads/package/oculus-openxr-mobile-sdk/
- This loader is required when building for Oculus / Meta headsets - The generic OpenXR loader is non-functional on these devices
# Ensure submodules are available
git submodule update --init
# The usual CMake build
# Instead of CMAKE_PREFIX_PATH, you may set CMAKE_INSTALL_PREFIX to the same as your VulkanSceneGraph project to locate VSG
mkdir build
cd build
cmake -DCMAKE_PREFIX_PATH="VulkanSceneGraph/lib/cmake/vsg/;VulkanSceneGraph/lib/cmake/vsg_glslang" ../
make
Models
Models created in Blender
- Should face 'forward' as normal
Export from blender to gltf:
- Include custom properties
- Include punctual lights
- +Y up
Convert to vsg via vsgconv model.glb model.vsgt
- Ensure vsgXchange is built with assimp support
Development Tips
Validation layers from the OpenXR SDK
set XR_API_LAYER_PATH="C:/dev/OpenXR-SDK-Source/build/src/api_layers/"
set XR_ENABLE_API_LAYERS=XR_APILAYER_LUNARG_core_validation