Getting started with the NVIDIA CloudXR Client Unity Plugin¶
This guide walks through the steps required to install, implement, and get started using the plugin within Unity. This guide is fairly straightforward, but it is sensitive deviations from the steps described.
The instructions provided are written for Meta Quest headsets (tested with the Meta Quest 2). Other headsets may work, but may require tweaks outside the scope of this guide and your results may vary.
NVIDIA CloudXR Client Unity Plugin software package.
An XR application integrated with CloudXR 4.0 or system runtime integrated with the CloudXR 4.0 Server SDK for testing.
This guide uses the CloudXR Server plugin installer for SteamVR provided within the SDK.
An OpenXR-supported headset
This guide was written using a Meta Quest 2 or Meta Quest Pro headset.
Unity version 2022.1.13f1, 2022.2.12fa (versions the plugin has been extensively tested against, later or earlier versions may or may not work)
Optional Supporting Software¶
The optional software below may be helpful in managing your Android-powered devices during installation and playback
SideQuest for mirror and device management
Meta Quest Developer Hub for managing Meta-based devices.
Unity Hub Configurations¶
Installation of the Unity game engine is managed through the Unity Hub. Ensure you have the latest version of the Unity Hub installed and it is properly licensed and running.
Open Unity Hub. Select Installs.
Choose your Unity version.
Note that recent LTS versions (2022.3) and other recent versions of Unity are likely unsupported; you may need to choose your Unity version from the Archive.
Note: This guide was written with Unity Version 2022.2.12fa, and the plugin was developed primarily with Unity Version 2022.1.13f1
Select the following:
Android Build Support (including any nested options)
Windows Build Support (IL2CPP)
Select Continue and let it install
Creating a New Unity Project¶
1. Select Projects within the Unity Hub. 1. Provide a name. 1. Select 3D URP.
You may need to select the Download Template icon if it is not already installed
Configure the Project for Android¶
This section configures the new Unity project to target Android as the build platform.
If not open already, open the newly created empty 3D (URP) Unity project.
Select File > Build Settings
Select Android and select the Switch Platform button.
Setting up Packages and Project Settings¶
The next step is to setup the project for XR within Unity. When selecting options within XR Plug-in Management, they are installed. Deselecting these options does NOT uninstall the package. The steps below use this nuance to our advantage.
Select Window > Package Manager
Select Edit > Project Settings
Within Project Settings, select XR Plugin Management and Install XR Plugin Management.
Once the plugin finishes installing, Select XR Plug-in Management again which should display the platforms to support with additional settings.
Select the tab with an icon resembling a monitor and check OpenXR. This will begin the installation of the OpenXR plugin automatically > Note: If you are prompted a warning to use a new input system, select: “Yes”
During installation Unity should reset and return with the Package Manager and Project Settings Windows open. The Package Manager now lists the OpenXR Plugin.
Within the Project Settings configuration window, under XR Plug-in Management, Select the Android Icon tab
Checkmark OpenXR. This will show a red warning which is ok, but requires additional configuration.
When done, you should only have checkboxes within OpenXR on the Desktop tab and OpenXR on the Android tab.
Within the Package Manager Window, switch your view to Unity Registry by selecting the Packages dropdown.
Select the XR Interaction Toolkit to install:
Note: This installation may prompt a message regarding XR InteractionLayerMask Update Required. If so, select “I Made a Backup, Go Ahead”
Note: If you receive an IOException Error indicating a reparse point buffer is invalid, chances are you’re using a directory that includes an incompatible character. Save your project within a different directory and try again.
Open the Project Settings window again and select Player
Verify you’re on the Android tab and expand the Other Settings section
Verify Auto Graphics API is not checked
Remove Vulkan within the Graphics API section leaving only OpenGLES3
Deselect Multithreaded Rendering
Scroll down to the Identification section and set Minimum API Level to Android 10 API Level 29
Within the Configuration section, change Scripting Backend to IL2CPP
For the Meta Quest platform (and most current Android-based platforms), within Target Architecture ensure only ARM64 is checked. Remove other checkmarks.
Within the Project Settings window, select Quality
Delete the High Fidelity and Balanced levels.
Ensure only the Performant level remains.
Within the same Project Settings page, select XR Plug-in Management.
Within the Android tab, Verify OpenXR is checked. The error indicated will be fixed in the next steps.
Expand the XR Plug-in Management panel on the left and select OpenXR.
Under the OpenXR Feature Groups, select Meta Quest Support. Unity will display an error icon which will be fixed next.
Select either the error icon or navigate to Project Validation within XR Plug-in Management.
Locate the error indicating the deprecation of the OpenXR.Input.PoseControl system and select Fix. This will take a few moments to complete.
Within XR Plug-in Management, again select OpenXR to return.
Within Interaction Profiles add the Oculus Touch Controller Profile.
Once done, any indicated errors should disappear.
Try a first build of the project¶
At this time, you’re free to design the client pre-connection lobby environment however you wish. However, to verify the configuration steps above were done properly, it’s recommended to try a generic build first.
Connect your Quest headset to your Unity developer PC through the USB-C connection. Within the headset you may be prompted to allow necessary permissions required to communicate with Unity.
Within the Unity Editor, if not already docked or open, open Build Settings within the File menu
Assuming the prior steps are complete, the Android Platform should already be set for the targeted build. Click Build to start compilation (or Build And Run to see the result on your connected HMD).
Congratulations! You should now have built an OpenXR project within Unity successfully for the Quest headset. Next steps involve implementing the CloudXR plugin and enabling XR interactivity into the project.
Import the CloudXR Plugin into Unity¶
The CloudXR Plugin for Unity comes as a tarball and includes the plugin code and sample scripts. The steps below explain how to install this into your project.
Open the Package Manger once more (Window > Package Manger)
Select the Plus Sign icon and Add package from tarball
Select the .tar file within the CloudXR Plugin package downloaded. Once complete, a new package named NVIDIA CloudXR Client for Unity will be added.
Select the Samples tab within the plugin and Import to include the sample content within the project.
Configure XR cameras/controllers and attach the CloudXR Plugin¶
The steps below assume implementation of the XR Interaction Toolkit. Your camera specifics may differ if using other package (XR Plugin Management or AR Foundation).
Within your Unity hierarchy, create a new XR Origin.
Right click within the hierarchy
Select XR > XR Origin
Delete the root/original Main Camera listed in the hierarchy. The newly created XR Origin will be the designated camera instead. A fully-generic hierarchy should look similar to the image.
Select the XR Origin object. Within the inspector change the Tracking Origin Mode to Device. This ensures the camera orientation and controllers appear properly when streaming.
Select the new Main Camera object within XR Origin > Camera Offset
Add a new Component to the object within the Inspector Panel. Select Add Component > NVIDIA > CloudXR Manager
Open the Project Settings once more (Edit > Project Settings). Select XR Plug-in Management > OpenXR. Select the Android Icon tab.
Check the CloudXR Pose Capture option within the feature group.
Congratulations! You now have an XR Unity Project with the CloudXR Plugin applied.
Adding the sample script that facilitates CloudXR server connections¶
The steps use the provided Unity sample script and shader for a project. These scripts are provided for education purposes only and are not intended for production workloads. It is strongly encouraged to review the source code and customize them for your desired client connectivity experience.
Within the project hierarchy, select Main Camera within XR Origin > Camera Offset
Within your Project browser window, expand Assets > Samples > NVIDIA CloudXR Client > 0.0.0 > CloudXR for Unity
Drag the AutoConnectCloudXR script to the Main Camera object’s inspector panel to apply.
Change the IP address to your network-reachable CloudXR test server
Add a shader to the project renderer¶
At this point CloudXR clients can connect to servers, but aren’t able to display returned frames. This adds a shader so returned frames from the server can be displayed on the client.
Within the Project browser window, select Settings > URP-Performant-Renderer
On the Project inspector, select Add Renderer Feature > CxrRenderer Feature
Find the CxrReprojection shader sample by either:
navigating within the Project browser window to Assets > Samples > NVIDIA CloudXR Client > 0.0.0 > CloudXR for Unity and dragging it over.
selecting the circle-looking icon on the Shader line and searching for CxrReprojection
Final build your project and test¶
Open your Build Settings (File > Build Settings) once more to verify Android is still the target platform
Select the Build and Run button. Either provide a new build name or overwrite the existing .apk file generated.
Once the build completes, the sample script will attempt to connect ot the IP address designated. If that IP is reachable on a network and running the correct version of CloudXR on the server or the application, the headset will Connect and stream.