Getting started with the NVIDIA CloudXR Client Unity Plugin

This guide walks through the steps required to install, implement, and get started using the plugin within Unity. This guide is fairly straightforward, but it is sensitive deviations from the steps described.

The instructions provided are written for Meta Quest headsets (tested with the Meta Quest 2). Other headsets may work, but may require tweaks outside the scope of this guide and your results may vary.

Requirements

  • NVIDIA CloudXR Client Unity Plugin software package.

  • An XR application integrated with CloudXR 4.0 or system runtime integrated with the CloudXR 4.0 Server SDK for testing.

    This guide uses the CloudXR Server plugin installer for SteamVR provided within the SDK.

  • An OpenXR-supported headset

    This guide was written using a Meta Quest 2 or Meta Quest Pro headset.

  • Unity version 2022.1.13f1, 2022.2.12fa (versions the plugin has been extensively tested against, later or earlier versions may or may not work)

Optional Supporting Software

The optional software below may be helpful in managing your Android-powered devices during installation and playback

  • SideQuest for mirror and device management

  • Meta Quest Developer Hub for managing Meta-based devices.

Unity Hub Configurations

Installation of the Unity game engine is managed through the Unity Hub. Ensure you have the latest version of the Unity Hub installed and it is properly licensed and running.

  1. Open Unity Hub. Select Installs.

  2. Choose your Unity version.

    Note that recent LTS versions (2022.3) and other recent versions of Unity are likely unsupported; you may need to choose your Unity version from the Archive.

    Note: This guide was written with Unity Version 2022.2.12fa, and the plugin was developed primarily with Unity Version 2022.1.13f1

  3. Select the following:

    • Android Build Support (including any nested options)

    • Windows Build Support (IL2CPP)

  4. Select Continue and let it install

Creating a New Unity Project

1. Select Projects within the Unity Hub. 1. Provide a name. 1. Select 3D URP.

You may need to select the Download Template icon if it is not already installed

Creating a Unity image

Creating a Unity image

Configure the Project for Android

This section configures the new Unity project to target Android as the build platform.

  1. If not open already, open the newly created empty 3D (URP) Unity project.

  2. Select File > Build Settings

  3. Select Android and select the Switch Platform button. Setting Android Build

Setting up Packages and Project Settings

The next step is to setup the project for XR within Unity. When selecting options within XR Plug-in Management, they are installed. Deselecting these options does NOT uninstall the package. The steps below use this nuance to our advantage.

  1. Select Window > Package Manager

  2. Select Edit > Project Settings

  3. Within Project Settings, select XR Plugin Management and Install XR Plugin Management.

  4. Once the plugin finishes installing, Select XR Plug-in Management again which should display the platforms to support with additional settings.

  5. Select the tab with an icon resembling a monitor and check OpenXR. This will begin the installation of the OpenXR plugin automatically > Note: If you are prompted a warning to use a new input system, select: “Yes” OpenXR Input Warning

  6. During installation Unity should reset and return with the Package Manager and Project Settings Windows open. The Package Manager now lists the OpenXR Plugin.

  7. Within the Project Settings configuration window, under XR Plug-in Management, Select the Android Icon tab

  8. Checkmark OpenXR. This will show a red warning which is ok, but requires additional configuration. Android OpenXR set

  9. When done, you should only have checkboxes within OpenXR on the Desktop tab and OpenXR on the Android tab.

  10. Within the Package Manager Window, switch your view to Unity Registry by selecting the Packages dropdown. Unity Registry Selector

  11. Select the XR Interaction Toolkit to install:

    Note: This installation may prompt a message regarding XR InteractionLayerMask Update Required. If so, select “I Made a Backup, Go Ahead”

    XRInteractionLayerMaskWarning

    XRInteractionLayerMaskWarning

    Note: If you receive an IOException Error indicating a reparse point buffer is invalid, chances are you’re using a directory that includes an incompatible character. Save your project within a different directory and try again.

  12. Open the Project Settings window again and select Player

    1. Verify you’re on the Android tab and expand the Other Settings section

      1. Verify Auto Graphics API is not checked

      2. Remove Vulkan within the Graphics API section leaving only OpenGLES3

      3. Deselect Multithreaded Rendering ProjectSettingsOptions

      4. Scroll down to the Identification section and set Minimum API Level to Android 10 API Level 29

      5. Within the Configuration section, change Scripting Backend to IL2CPP

      6. For the Meta Quest platform (and most current Android-based platforms), within Target Architecture ensure only ARM64 is checked. Remove other checkmarks. ProjectSettingsBottom

  13. Within the Project Settings window, select Quality

    1. Delete the High Fidelity and Balanced levels.

    2. Ensure only the Performant level remains. renderquality

  14. Within the same Project Settings page, select XR Plug-in Management.

    1. Within the Android tab, Verify OpenXR is checked. The error indicated will be fixed in the next steps.

    2. Expand the XR Plug-in Management panel on the left and select OpenXR.

      1. Under the OpenXR Feature Groups, select Meta Quest Support. Unity will display an error icon which will be fixed next. xrpluginmetaquest

    3. Select either the error icon or navigate to Project Validation within XR Plug-in Management.

      1. Locate the error indicating the deprecation of the OpenXR.Input.PoseControl system and select Fix. This will take a few moments to complete. projectvalidationfix

    4. Within XR Plug-in Management, again select OpenXR to return.

      1. Within Interaction Profiles add the Oculus Touch Controller Profile. openxroculuscontrollerprofile

      2. Once done, any indicated errors should disappear.

Try a first build of the project

At this time, you’re free to design the client pre-connection lobby environment however you wish. However, to verify the configuration steps above were done properly, it’s recommended to try a generic build first.

  1. Connect your Quest headset to your Unity developer PC through the USB-C connection. Within the headset you may be prompted to allow necessary permissions required to communicate with Unity.

  2. Within the Unity Editor, if not already docked or open, open Build Settings within the File menu

  3. Assuming the prior steps are complete, the Android Platform should already be set for the targeted build. Click Build to start compilation (or Build And Run to see the result on your connected HMD).

Congratulations! You should now have built an OpenXR project within Unity successfully for the Quest headset. Next steps involve implementing the CloudXR plugin and enabling XR interactivity into the project.

Import the CloudXR Plugin into Unity

The CloudXR Plugin for Unity comes as a tarball and includes the plugin code and sample scripts. The steps below explain how to install this into your project.

  1. Open the Package Manger once more (Window > Package Manger)

  2. Select the Plus Sign icon and Add package from tarball

  3. Select the .tar file within the CloudXR Plugin package downloaded. Once complete, a new package named NVIDIA CloudXR Client for Unity will be added. addpluginpackage

  4. Select the Samples tab within the plugin and Import to include the sample content within the project. xrsamples

Configure XR cameras/controllers and attach the CloudXR Plugin

The steps below assume implementation of the XR Interaction Toolkit. Your camera specifics may differ if using other package (XR Plugin Management or AR Foundation).

  1. Within your Unity hierarchy, create a new XR Origin.

    • Right click within the hierarchy

    • Select XR > XR Origin

  2. Delete the root/original Main Camera listed in the hierarchy. The newly created XR Origin will be the designated camera instead. A fully-generic hierarchy should look similar to the image. xrhierarchy

  3. Select the XR Origin object. Within the inspector change the Tracking Origin Mode to Device. This ensures the camera orientation and controllers appear properly when streaming. xrtrackingmode

  4. Select the new Main Camera object within XR Origin > Camera Offset

  5. Add a new Component to the object within the Inspector Panel. Select Add Component > NVIDIA > CloudXR Manager XRAddComponent

  6. Open the Project Settings once more (Edit > Project Settings). Select XR Plug-in Management > OpenXR. Select the Android Icon tab.

  7. Check the CloudXR Pose Capture option within the feature group.

Congratulations! You now have an XR Unity Project with the CloudXR Plugin applied.

Adding the sample script that facilitates CloudXR server connections

The steps use the provided Unity sample script and shader for a project. These scripts are provided for education purposes only and are not intended for production workloads. It is strongly encouraged to review the source code and customize them for your desired client connectivity experience.

  1. Within the project hierarchy, select Main Camera within XR Origin > Camera Offset

  2. Within your Project browser window, expand Assets > Samples > NVIDIA CloudXR Client > 0.0.0 > CloudXR for Unity

  3. Drag the AutoConnectCloudXR script to the Main Camera object’s inspector panel to apply. upiaddsample

  4. Change the IP address to your network-reachable CloudXR test server

Add a shader to the project renderer

At this point CloudXR clients can connect to servers, but aren’t able to display returned frames. This adds a shader so returned frames from the server can be displayed on the client.

  1. Within the Project browser window, select Settings > URP-Performant-Renderer

  2. On the Project inspector, select Add Renderer Feature > CxrRenderer Feature urprenderer

  3. Find the CxrReprojection shader sample by either:

    1. navigating within the Project browser window to Assets > Samples > NVIDIA CloudXR Client > 0.0.0 > CloudXR for Unity and dragging it over.

    OR rather

    1. selecting the circle-looking icon on the Shader line and searching for CxrReprojection

cxrreprojection

cxrreprojection

Final build your project and test

  1. Open your Build Settings (File > Build Settings) once more to verify Android is still the target platform

  2. Select the Build and Run button. Either provide a new build name or overwrite the existing .apk file generated.

Once the build completes, the sample script will attempt to connect ot the IP address designated. If that IP is reachable on a network and running the correct version of CloudXR on the server or the application, the headset will Connect and stream.