Release Notes

Version 4.0

  • Updated how app paths are given to CloudXR, it is now passed in a ‘params’ structure, and not through its own API. See CloudXR File Storage. This is most important for Android, in which most recent OSes now ‘sandbox’ app file access, and thus the simplest solution is to use the app-specific data folder. See Android Scoped Storage for more.

  • Reversed overall logging approach. Removed API to send app messages into CloudXR, replacing it with a new client callback to receive messages from CloudXR. See description of the revised CloudXR logging for implementation details.

  • Overhauled input/controller handling with new forward-looking APIs and approach. See CloudXR Input System for more information.

  • Added option for client to tag pose updates with a unique ID, by setting the cxrHmdTrackingFlags_HasPoseID flag and the 64-bit poseID member in cxrHmdTrackingState. Later, the cxrFramesLatched struct will return back to the client app which poseID was used in rendering the current video frames.

  • Improved pose prediction algorithm.

  • Improved flexibility of video stream setup/configuration. New APIs to explicitly define streams needed. See Client Setup for more details.

  • Added support for enabling L4S/ECN congestion handling for 5G networks. See CloudXR Network Setup for more information.

  • Improved error codes, messages, and reporting throughout the SDK.

  • Added experimental server sample application for Windows, that talks to CloudXR directly and doesn’t need SteamVR. See CloudXR Experimental Server Sample.

  • Deprecated ImageReader decoder due to stability issues across devices. Will revisit in future release.

  • Removed support for experimental SXR decoder from the Android client library.

  • Removed support from the CloudXR Server for pre-configured client device descriptions. Will look to add similar functionality in a future release.

  • Removed WaveVR sample from the SDK, as HTC has a first-party sample client on github: HTC CloudXR Client Sample

Internal Release

Note

This was an NVIDIA internal-only release on the 3.x branch, that we’ve kept separate here for legibility.

  • QoS optimizations for VR streaming to prevent quality differences in left and right eye video.

  • Added option for client to send scale factor to server by setting cxrHmdTrackingFlags_HasScaling flag and specifying the scale factor in the new scaling member of cxrHmdTrackingState.

  • Fixed bug that was causing a hang when resuming the iOS client.

  • Fixed issue that caused video to look dark with Android ARCore client, due to missing cxrDebugFlags_OutputLinearRGBColor in client descriptor flags.

  • Fixed bug in Windows client where PNGs saved using -dump-images were distorted.

  • Fixed bug that caused Android client logs to be truncated.

  • 90hz support for Oculus Quest 2 sample client no longer auto-enables the AImageReader decoder.

  • Removed the deprecated iOS Objective-C client.

  • Changes to handling of client app data directory. See CloudXR File Storage for identifying the client application data folder, and new requirements specific to recent Android changes.

  • Added new section How to Launch on Android that brings together information across the docs, and call out changes to using launch option files on Android devices.

Version 3.2

  • New Client API to query real-time QoS and network feedback.

  • New Client API to specify network interface and topology so CloudXR can select optimized QoS settings.

  • 90hz support for Oculus Quest 2 sample client.

    • Auto-enables the AImageReader decoder.

  • Major stability improvements to the optimized Android AImageReader decoder.

  • New flag for Android clients to request linear or sRGB output. Helps address issues with colors being too dark or too bright.

  • Improved color output on Focus 3 by using linear swapchain instead of sRGB.

  • New iOS ARKit sample client written in Swift.

    • Feature parity with Objective C client.

    • Objective C client is deprecated and will be removed in future public releases.

  • iOS sample client now publishes its logs to a shared location to allow easy access with ‘Files’ application.

  • cxrTrackedDevicePose now uses a vector + quaternion to represent pose transformation, instead of a cxrMatrix34.

  • Clients can optionally send a user-defined 64-bit ID with pose updates. The server will return that 64-bit ID with the frame rendered from that pose.

  • Clients can optionally update the headset projection parameters and send revised IPD in pose updates.

  • Server can be pre-configured for a client device to allow starting server application prior to any client connecting.

  • Removed auto-reconnect from sample clients when connection fails.

  • Removed support for cloud anchors in ARCore sample.

Version 3.1.1

  • Improved stability for Oculus Quest 2 and HTC VIVE Focus 3 sample applications.

  • Fixed iOS sample client library paths.

  • Removed the default Cloud API key from ARCore sample manifest.

Version 3.1

  • Oculus client sample is now updated to Oculus Mobile SDK 1.46.

  • ARCore client sample is now updated to ARCore SDK 1.26.

  • WaveVR client sample is now updated to WaveVR SDK 4.1.1.

  • Android 64-bit builds are now the default, and 32-bit support has been deprecated.

  • HTC VIVE Focus 3 support has been added to the WaveVR client, and HTC VIVE Focus Plus support has been deprecated.

  • The pre-compiled Oculus client .apk has been removed from CloudXR SDK package.

  • Android headset sRGB handling has been improved for both Oculus and WaveVR clients, and should now deliver expected visuals.

  • Android headsets should now exit in cases where they’d seemingly hang. This includes server not responding and server disconnected cases.

  • Other various bug fixes.

Version 3.0

  • Updated the iOS ARKit sample client.

    Improvements include the following:

    • Fixed viewport rotation warping issues.

    • Detected planes are visualized to make world origin selection easier.

    • Short-tap on plane sets world origin to location of touch.

    • Long-tap resets world origin.

    • Added sliders for rotation and scaling.

    • Fixed other issues to improve experience.

  • Added an audio back-channel to send client input audio (for example, microphone) to the server.

  • Fixed support for custom audio playback on Android via the RenderAudio client callback.

  • Updated Android samples to playback and capture audio using the Google Oboe audio library.

  • Fixed color and gamma handling in Wave VR client.

    Updated the Oculus client to use correct color space conversion matrix.

  • Fixed the transparency issue in the ARCore client where opaque areas were semi-transparent.

  • Added launch/command-line options to dump the server and client video bitstreams.

  • Fixed intermittent video corruption on Android devices when using -tle (Trace Local Events) logging.

  • Updated foveation support with custom 6x6 downsample filter for improved quality

  • Updated the Windows SDK from 8.1 to 10.0.17763.0.

  • Restricted the Oculus VR sample to 72hz currently.

    Will support higher refresh rates in future update.

  • Removed older virtual audio device instances when performing a new CloudXR server installation.

  • Multiple API changes.

    See CloudXR SDK API for more information.

    The changes include the following:

    • Added cxrSendAudio() for a new audio back-channel.

    • Added cxrConnectionFlags parameter to cxrConnect() to select between the synchronous and asynchronous connections.

      An asynchronous connect occurs in a background thread, which keeps the clients responsive during an attempt to establish a connection.

    • Added a UpdateClientState callback to notify the client when a connection state change occurs.

      It also provides a reason for connection failure or unexpected disconnect. Removed redundant Disconnect() client callback.

    • cxrLatchFrame(), cxrReleaseFrame(), cxrBlitFrame() have been revised and unified around new cxrFramesLatched structure.

      Older cxrLatchFrameXR() and cxrReleaseFrameXR() are removed, use the new unified functions instead. ARCore client now must render video frames using cxrBlitFrame() like the Android VR clients.

    • Removed the unused cxrIsRunning().

    • Removed cxrSendChaperone().

      The cxrChaperone structure is now part of the cxrDeviceDesc structure, and passed to the server upon a successful connection.

    • Added client and server support for cxrControllerType_None for headsets without controllers.

  • Various other minor bug fixes and optimizations.

  • In keeping with the highest standards for CloudXR, CloudXR 3.0 will not support the Microsoft HoloLens 2 (HL2).

    The previously provided HL2 sample application and supporting client library have been removed.

Version 2.1

  • Added an iOS client sample based on Apple ARKit for CloudXR AR streaming.

    Currently, only H.264 is supported.

  • Updated the WaveVR client sample to WaveVR SDK 3.2.0.

  • Updated the Windows client sample to use SDL version 2.0.14.

  • Updated the Android client samples to use Gradle version 6.1.1.

  • The fix for building a Windows client that has dependencies on some DLLs that were not properly included.

  • Tweaked streaming parameters to better handle poor networks with high jitter.

  • Video bitstreams now signal sRGB transfer characteristics in bitstream header.

  • Various minor bug fixes and optimizations.

  • The CloudXR server installer now also opens the UDP ports 47999, 48000, 47998, 48005, and 48002 as a temporary workaround while we address connectivity issues that were experienced with some client devices.

Known Issues

Server Crashing on Reconnect

If you run the server with the -tle launch option (to capture server event trace), connect and disconnect a client, then attempt to reconnect the client, the server will crash. The issue is understood and will be fixed in a subsequent release. If you need to capture traces, please restart the server after a client disconnects.

Changing Connected Clients

The CloudXR server caches the device descriptor of the first connected device, so the server must be restarted when changing to different controllers or a device with a different cxrDeviceDesc. It will write to the log that there was a mismatch in device descriptor, and terminate the connection.

Microsoft Remote Desktop

Microsoft Remote Desktop is not compatible with CloudXR. See Remote Desktop Software for more information.

Inconsistent sRGB handling on Android

Certain Android devices have exhibited issues displaying sRGB output, resulting in banding or a darkened image. In order to mitigate this a new debug flag cxrDebugFlags_OutputLinearRGBColor has been added which will allow SDK users to switch from sRGB to linear output.

Lifecycle issues on iOS

Suspending and resuming the iOS sample application while connected to a server does not work as expected. In order to workaround this the iOS client must be force quit before attempting to reconnect by sliding it off the app carousel.

SteamVR Safe Mode

Occasionally, you might see a Safe Mode issue on the server. This error is specific to SteamVR, and here is a workaround:

  1. Switch SteamVR to Beta on the server.

  2. Continue to click Disable Safe Mode until this dialog disappears.

  3. To start streaming, follow this exact sequence:

    1. Start SteamVR on server.

    2. Connect Client.

    3. Start VR app on server.

If you do not strictly follow the order, you might face issues.

SteamVR Bone Input

These games use bone-based input techniques that CloudXR does not currently support and will not work properly:

  • Boneworks

  • Pavlov VR

HTC VIVE Focus 3 Controller Support

Note that the controller support for the Focus 3 is currently mapped through the ‘Oculus Touch’ profile. This means that input should closely match Quest 2 in mapping and overall funtionality, however it has the side effect that when using the Focus 3 with CloudXR, you will see the Oculus Touch controller models. The CloudXR team is working with HTC on the proper long-term solution to fully support their controllers.

Low-Latency Audio Support on Android

Some Android devices don’t support Oboe low-latency mode, and to run properly you would need to disable that mode in your client either in general or when specifically detecting a device determined to have audio processing issues.