NVIDIA CloudXR SDK Overview#
NVIDIA CloudXR™ is a GPU-accelerated XR streaming platform that delivers high-quality XR experiences from powerful servers to lightweight client devices over standard networks (Ethernet, Wi-Fi, or cellular).
Core Architecture: CloudXR decouples rendering from display. Computationally intensive XR applications run on high-performance servers, while users interact through lightweight clients, enabling XR on devices that cannot generate graphics for demanding content.
Components:
CloudXR Runtime: An OpenXR-compliant runtime for Windows and Linux systems that enables developers to build OpenXR-compatible server applications that stream content to remote clients
Client SDKs: Platform-specific SDKs for receiving and rendering streamed content, including CloudXR Framework for Apple native clients and CloudXR.js for web clients
Important
CloudXR is an end-to-end streaming system that requires both components to function. A server application using CloudXR Runtime must be paired with a compatible client SDK; neither can operate independently. Your server and client must both be properly configured to establish a working streaming session.
CloudXR Runtime#
CloudXR Runtime provides libraries for building OpenXR server applications that stream content to remote clients. Designed for Windows and Linux systems, it acts as an OpenXR runtime that applications use through the standard OpenXR API, automatically handling streaming functionality.
Key features:
OpenXR Runtime: The complete OpenXR API implementation. Applications use the standard API without modification while streaming to remote clients.
Video and audio streaming: GPU-accelerated video encoding for high-quality visuals. Audio captured and streamed alongside video.
Note
Audio streaming is currently supported in Windows only. Linux support is planned for a future release.
Client input integration: Receives and processes client input (e.g. head tracking, controllers, hand tracking) via the standard OpenXR action system.
Advanced features:
Foveated rendering: Quality reduction in peripheral vision areas to optimize bandwidth
Opaque data channel: Custom bidirectional communication channel for application-specific data using the XR_NV_opaque_data_channel extension
Platforms: Windows, Linux
Integration: CloudXR is provided as libraries. Once integrated, applications use the standard OpenXR API; CloudXR Runtime handles streaming automatically.
CloudXR Client SDKs#
The CloudXR client SDKs enable devices to receive and render streamed XR content from applications built with CloudXR Runtime. They are provided in platform-specific implementations optimized for each target device ecosystem.
CloudXR Framework (Apple)#
CloudXR Framework is a native Swift framework for Apple clients streaming from CloudXR Runtime servers. It uses native stream protocol with seamless SwiftUI and RealityKit integration.
Key features:
Native integration: Swift-based SwiftUI applications with full RealityKit support (visionOS/iOS)
Session management: A simple API for local network and cloud connections
Hardware-accelerated rendering: Optimized video decoding via Metal and Apple Silicon
Tracking: 6DOF head (ARKit), hand tracking (visionOS), and Bluetooth controllers (DualSense)
Low-latency streaming: Optimized frame timing and network handling
Opaque data channel: Custom bidirectional communication
Development tools: Streaming statistics HUD, network diagnostics
Platforms: visionOS 2.4+, iOS 18.0+
Distribution: Swift Package, including DocC documentation with tutorials
CloudXR.js (Web, Early Access)#
Note
Early Access: Features and APIs may change. Refer to CloudXR.js (Early Access) for access.
A JavaScript framework for web clients streaming from CloudXR Runtime servers. It uses WebRTC protocol with WebXR integration for browser-based immersive experiences.
Key features:
WebXR integration: Native WebXR Device API for VR/AR in browsers
Session management: A simple connection API
Framework agnostic: Works with vanilla WebGL, Three.js, React Three Fiber, or any WebXR-compatible framework
Cross-platform: A single client runs on Meta Quest 3, Pico 4 Ultra, and desktop browsers without modification
Opaque data channel: Custom bidirectional communication
Low-latency WebRTC: Streaming compliant with modern web standards
Development Tools: HTTPS support, WebSocket proxy for production
Platforms: Meta Quest 3 (OS 79+), Pico 4 Ultra (Pico OS 15.4.4U+), desktop browsers (with IWER for WebXR emulation)
Distribution: npm package, including API documentation and examples