Session API#

The CloudXR.js Session API provides comprehensive management of streaming sessions with CloudXR Runtime. This guide covers lifecycle, configuration, key operations, event handling, and integration patterns.

Session Lifecycle#

CloudXR sessions follow this lifecycle:

Initialized -> Connecting -> Connected -> Disconnecting -> Disconnected

The runtime can also transition to an error state if connection or streaming fails.

Session States#

  • Initialized: Session created but not yet connected

  • Connecting: Attempting to establish connection with CloudXR Runtime

  • Connected: Active streaming session with video streaming

  • Disconnecting: Session is in the process of disconnecting

  • Disconnected: Session has been disconnected and can reconnect

  • Error: Session encountered a critical error and cannot continue

Creating a Session#

import { createSession } from '@nvidia/cloudxr';

const session = createSession({
  serverAddress: 'your-server',
  serverPort: 49100,
  useSecureConnection: false,
  gl: webglContext,
  perEyeWidth: 2048,
  perEyeHeight: 1792,
  referenceSpace: xrReferenceSpace,
  deviceFrameRate: 90,
  maxStreamingBitrateKbps: 150000
});

Note

perEyeWidth and perEyeHeight define per-eye stream resolution. perEyeWidth must be a multiple of 16, and perEyeHeight must be a multiple of 64.

Advanced Connection Options#

CloudXR.js supports additional options for NAT, proxies, and authenticated signaling:

  • mediaAddress and mediaPort for media endpoints that differ from signaling endpoints

  • signalingQueryParameters for app-level auth/session query strings

  • signalingResourcePath for proxy-based routing to target runtimes

  • iceServers for STUN/TURN in restrictive network environments

Option

Description

mediaAddress

Media endpoint IP for NAT or split signaling/media routing. When using this option, the CloudXR Runtime enable-ice property must be set to false or the connection will fail. See enable-ice for details.

mediaPort

Media endpoint port. Use 0 for server-provided port (requires mediaAddress).

signalingQueryParameters

Query string appended to signaling URL (for auth/session context).

signalingResourcePath

Proxy resource path used for target runtime routing.

iceServers

STUN/TURN configuration for restrictive NAT/firewall environments.

ICE Server Configuration (STUN/TURN)#

const session = createSession({
  serverAddress: 'your-server',
  serverPort: 49100,
  useSecureConnection: true,
  iceServers: {
    iceServers: [
      { urls: 'stun:stun.example.com:3478' },
      {
        urls: 'turn:turn.example.com:3478',
        username: 'user',
        credential: 'password'
      }
    ],
    iceTransportPolicy: 'all'
  },
  gl: webglContext,
  perEyeWidth: 2048,
  perEyeHeight: 1792,
  referenceSpace: xrReferenceSpace
});

Proxy Routing Example#

const session = createSession({
  serverAddress: 'proxy.example.com',
  serverPort: 443,
  useSecureConnection: true,
  signalingResourcePath: '/192.168.1.100',
  gl: webglContext,
  perEyeWidth: 2048,
  perEyeHeight: 1792,
  referenceSpace: xrReferenceSpace
});

// Generates: wss://proxy.example.com:443/192.168.1.100/sign_in?...

Core Operations#

  • session.connect() starts signaling and media negotiation

  • session.disconnect() cleanly tears down streaming

  • session.sendTrackingStateToServer(...) submits head/hand/controller tracking

  • session.render(...) renders decoded CloudXR frames to your XR layer

  • session.sendServerMessage(...) sends app-specific messages to the runtime

Session Management Examples#

try {
  session.connect();
  console.log('Session connection initiated');
} catch (error) {
  console.error('Failed to initiate connection:', error.message);
}

session.disconnect();

const ok = session.sendTrackingStateToServer(timestamp, xrFrame);
if (!ok) {
  console.warn('Failed to send tracking state');
}

session.render(timestamp, xrFrame, xrWebGLLayer);

try {
  session.sendServerMessage(messageObject);
} catch (error) {
  console.error('Failed to send server message:', error.message);
}

Event Handling#

Configure delegates when creating a session to respond to stream lifecycle and integration events:

const delegates = {
  onStreamStarted: () => {
    console.log('Streaming started');
  },
  onStreamStopped: (error) => {
    if (error) {
      console.log('Streaming stopped due to error:', error);
    } else {
      console.log('Streaming stopped normally');
    }
  },
  onWebGLStateChangeBegin: () => {},
  onWebGLStateChangeEnd: () => {}
};

const session = createSession(options, delegates);

WebXR Integration#

A typical CloudXR.js integration follows this pattern: request a WebXR session, obtain the WebGL context and reference space from that session, create a CloudXR session with those objects, connect, and enter the render loop.

// 1. Request a WebXR immersive session
const xrSession = await navigator.xr.requestSession('immersive-vr', {
  requiredFeatures: ['local-floor']
});

// 2. Obtain WebGL context and reference space
const gl = xrSession.renderState.baseLayer.context;
const referenceSpace = await xrSession.requestReferenceSpace('local-floor');

// 3. Create CloudXR session
const cloudxrSession = createSession({
  serverAddress: 'your-server',
  serverPort: 49100,
  useSecureConnection: false,
  gl: gl,
  perEyeWidth: 2048,
  perEyeHeight: 1792,
  referenceSpace: referenceSpace,
  deviceFrameRate: 90,
  maxStreamingBitrateKbps: 150000
}, {
  onStreamStarted: () => console.log('CloudXR streaming started'),
  onStreamStopped: (error) => console.log('CloudXR streaming stopped', error)
});

// 4. Connect to CloudXR Runtime
try {
  cloudxrSession.connect();
} catch (error) {
  console.error('Failed to initiate CloudXR connection:', error.message);
}

// 5. Render loop
function renderFrame(time, frame) {
  cloudxrSession.sendTrackingStateToServer(time, frame);
  cloudxrSession.render(time, frame, xrSession.renderState.baseLayer);
  xrSession.requestAnimationFrame(renderFrame);
}

xrSession.requestAnimationFrame(renderFrame);

Controller and Hand Tracking#

CloudXR automatically handles controller input and hand tracking through the WebXR session. All tracking data is sent to the server when you call sendTrackingStateToServer() with the current XR frame:

function renderFrame(time, frame) {
  // Automatically includes all available input data
  cloudxrSession.sendTrackingStateToServer(time, frame);
  cloudxrSession.render(time, frame, xrWebGLLayer);
}

The server receives:

  • Controller button presses and trigger values

  • Controller pose and orientation

  • Hand tracking joint data (if supported by the device)

  • Head tracking and viewer pose

Error Handling#

Use both delegate callbacks and operation-level error handling:

try {
  session.connect();
} catch (error) {
  console.error('Failed to initiate connection:', error.message);
}

if (!session.sendTrackingStateToServer(timestamp, frame)) {
  console.warn('Cannot send tracking state - session not connected');
}

Monitoring Session State#

console.log('Current state:', session.state);

Performance Monitoring#

For profiling and debugging WebXR application performance:

See Performance Guide for CloudXR.js-specific tuning parameters (resolution, foveation, bitrate, frame rate).