CloudXR SDK API

This is a high-level reference of the CloudXR functions, structures, and variables. For deeper details in how to put this into use, please see the section on Client Development.

API Functions

cxrError cxrCreateReceiver(const cxrReceiverDesc *description, cxrReceiverHandle *receiver)

Initialize the CloudXR client and create a Receiver handle to be used by all other API calls.

Note

The returned cxrReceiverHandle must be passed as the first parameter to all other CloudXR client API calls.

Parameters

  • description – [in] A filled-out client receiver descriptor struct (see cxrReceiverDesc).

  • receiver – [out] A pointer to return back to the client an opaque handle representing the Receiver interface.

Returns cxrError_Required_Parameter

One of the required parameters was missing

Returns cxrError_Invalid_API_Version

The requested version of the client library was invalid

Returns cxrError_Invalid_Number_Of_Streams

The requested number of streams was invalid for the requested mode

Returns cxrError_Invalid_Device_Descriptor

The provided device descriptor is invalid

Returns cxrError_Invalid_Graphics_Context

The graphics context is not valid for the OS/platform

Returns cxrError_Decoder_Setup_Failed

The platform specific decoder could not be instantiated

cxrError cxrConnect(cxrReceiverHandle receiver, const char *serverAddr, cxrConnectionDesc *description)

Establish a connection to a CloudXR server.

Parameters

  • receiver – [in] cxrReceiverHandle handle that was obtained from cxrCreateReceiver().

  • serverAddr – [in] IP address of the CloudXR server.

  • description – [in] Optional settings that control connection behavior (see cxrConnectionDesc).

Returns cxrError_Required_Parameter

One of the required parameters was missing

Returns cxrError_Not_Connected

The receiver has initiated an asynchronous connection which has not completed yet

Returns cxrError_Module_Load_Failed

The streamer library could not be loaded

Returns cxrError_Client_Setup_Failed

The streamer library could not be initialized

Returns cxrError_Client_Version_Old

The client version is too old for the server

Returns cxrError_Client_Unauthorized

The client authorization header was rejected

Returns cxrError_Server_Handshake_Failed

The client could not complete the handshake with the server (This could be due to an incorrect address, a blocked RTSP port or a temporary network interruption)

Returns cxrError_Server_Version_Old

The server version is too old for the client

Returns cxrError_Server_Feature_Disabled_*

The server disabled the requested feature (HEVC, VVSync, Pose Prediction, Send Audio or Receive Audio)

void cxrDestroyReceiver(cxrReceiverHandle receiver)

Terminate a streaming session and free the associated CloudXR resources.

Note

The Receiver handle is invalid upon return from this call.

Parameters
cxrError cxrLatchFrame(cxrReceiverHandle receiver, cxrFramesLatched *framesLatched, uint32_t frameMask, uint32_t timeoutMs)

Acquire the next available decoded video frames from network stream.

This call attempts to acquire the next available decoded video frames for all streams in streamMask, together in lock-step. It can be called repeatedly with different masks in order to acquire frames without concurrent locking.

Parameters

  • receiver – [in] cxrReceiverHandle handle from cxrCreateReceiver()

  • framesLatched – [inout] pointer to a cxrFramesLatched struct with a lifetime until cxrReleaseFrame.

  • frameMask – [in] a bitfield mask set to 1<<[frame number] for generic frames, or use special values like cxrFrameMask_All.

  • timeoutMs – [in] number of milliseconds to wait for frame(s) before returning in error.

Returns cxrError_Required_Parameter

One of the required parameters was missing

Returns cxrError_Not_Connected

The receiver has not connected yet

Returns cxrError_Not_Streaming

The server is not streaming yet

Returns cxrError_Frames_Not_Released

There are previously latched frames that have not yet been released

Returns cxrError_Frames_Not_Ready

A frame could not be latched within the requested timeout

Returns cxrError_Frame_Not_Latched

There was a platform specific decoder issue

Returns cxrError_Decoder_No_Texture

The decoder could not acquire a texture to decode into

Returns cxrError_Decoder_Frame_Not_Ready

The decoded frame was not ready to be latched

cxrError cxrBlitFrame(cxrReceiverHandle receiver, cxrFramesLatched *framesLatched, uint32_t frameMask)

ANDROID-ONLY Render a masked set of the latched video frame(s) to the currently bound target surface.

Parameters
Returns cxrError_Required_Parameter

One of the required parameters was missing

Returns cxrError_Invalid_Number_Of_Streams

The frames latched count or the frame mask contained invalid values

cxrError cxrReleaseFrame(cxrReceiverHandle receiver, cxrFramesLatched *framesLatched)

Release a previously latched set of video frames from cxrLatchFrame.

Note

The cxrFramesLatched data is invalid upon return from this call.

Parameters
Returns cxrError_Required_Parameter

One of the required parameters was missing

Returns cxrError_Frames_Not_Latched

There are no latched frames to release

cxrError cxrAddController(cxrReceiverHandle receiver, const cxrControllerDesc *desc, cxrControllerHandle *outHandle)

Adds a controller.

Parameters

  • receiver – [in] cxrReceiverHandle handle from cxrCreateReceiver()

  • desc – [in] structure to identify/describe this controller and the inputs it produces.

  • outHandle – [inout] pointer to a cxrControllerHandle that receives a handle for the newly added controller.

Returns cxrError_Required_Parameter

One of the required parameters was missing

Returns cxrError_Controller_Id_In_Use

The id field in controller description has already been used for another controller

Returns cxrError_Role_Too_Long

Description’s role is longer than CXR_MAX_CONTROLLER_ROLE

Returns cxrError_Name_Too_Long

Description’s name is longer than CXR_MAX_CONTROLLER_NAME, or an input path is longer than CXR_MAX_INPUT_PATH_LENGTH

Returns cxrError_Too_Many_Inputs

Description’s input count is more than CXR_MAX_CONTROLLER_INPUT_COUNT

cxrError cxrSendControllerPoses(cxrReceiverHandle receiver, uint32_t poseCount, const cxrControllerHandle *controllerHandles, const cxrControllerTrackingState *const *states)

Sends a number of poses for one or more controllers.

Parameters

  • receiver – [in] cxrReceiverHandle handle from cxrCreateReceiver()

  • poseCount – [in] number of poses to send.

  • controllerHandles – [in] array containing poseCount controller handles.

  • states – [in] array containing poseCount controller tracking states.

Returns cxrError_Required_Parameter

One of the required parameters was missing

cxrError cxrFireControllerEvents(cxrReceiverHandle receiver, cxrControllerHandle controller, const cxrControllerEvent *events, uint32_t eventCount)

Sends events for a controller.

Parameters

  • receiver – [in] cxrReceiverHandle handle from cxrCreateReceiver()

  • controller – [in] handle of the controller.

  • events – [in] array containing eventCount controller events.

  • eventCount – [in] number of events to send.

Returns cxrError_Required_Parameter

One of the required parameters was missing

Returns cxrError_Data_Too_Large

A blob value in an action excceds CXR_MAX_BLOB_BYTES_PER_INPUT bytes

cxrError cxrRemoveController(cxrReceiverHandle receiver, cxrControllerHandle handle)

Removes a controller.

Parameters

  • receiver – [in] cxrReceiverHandle handle from cxrCreateReceiver()

  • handle – [in] handle of the controller to remove.

Returns cxrError_Required_Parameter

One of the required parameters was missing

cxrError cxrSendLightProperties(cxrReceiverHandle receiver, const cxrLightProperties *lightProps)

Provide estimated world lighting properties to the server.

This is used to send ARCore or ARKit lighting estimates to the server, for live integration into 3D rendered scene lighting. It supports a primary light source as well as ambient spherical harmonics. On the server side, apps can query this data using: 

   IVRSystem::GetArrayTrackedDeviceProperty(cxrTrackedDeviceProperty)

Returns cxrError_Required_Parameter

One of the required parameters was missing

Returns cxrError_Not_Connected

The receiver has not connected yet

cxrError cxrSendInputEvent(cxrReceiverHandle receiver, const cxrInputEvent *inputEvent)

Send non-VR-controller input events to the server for app-specific handling.

Returns cxrError_Required_Parameter

One of the required parameters was missing

Returns cxrError_Not_Connected

The receiver has not connected yet

void cxrTraceEvent(char *name, uint32_t eventId, cxrBool begin)

Add an event to the trace timeline.

This function allows for the insertion of custom events into the trace timeline of CloudXR. Note that to log an event this function must be called twice.

Example of logging an event: void foo() { cxrTraceEvent(“ButtonPressed”, 0x23, cxrTrue); … do some work here … cxrTraceEvent(“ButtonPressed”, 0x23, cxrFalse); }

Parameters

  • name – [in] name of the event to add to the timeline

  • eventId – [in] a custom Id value that can be used to identify the event in the timeline

  • begin – [in] true indicates the beginning of an event, false indicates the end of the event.

cxrError cxrSendAudio(cxrReceiverHandle receiver, const cxrAudioFrame *audioFrame)

Send client input audio (i.e. microphone) to the server.

Note

To use this API set the sendAudio boolean (see cxrDeviceDesc).

Parameters

  • receiver – [in] cxrReceiverHandle handle from cxrCreateReceiver()

  • audioFrame – [in] A filled-out struct containing the audio data to send (see cxrAudioFrame).

Returns cxrError_Required_Parameter

One of the required parameters was missing

Returns cxrError_Not_Connected

The receiver has not connected yet

Returns cxrError_Data_Too_Large

The audio frame data was larger than the max size of 64kb

cxrError cxrSetAuthorizationHeader(cxrReceiverHandle receiver, const char *header)

Set an authorization header that is validated by the server.

This api allows a client to specify credentials via a header that is exchanged prior to the connection to the server over HTTPS. If used, this API must be called after cxrCreateReceiver and before cxrConnect.

Note

In order to send credentials the client OS must trust the server certificate.

Parameters

  • receiver – [in] cxrReceiverHandle handle from cxrCreateReceiver()

  • header – [in] The value of the authorization header to send the server during cxrConnect().

Returns cxrError_Required_Parameter

One of the required parameters was missing

cxrError cxrSendPose(cxrReceiverHandle receiver, const cxrVRTrackingState *trackingState)

Explicitly push XR tracking state.

Parameters
Returns cxrError_Required_Parameter

One of the required parameters was missing

Returns cxrError_Not_Connected

The receiver has not connected yet

Returns cxrError_Pose_Callback_Provided

The receiver was configured to poll for poses via callback rather than have the client send them

cxrError cxrGetConnectionStats(cxrReceiverHandle receiver, cxrConnectionStats *stats)

Get Connection statistics during the lifetime of the session.

Note

For best results it is reccommended to poll this API no more than once per second.

Parameters
Returns cxrError_Required_Parameter

One of the required parameters was missing

Returns cxrError_Not_Connected

The receiver has not connected yet

API Structures

enum cxrConnectionQuality

Values:

enumerator cxrConnectionQuality_Unstable = 0

Initial value while estimating quality. Thereafter, expect disconnects. Details in cxrConnectionQualityReason.

enumerator cxrConnectionQuality_Bad = 1

Expect very low bitrate and/or high latency. Details in cxrConnectionQualityReason.

enumerator cxrConnectionQuality_Poor = 2

Expect low bitrate and/or high latency. Details in cxrConnectionQualityReason.

enumerator cxrConnectionQuality_Fair = 3

Expect frequent impact on bitrate or latency. Details in cxrConnectionQualityReason.

enumerator cxrConnectionQuality_Good = 4

Expect ocassional impacts on bitrate or latency.

enumerator cxrConnectionQuality_Excellent = 5

Expect infrequent impacts on bitrate or latency.

enum cxrConnectionQualityReason

Values:

enumerator cxrConnectionQualityReason_EstimatingQuality = 0x0

Inital value while estimating quality.

enumerator cxrConnectionQualityReason_LowBandwidth = 0x1

The percentage of unutilized bandwidth is too low to maintain bitrate.

enumerator cxrConnectionQualityReason_HighLatency = 0x2

The round trip time is too high to maintain low latency.

enumerator cxrConnectionQualityReason_HighPacketLoss = 0x4

The packet loss is too high to overcome without re-transmission.

enum cxrTrackedDeviceProperty

Keys for CloudXR specific vr::ETrackedDeviceProperty values.

Values:

enumerator Prop_CloudXRServerState_Int32 = 10100

State of the CXR server driver as cxrServerState.

enumerator Prop_ArLightColor_Vector3

Primary light color sent by client API cxrSendLightProperties.

enumerator Prop_ArLightDirection_Vector3

Primary light direction sent by client API cxrSendLightProperties.

enumerator Prop_ArAmbientLightSh0_Vector3

First spherical harmonic component followed by a total of CXR_MAX_AMBIENT_LIGHT_SH sent by client API cxrSendLightProperties.

enum cxrDeviceActivityLevel

Values:

enumerator cxrDeviceActivityLevel_Unknown = -1

Unknown state.

enumerator cxrDeviceActivityLevel_Idle = 0

No activity for the last 10 seconds.

enumerator cxrDeviceActivityLevel_UserInteraction = 1

Activity is happening now (movement or prox sensor)

enumerator cxrDeviceActivityLevel_UserInteraction_Timeout = 2

No activity for the last 0.5 seconds.

enumerator cxrDeviceActivityLevel_Standby = 3

Idle for at least 5 seconds (configurable in Settings : Power Management)

enum cxrTrackingResult

Values:

enumerator cxrTrackingResult_Uninitialized = 1

Tracking hasn’t been initialized yet.

enumerator cxrTrackingResult_Calibrating_InProgress = 100

Calibration in progress.

enumerator cxrTrackingResult_Calibrating_OutOfRange = 101

Devices are outside of the calibration range.

enumerator cxrTrackingResult_Running_OK = 200

Tracking is running OK.

enumerator cxrTrackingResult_Running_OutOfRange = 201

Devices are out of the tracking range.

enumerator cxrTrackingResult_Fallback_RotationOnly = 300

Tracking is limited to rotation-only, usually IMU-based, as a fallback.

enum cxrUniverseOrigin

Values:

enumerator cxrUniverseOrigin_Seated

Indicates the user is in a seated position.

enumerator cxrUniverseOrigin_Standing

Indicates the user is in a standing position.

enum cxrHmdTrackingFlags

Values:

enumerator cxrHmdTrackingFlags_HasProjection = 0x00000001

Tracking state has updated projection parameters.

enumerator cxrHmdTrackingFlags_HasIPD = 0x00000002

Tracking state has updated IPD.

enumerator cxrHmdTrackingFlags_HasRefresh = 0x00000004

Tracking state has updated refresh rate.

enumerator cxrHmdTrackingFlags_HasPoseID = 0x00000008

Tracking state contains a 64-bit pose ID.

enumerator cxrHmdTrackingFlags_HasScaling = 0x00000010

Tracking state contains a float scale.

enum cxrInputValueType

Values:

enumerator cxrInputValueType_blob = 1

Freeform input data, cannot exceed CXR_MAX_BLOB_BYTES_PER_INPUT in size.

enumerator cxrInputValueType_boolean = 2

Boolean input data.

enumerator cxrInputValueType_int32 = 3

Int32 input data.

enumerator cxrInputValueType_float32 = 4

Float32 input data.

enum cxrFoveationMode

Values:

enumerator cxrFoveation_PiecewiseQuadratic = 0
enum cxrClientSurfaceFormat

Values:

enumerator cxrClientSurfaceFormat_RGB

32bpp packed R8B8G8X8, sRGB colorspace, alpha channel is not transmitted

enumerator cxrClientSurfaceFormat_RGBA

32bpp packed R8B8G8A8, sRGB colorspace

enumerator cxrClientSurfaceFormat_NV12

12bpp packed YUV 4:2:0 planar with interleaved chroma, sRGB colorspace

enum cxrGraphicsContextType

Values:

enumerator cxrGraphicsContext_D3D11
enumerator cxrGraphicsContext_D3D12
enumerator cxrGraphicsContext_GL
enumerator cxrGraphicsContext_GLES
enumerator cxrGraphicsContext_Vulkan
enumerator cxrGraphicsContext_Metal
enumerator cxrGraphicsContext_Cuda
enumerator cxrGraphicsContext_NvMedia
enumerator cxr_GraphicsContext_Max = ~0
enum cxrDebugFlags

These bitfield masks denote various debugging flags handled by either the Receiver or by the Server. Note that many of these may have some significant performance impact either constantly or at some interval.

Values:

enumerator cxrDebugFlags_LogVerbose = 0x00000001

record very verbose output

enumerator cxrDebugFlags_LogQuiet = 0x00000002

do not open normal log files at all

enumerator cxrDebugFlags_LogPrivacyDisabled = 0x00000010

disable any privacy filtering in output

enumerator cxrDebugFlags_TraceLocalEvents = 0x00000100

record trace of local event timing

enumerator cxrDebugFlags_TraceStreamEvents = 0x00000200

record trace of c/s stream event timing

enumerator cxrDebugFlags_TraceQosStats = 0x00000400

record QoS statistics

enumerator cxrDebugFlags_DumpImages = 0x00001000

Dump images from stream.

enumerator cxrDebugFlags_DumpAudio = 0x00002000

Dump audio from stream.

enumerator cxrDebugFlags_EmbedServerInfo = 0x00004000

server embeds stats into frame buffers for streaming

enumerator cxrDebugFlags_EmbedClientInfo = 0x00008000

client embeds stats into frame buffers or overlays

enumerator cxrDebugFlags_CaptureServerBitstream = 0x00010000

record the server-sent video bitstream

enumerator cxrDebugFlags_CaptureClientBitstream = 0x00020000

record the client-received video bitstream

enumerator cxrDebugFlags_FallbackDecoder = 0x00100000

try to use a fallback decoder for platform

enumerator cxrDebugFlags_EnableAImageReaderDecoder = 0x00400000

[ANDROID only] enable the AImageReader decoder

enumerator cxrDebugFlags_OutputLinearRGBColor = 0x00800000

[ANDROID only] cxrBlitFrame output will be linear instead of sRGB

enumerator cxrDebugFlags_DisableMultistreamQoS = 0x01000000

QoS will be computed on each stream independently.

enum cxrServerState

Values:

enumerator cxrServerState_NotRunning = 0

Initial state, not yet running.

enumerator cxrServerState_Connecting = 1

Server is connecting to client, finished the RTSP handshake.

enumerator cxrServerState_Running = 2

Server is connected and ready to accept video and audio input for streaming

enumerator cxrServerState_HMD_Active = 3

[VR only] HMD is active

enumerator cxrServerState_HMD_Idle = 4

[VR only] HMD is idle

enumerator cxrServerState_Disconnected = 5

Server is disconnected from the client.

enumerator cxrServerState_Error = 6

Server in an error state.

enum cxrClientState

Values:

enumerator cxrClientState_ReadyToConnect = 0

Initial state, no connection attempt pending.

enumerator cxrClientState_ConnectionAttemptInProgress = 1

Attempting to connect to server.

enumerator cxrClientState_ConnectionAttemptFailed = 2

Error occurred during connection attempt.

enumerator cxrClientState_StreamingSessionInProgress = 3

Connected, streaming session in progress.

enumerator cxrClientState_Disconnected = 4

Streaming session has ended.

enumerator cxrClientState_Exiting = 5

Client is shutting down.

enum cxrNetworkInterface

Values:

enumerator cxrNetworkInterface_Unknown = 0

Network interface is unknown.

enumerator cxrNetworkInterface_Ethernet = 1

Network interface is Ethernet.

enumerator cxrNetworkInterface_WiFi5Ghz = 2

Network interface is 5GHz Wi-Fi.

enumerator cxrNetworkInterface_WiFi24Ghz = 3

Network interface is 2.4GHz Wi-Fi.

enumerator cxrNetworkInterface_MobileLTE = 4

Network interface is mobile LTE.

enumerator cxrNetworkInterface_Mobile5G = 5

Network interface is mobile 5G.

enum cxrNetworkTopology

Values:

enumerator cxrNetworkTopology_Unknown = 0

Client Server connection unknown.

enumerator cxrNetworkTopology_LAN = 1

Client and Server are connected over LAN.

enumerator cxrNetworkTopology_WAN = 2

Client and Server are connected over WAN.

enum cxrError

Values:

enumerator cxrError_Success = 0x0

The operation succeeded.

enumerator cxrError_Group_General = 0x100

Errors which can be returned from most API functions

enumerator cxrError_Failed = 0x101

The operation failed.

enumerator cxrError_Timeout = 0x102

The operation timed out.

enumerator cxrError_Not_Connected = 0x103

The client is not connected.

enumerator cxrError_Not_Streaming = 0x104

The client is not streaming.

enumerator cxrError_Not_Implemented = 0x105

The method is not implemened.

enumerator cxrError_Required_Parameter = 0x106

A required parameter was missing.

enumerator cxrError_Module_Load_Failed = 0x107

A library could not be loaded.

enumerator cxrError_Group_Validation = 0x200

Errors validating API input parmeters

enumerator cxrError_Invalid_API_Version = 0x201

The requested API version is invalid.

enumerator cxrError_Invalid_Number_Of_Streams = 0x202

The number of streams is invalid.

enumerator cxrError_Invalid_Device_Descriptor = 0x203

The device descriptor is invalid.

enumerator cxrError_Invalid_Graphics_Context = 0x204

The graphics context is invalid.

enumerator cxrError_Invalid_Video_Format = 0x205

The video format is invalid.

enumerator cxrError_Invalid_Video_Width = 0x206

The video width is invalid.

enumerator cxrError_Invalid_Video_Height = 0x207

The video height is invalid.

enumerator cxrError_Invalid_Video_Fps = 0x208

The video fps is invalid.

enumerator cxrError_Invalid_Video_Max_Bitrate = 0x209

The video max bitrate is invalid.

enumerator cxrError_Invalid_Frame_Mask = 0x20A

The frame mask is invalid.

enumerator cxrError_Group_Server = 0x400

Errors originating from the server

enumerator cxrError_Server_Setup_Failed = 0x401

The setup of the server failed.

enumerator cxrError_Group_Server_Connection = 0x410

Errors connecting caused by issues on the server

enumerator cxrError_Server_Version_Old = 0x411

The server version is too old for the client.

enumerator cxrError_Server_Untrusted_Certificate = 0x412

The server SSL certificate was not trusted by the client.

enumerator cxrError_Server_Initiated_Disconnect = 0x413

The server initiated a disconnect.

enumerator cxrError_Server_Handshake_Failed = 0x414

The handshake with the server failed.

enumerator cxrError_Server_Stream_Configuration_Failed = 0x415

The server could not be setup to offer the requested streams.

enumerator cxrError_Server_Firewall_Configuration_Failed = 0x416

The server firewall could not be traveresed by hole-punching.

enumerator cxrError_Group_Server_Feature_Disabled = 0x420

Errors due to features being disabled by the server

enumerator cxrError_Server_Feature_Disabled_HEVC = 0x421

The server disabled the feature HEVC.

enumerator cxrError_Server_Feature_Disabled_VVSync = 0x422

The server disabled the feature VVSync.

enumerator cxrError_Server_Feature_Disabled_Pose_Prediction = 0x423

The server disabled the feature Pose Prediction.

enumerator cxrError_Server_Feature_Disabled_Send_Audio = 0x424

The server disabled the feature Send Audio.

enumerator cxrError_Server_Feature_Disabled_Receive_Audio = 0x425

The server disabled the feature Receive Audio.

enumerator cxrError_Group_Client = 0x800

Errors originating from the client

enumerator cxrError_Client_Setup_Failed = 0x801

The client setup failed.

enumerator cxrError_Group_Client_Connection = 0x810

Errors connecting caused by issues on the client

enumerator cxrError_Client_Version_Old = 0x811

The client version is too old for the server.

enumerator cxrError_Client_Unauthorized = 0x812

The client was not authorized.

enumerator cxrError_Group_Frame = 0x1000

Errors in the frame lifecycle

enumerator cxrError_Frame_Not_Released = 0x1001

The frame was not released.

enumerator cxrError_Frame_Not_Latched = 0x1002

The frame was not latched.

enumerator cxrError_Frame_Not_Ready = 0x1003

The frame was not ready.

enumerator cxrError_Group_Decoder = 0x2000

Errors in the decoder lifecycle

enumerator cxrError_Decoder_Setup_Failed = 0x2001

The decoder setup failed.

enumerator cxrError_Decoder_No_Texture = 0x2002

The decoder could not acquire a texture.

enumerator cxrError_Decoder_Frame_Not_Ready = 0x2003

The decoder was not ready with a frame.

enumerator cxrError_Group_Input = 0x4000

Errors in sending pose or input to the server

enumerator cxrError_Pose_Callback_Provided = 0x4001

The pose could not be sent because a callback was provided.

enumerator cxrError_Name_Too_Long = 0x4002

Name string exceeds max size.

enumerator cxrError_Too_Many_Inputs = 0x4003

Too many inputs provided for single controller.

enumerator cxrError_Controller_Id_In_Use = 0x4004

Controller ID already in use.

enumerator cxrError_Role_Too_Long = 0x4005

Controller role too long.

enumerator cxrError_Group_Data = 0x8000

Errors in sending other forms of data to the server

enumerator cxrError_Data_Too_Large = 0x8001

The data was too large to send.

enum cxrLogLevel

This enum tries to define logging verbosity in a way similar to various platforms, initially Android, but also influenced by iOS and others.

Values:

enumerator cxrLL_Verbose = 0
enumerator cxrLL_Debug = 1
enumerator cxrLL_Info = 2
enumerator cxrLL_Warning = 3
enumerator cxrLL_Error = 4
enumerator cxrLL_Critical = 5
enumerator cxrLL_Silence = 6

The Silence level means no logging at all, no log file created, we print one line just to note this mode.

enum cxrMessageCategory

Values:

enumerator cxrMC_Correctness = 0
enumerator cxrMC_Performance = 1
typedef struct cxrReceiver *cxrReceiverHandle

CloudXR SDK client interface.

The Receiver is the primary interface handle for a client of the CloudXR SDK to connect to a server, describe client device properties, send position and input updates, stream live video/audio, and acquire video frames to render out locally.

typedef uint8_t cxrBool
typedef void *cxrControllerHandle
typedef void *cxrTextureHandle
typedef void (*cxrMessageCallbackFunc)(void *context, cxrLogLevel level, cxrMessageCategory category, void *extra, const char *tag, const char *const messageText)

This is the definition for a generic message/logging callback function, and is declared here to be identically structured for both client and server implementations.

const uint32_t cxrFrameMask_Mono = 0x01

Maps to index/stream 0 for mono mode.

const uint32_t cxrFrameMask_Left = 0x01

Maps to index/stream 0 for stereo mode.

const uint32_t cxrFrameMask_Right = 0x02

Maps to index/stream 1 for stereo mode.

const uint32_t cxrFrameMask_Mono_With_Alpha = 0x01 | 0x02

Maps to index/stream 0 for mono mode with alpha.

const uint32_t cxrFrameMask_Left_With_Alpha = 0x01 | 0x02

Maps to index/stream 0 for stereo mode with alpha.

const uint32_t cxrFrameMask_Right_With_Alpha = 0x04 | 0x08

Maps to index/stream 1 for stereo mode with alpha.

const uint32_t cxrFrameMask_All = 0xFFFFFFFF

Requests the system map to all available frames.

const unsigned int CXR_MAX_AMBIENT_LIGHT_SH = 9

Maximum number of spherical harmonic components for ambient lighting information.

const int CXR_NUM_CONTROLLERS = 2

The number of controllers tracking states that can be sent with the hmd tracking state.

const uint32_t CXR_AUDIO_CHANNEL_COUNT = 2

Audio is currently always stereo.

const uint32_t CXR_AUDIO_SAMPLE_SIZE = sizeof(int16_t)

Audio is currently singed 16-bit samples (little-endian)

const uint32_t CXR_AUDIO_SAMPLING_RATE = 48000

Audio is currently always 48khz.

const uint32_t CXR_AUDIO_FRAME_LENGTH_MS = 5

Sent audio has a 5 ms default frame length. Received audio has 5 or 10 ms frame length, depending on the configuration.

const uint32_t CXR_AUDIO_BYTES_PER_MS = CXR_AUDIO_CHANNEL_COUNT * CXR_AUDIO_SAMPLE_SIZE * CXR_AUDIO_SAMPLING_RATE / 1000

Total bytes of audio per ms.

const char *const cxrUserDataFileName = "CloudXR User Data"
const char *const cxrUserDataMutexName = "CloudXR User Data Mutex"
const uint32_t cxrUserDataMaxSize = 8 * 1024
const char *cxrErrorString(cxrError E)
static inline char cxrLLToChar(cxrLogLevel ll)

This helper converts a cxrLogLevel enum value into a single ASCII char to easily display the priority of a given message.

cxrFalse 0
cxrTrue 1
CXR_MAX_NUM_VIDEO_STREAMS 4

Max number of video streams supported.

CXR_MAX_VIDEO_STREAM_WIDTH 4096
CXR_MAX_VIDEO_STREAM_HEIGHT 4096
CXR_MAX_VIDEO_STREAM_FPS 144.0f
CXR_DEFAULT_VIDEO_STREAM_FPS 90.0f
CXR_MAX_VIDEO_STREAM_BITRATE 100000
CXR_DEFAULT_VIDEO_STREAM_BITRATE 50000
CXR_NUM_VIDEO_STREAMS_XR 2

In Receiver_XR mode number of streams is always 2.

CXR_MAX_PATH 4096

Max length for path inputs.

CXR_MAX_CONTROLLER_ROLE 63

Max length for controller role.

CXR_MAX_CONTROLLER_NAME 127

Max length for controller name.

CXR_MAX_CONTROLLER_INPUT_COUNT 128

Max number of controller input paths.

CXR_MAX_INPUT_PATH_LENGTH 127

Max length of controller input path.

CXR_MAX_BLOB_BYTES_PER_INPUT 1024

Max number of bytes that can be sent as blob input.

CLOUDXR_LOG_MAX_DEFAULT -1

Constant for logging systems to use their default settings for maximum size, age, and similar properties.

MAX_LOG_LINE_LEN 4096

The maximum size for a given log message C-string.

MAX_TAG_LEN 256

The maximum size for a given log tag C-string.

struct cxrClientCallbacks
#include <CloudXRClient.h>

Callback functions for the Receiver to interact with client code.

These functions are used to either request something from the client or to provide data or notifications to the client.

Public Members

void (*GetTrackingState)(void *context, cxrVRTrackingState *trackingState)

Return HMD and controller states and poses. Called at a fixed frequency by the Receiver, as specified by cxrDeviceDesc::posePollFreq. This callback must be implemented for the server to render changing position and orientation of client device and controller(s).

void (*TriggerHaptic)(void *context, const cxrHapticFeedback *haptic)

Notify the client it should trigger controller vibration.

cxrBool (*RenderAudio)(void *context, const cxrAudioFrame *audioFrame)

Notify the client to play out the passed audio buffer.

Returns

cxrTrue if data consumed/played, cxrFalse if cannot play or otherwise discarded

void (*ReceiveUserData)(void *context, const void *data, uint32_t size)

Reserved for Future Use

void (*UpdateClientState)(void *context, cxrClientState state, cxrError error)

Notify the client of underlying state changes and why the changed occurred, for both abnormal and expected cases. This callback must be implemented if you call cxrConnect() with the async flag set in cxrConnectionDesc in order to be notified of connection state changes in the background thread.

cxrMessageCallbackFunc LogMessage

Notify the client of message strings to be logged, via whatever log facility the client has implemented. The typedef for LogMessage is in CloudXRCommon.h: void (func)(void context, cxrLogLevel level, cxrMessageCategory category, void* extra, const char* tag, const char* const messageText);

void *clientContext

This is an optional app-specific pointer that is sent back to client in all the callback functions as the first parameter (void* context). It would typically be a pointer to a core object/struct necessary to access/implement functionality inside of the callback.

struct cxrReceiverDesc
#include <CloudXRClient.h>

This contains details of the client device and overall session characteristics. It is passed to cxrCreateReceiver for construction of the cxrReceiverHandle instance and connection to server for streaming.

Public Members

uint32_t requestedVersion

Must set to CLOUDXR_VERSION_DWORD.

cxrDeviceDesc deviceDesc

Describes the client device hardware.

cxrClientCallbacks clientCallbacks

Pointers to client callback functions (NULL for unsupported callbacks)

const cxrGraphicsContext *shareContext

The API-specific client graphics context to be shared with CloudXR.

Note

On certain graphics APIs, such as DirectX, the share context may be NULL. However, the share context cannot be NULL on OpenGL ES on Android devices.

uint32_t debugFlags

Features to aid debugging (see cxrDebugFlags)

int32_t logMaxSizeKB

Maximum size for the client log in KB. -1 for default, 0 to disable size limit.

int32_t logMaxAgeDays

Delete log and diagnostic files older than this. -1 for default, 0 to disable.

char appOutputPath[CXR_MAX_PATH]

The client subdirectory where the library can output logs, captures, traces, etc.

struct cxrConnectionStats
#include <CloudXRClient.h>

This structure is passed into cxrGetConnectionStats. There are three categories of statistics:

The frame stats relate to the timing of video frames as they move from server to client and are prefixed with frame.

The network stats relate to the performance of the connection between server and client and are prefixed with bandwidth, roundTrip, jitter and totalPackets.

The quality stats relate to the overall health of the connection and are prefixed with quality.

Public Members

float framesPerSecond

The number of frames per second.

float frameDeliveryTimeMs

The average time a frame takes to reach the client (for XR streaming this includes pose latency)

float frameQueueTimeMs

The average time a frame spends queued on the client.

float frameLatchTimeMs

The time the client application spent waiting for a frame to be latched.

uint32_t bandwidthAvailableKbps

The estimated available bandwidth from server to client.

uint32_t bandwidthUtilizationKbps

The average video streaming rate from server to client.

uint32_t bandwidthUtilizationPercent

The estimated bandwidth utilization percent from server to client.

uint32_t roundTripDelayMs

The estimated network round trip delay between server and client in milliseconds.

uint32_t jitterUs

The estimated jitter from server to client in microseconds.

uint32_t totalPacketsReceived

The cumulative number of video packets received on the client.

uint32_t totalPacketsLost

The cumulative number of video packets lost in transit from server to client.

uint32_t totalPacketsDropped

The cumulative number of video packets dropped without getting displayed on the client.

cxrConnectionQuality quality

The quality of the connection as one of the five values in cxrConnectionQuality.

Note

If the quality is cxrConnectionQuality_Fair the qualityReasons flags will be populated.

uint32_t qualityReasons

The reasons for connection quality as one or more cxrConnectionQualityReason flags.

struct cxrConnectionDesc

Public Members

cxrBool async

Calls to cxrConnect will spawn a background thread to attempt the connection and return control. to calling thread immediately without blocking until a connection is established or fails.

cxrBool useL4S

On networks where available, enable L4S congestion handling.

cxrNetworkInterface clientNetwork

Network adapter type used by the client to connect to the server.

cxrNetworkTopology topology

Topology of the connecton between Client and Server.

struct cxrMatrix34
#include <CloudXRCommon.h>

CloudXR uses a right-handed system, where: +y is up +x is to the right -z is forward Distance units are in meters.

Public Members

float m[3][4]

Matrix components.

struct cxrVector2

Public Members

float v[2]

x, y values of the vector.

struct cxrVector3

Public Members

float v[3]

x, y, z values of the vector.

struct cxrQuaternion

Public Members

float w

w value.

float x

x value.

float y

y value.

float z

z value.

struct cxrLightProperties

Public Members

cxrVector3 primaryLightColor

Primary light color.

cxrVector3 primaryLightDirection

Primary light direction.

cxrVector3 ambientLightSh[CXR_MAX_AMBIENT_LIGHT_SH]

Ambient light spherical harmonic components upto CXR_MAX_AMBIENT_LIGHT_SH.

struct cxrChaperone

Public Members

cxrUniverseOrigin universe

Whether the user is seated or standing.

cxrMatrix34 origin

The origin of the chaperone boundary.

cxrVector2 playArea

The extents of the chaperone boundary.

struct cxrTrackedDevicePose

Public Members

cxrVector3 position

Device 3D position.

cxrQuaternion rotation

Normalized quaternion representing the orientation.

cxrVector3 velocity

Device velocity in tracker space in m/s.

cxrVector3 angularVelocity

Device angular velocity in radians/s.

cxrVector3 acceleration

Device acceleration in tracker space in m/s^2.

cxrVector3 angularAcceleration

Device angular acceleration in radians/s^2.

cxrTrackingResult trackingResult

Device’s tracking result.

cxrBool poseIsValid

Whether this pose is valid or not.

cxrBool deviceIsConnected

Whether the device is physically connected or not.

struct cxrHmdTrackingState

Public Members

uint64_t clientTimeNS

Client-side time when this event happened. The client side clock must be monotonically increasing, but no epoch constraints exist.

uint64_t flags

A bitwise-mask of values from cxrHmdTrackingFlags

float proj[2][4]

If cxrHmdTrackingFlags_HasProjection is set, allows you to update the projection parameters specified in the device description at a per-pose granularity.

float ipd

If cxrHmdTrackingFlags_HasIPD is set, allows you to update the headset interpupilary distance parameters specified in the device description at a per-pose granularity.

float displayRefresh

If cxrHmdTrackingFlags_HasRefresh is set, overrides the headset display target refresh/fps specified in the device description (up to per-pose granularity).

uint64_t poseID

If cxrHmdTrackingFlags_HasPoseID is set, this contains a user-specified 64-bit value associated with this pose. The same value will be delivered to the client in the poseID of the cxrFramesLatched structure associated with this pose.

float scaling

If cxrHmdTrackingFlags_HasScaling is set, allows you to specify an all-axis scale, defaults to 1.0.

cxrTrackedDevicePose pose

The headset’s 3D pose.

cxrDeviceActivityLevel activityLevel

The headset’s activity level.

struct cxrBlob

Public Members

uint64_t size

Blob data size in bytes which cannot exceed CXR_MAX_BLOB_BYTES_PER_INPUT.

void *ptr

Blob data contents which are copied when cxrFireControllerEvents is called.

struct cxrControllerInputValue

Public Members

cxrInputValueType valueType

Value type for this input event based on which one of the below properties should be set.

cxrBlob vBlob

Blob data for this input event.

cxrBool vBool

Boolean data for this input event.

uint32_t vI32

Int32 data for this input event.

float vF32

Float32 data for this input event.

struct cxrControllerEvent

Public Members

uint64_t clientTimeNS

Time this event happened - no epoch is specified, but durations must reflect wall time in nanoseconds.

uint16_t clientInputIndex

Index of this input event in the corresponding controller description provided to cxrAddController().

cxrControllerInputValue inputValue

Value of the input event.

struct cxrControllerDesc

Public Members

uint64_t id

A unique ID for this controller. This ID should persist through a disconnect/reconnect. This is currently expected for general hand controllers to be 0 for left hand and 1 for right hand.

const char *role

A unique role for the controller. This is currently used to indicate handedness, via the URIs “cxr://input/hand/left” and “cxr://input/hand/right”.

const char *controllerName

An identifying product name for the controller. This is used to determine the server rendering of the controller, and may be used for other features.

uint32_t inputCount

The number of inputs paths supported by this controller.

const char **inputPaths

An array of input paths with inputCount elements.

const cxrInputValueType *inputValueTypes

An array of input types with inputCount elements corresponding to the inputPaths.

struct cxrControllerTrackingState

Public Members

uint64_t clientTimeNS

Time this event happened - no epoch is specified, but durations must reflect wall time in nanoseconds.

cxrTrackedDevicePose pose

The controller’s 3D pose.

struct cxrVRTrackingState
#include <CloudXRCommon.h>

This is used to send hmd tracking state and may also be used for left/right controller poses. Any additional controllers/poses may be sent using cxrSendControllerPoses

Public Members

cxrHmdTrackingState hmd

Tracking state for the headset.

cxrControllerTrackingState controller[CXR_NUM_CONTROLLERS]

Tracking state for left/right hand controller index 0 is defined as the left hand.

float poseTimeOffset

Offset in seconds from the actual time of the pose that affects the amount of pose extrapolation. Default is 0.

struct cxrClientVideoStreamDesc

Public Members

uint32_t width

Requested render width, typically this is display width.

uint32_t height

Requested render height, typically this is display height.

Note

Width and height should be even values, ideally a multiple of 32.

float fps

Device display refresh rate, client needs server to provide stream at this rate.

uint32_t maxBitrate

Bitrate for all video traffic is capped at this value. A value of 0 means unlimited.

struct cxrDeviceDesc

Public Members

float maxResFactor

Suggested maximum oversampling size on server. Server downsamples larger res to this. Valid range 0.5-2.0, with 1.2 as a common value.

float ipd

Inter-pupillary distance.

float proj[2][4]

Per eye raw projection extents: tangents of the half-angles from the center view axis. 0|1 for left|right eye, then 0-3 for left, right, top, bottom edges.

float predOffset

A time offset in seconds to adjust the server time prediction model to account for device-specific delay rendering stream to screen.

float maxClientQueueSize

Maximum size of the decoded frame FIFO queue on the client(unit is frames)

cxrBool stereoDisplay

This device can display stereo output so the server needs to render in stereo.

cxrBool receiveAudio

This device can receive and play audio FROM the server (e.g. game/app sounds)

cxrBool sendAudio

This device can send captured audio TO the server (e.g. microphone)

cxrBool embedInfoInVideo

Request the server embed debug information in each frame prior to encoding.

cxrBool disablePosePrediction

Use the raw client-provided pose without any pose extrapolation.

cxrBool angularVelocityInDeviceSpace

True if angular velocity in device space, false if in world space.

cxrBool disableVVSync

Disables VVSync and instead runs server at fixed frame rate based on client fps.

uint32_t foveatedScaleFactor

Request server foveate rendered frames at % of display res. Values [25-100], zero to disable. 50 is reasonable for most uses.

uint32_t posePollFreq

Frequency to poll pose updates. Set 0 for default, or rate per second to invoke polling. Default is 250, maximum is 1000. Recommend at least 2x display refresh.

uint32_t foveationModeCaps

A bitmask of supported foveation modes (1 bit per cxrFoveationMode enum value)

cxrChaperone chaperone

SteamVR-equivalent chaperone or ‘play space’.

struct cxrHapticFeedback
struct cxrGraphicsContext
struct cxrVideoFrame

Public Members

cxrTextureHandle texture

This frame’s texture handle can be cast to a platform texture when needed.

uint32_t width

Width of frame in texels.

uint32_t height

Height of frame in texels.

uint32_t pitch

Width of frame in bytes.

uint32_t widthFinal

Final width of frame after internal transforms, such as de-foveation.

uint32_t heightFinal

Final height of frame after internal transforms, such as de-foveation.

uint32_t streamIdx

Index of video stream this frame comes from.

uint64_t timeStamp

Timestamp when the frame was captured on the server.

struct cxrFramesLatched
#include <CloudXRCommon.h>

This structure is passed into cxrLatchFrame(), and if the latch succeeds it will contain data for the latched frames. The structure’s data must be freed by passing to cxrReleaseFrame() when done using it.

Public Members

uint32_t count

The number of frames latched/acquired.

cxrVideoFrame frames[CXR_MAX_NUM_VIDEO_STREAMS]

The video frame data for each frame latched.

cxrMatrix34 poseMatrix

The device position at that moment in time.

uint64_t poseID

The user-specified 64-bit value that was passed in the cxrHmdTrackingState structure and that is associated with the headset pose used for rendering this frame.

struct cxrAudioFrame