Unreal Engine Integration Guide#

This guide explains how to integrate CloudXR Runtime and the opaque data extension into a fresh Unreal Engine 5.5 project. It targets C++ engineers who need to build custom Unreal plugin modules that incorporate the CloudXR C API and OpenXR extension. This guide shows you how to author the Unreal-side integration from scratch.

The integration you build will allow you to:

  • Bootstrap the CloudXR runtime service using the C API defined in cxrServiceAPI.h.

  • Expose service configuration and lifecycle management to your Unreal application.

  • Implement NVIDIA’s XR_NV_opaque_data_channel OpenXR extension (defined in XR_NV_opaque_data_channel.h) to carry arbitrary data bidirectionally between the CloudXR client and your Unreal server.

Prerequisites#

  • Windows 10/11 with a Visual Studio 2022 toolchain configured for Unreal C++ builds

  • Unreal Engine 5.5 with the built-in OpenXR plugin enabled

  • CloudXR Runtime

  • The Generic Viewer sample client application (see Samples Overview).

  • An Unreal project created from the Games → Virtual Reality template (or any XR-ready project).

File Layout and Build Steps#

When you download CloudXR Runtime, you receive:

  • An include\ folder containing headers: cxrServiceAPI.h, XR_NV_opaque_data_channel.h, and openxr_extension_helpers.h.

  • Windows binaries: cloudxr.dll, plus dependencies and the CloudXR runtime manifest openxr_cloudxr.json.

To integrate these into your Unreal project:

  1. Create a plugin scaffold. In your Unreal project root, create Plugins\YourCloudXRPlugin\ (name it whatever you like). Use the Unreal Editor’s plugin creation wizard or manually author a .uplugin descriptor that declares two modules: one for runtime management and one for opaque data transport.

  2. Stage the SDK files. Copy the CloudXR include\ folder to Plugins\YourCloudXRPlugin\ThirdParty\CloudXR\include\ and the Windows binaries to Plugins\YourCloudXRPlugin\ThirdParty\CloudXR\lib\windows-x86_64\. Ensure openxr_cloudxr.json is present alongside the binaries.

  3. Create module source folders. For example:

    • Source\CloudXRRuntime\: For managing the CloudXR service lifecycle

    • Source\CloudXROpaqueDataChannel\: For implementing the OpenXR opaque data channel extension

    Add corresponding .Build.cs files that reference the SDK headers and link any required Unreal modules (OpenXRHMD, OpenXRInput, Projects, etc.).

  4. Regenerate project files and compile. Reopen your .uproject file. Unreal prompts you to rebuild the new modules. Once they are built, the plugin binaries appear under Plugins\YourCloudXRPlugin\Binaries\Win64\.

  5. Enable the plugin. In the Unreal Editor, enable your newly created plugin and restart the editor so subsystems and console commands (if any) register correctly.

The rest of this guide describes the C API calls and OpenXR extension functions you must integrate into your custom modules.

CloudXR Runtime Module#

Your runtime module is responsible for:

  • Dynamically loading the CloudXR DLL at runtime

  • Resolving function pointers from cxrServiceAPI.h

  • Creating and managing the nv_cxr_service handle

  • Configuring service properties before starting

  • Starting and stopping the streaming service

  • Initializing Unreal’s OpenXR HMD to use the CloudXR runtime instead of a local VR runtime

Loading the CloudXR Library#

Your module must implement a function that loads the CloudXR library. You can name it anything you want, e.g., LoadCloudXRLibrary. The function must perform these tasks:

  1. Construct the absolute path to your ThirdParty folder, based on the plugin’s install location.

  2. Verify that the folder exists and contains openxr_cloudxr.json.

  3. Set the environment variable XR_RUNTIME_JSON to the absolute path of openxr_cloudxr.json. This tells Unreal’s OpenXR loader to use CloudXR’s runtime.

  4. Call FPlatformProcess::PushDllDirectory with the binaries’ path, then FPlatformProcess::GetDllHandle("cloudxr.dll"), and finally FPlatformProcess::PopDllDirectory.

Example snippet:

FString BinariesPath = FPaths::Combine(
    FPaths::ProjectPluginsDir(),
    TEXT("YourCloudXRPlugin"),
    TEXT("ThirdParty"),
    TEXT("CloudXR"),
    TEXT("lib"),
    TEXT("windows-x86_64")
);
FString RuntimeJsonPath = FPaths::Combine(BinariesPath, TEXT("openxr_cloudxr.json"));
FPlatformMisc::SetEnvironmentVar(TEXT("XR_RUNTIME_JSON"), *RuntimeJsonPath);

FPlatformProcess::PushDllDirectory(*BinariesPath);
void* CloudXRLibraryHandle = FPlatformProcess::GetDllHandle(TEXT("cloudxr.dll"));
FPlatformProcess::PopDllDirectory(*BinariesPath);

If GetDllHandle returns null, the SDK is missing or incompatible. Log an error and make initialization fail.

Resolving SDK Function Pointers#

Once the library is loaded, use FPlatformProcess::GetDllExport to resolve each function declared in cxrServiceAPI.h:

  • nv_cxr_get_library_api_version

  • nv_cxr_get_runtime_version

  • nv_cxr_service_create

  • nv_cxr_service_set_string_property

  • nv_cxr_service_set_boolean_property

  • nv_cxr_service_set_int64_property

  • nv_cxr_service_get_string_property

  • nv_cxr_service_get_boolean_property

  • nv_cxr_service_get_int64_property

  • nv_cxr_service_start

  • nv_cxr_service_stop

  • nv_cxr_service_join

  • nv_cxr_service_destroy

Store these as typed function pointers (using the PFN_* typedefs from cxrServiceAPI.h) in your module class. If any symbol fails to resolve, log the function name and make initialization fail.

Example:

PFN_nv_cxr_service_create pfn_ServiceCreate = (PFN_nv_cxr_service_create)FPlatformProcess::GetDllExport(CloudXRLibraryHandle, TEXT("nv_cxr_service_create"));
if (!pfn_ServiceCreate)
{
    UE_LOG(LogYourPlugin, Error, TEXT("Failed to resolve nv_cxr_service_create"));
    return false;
}

Creating the Service#

After you load the library and resolve functions, call nv_cxr_service_create to obtain an nv_cxr_service* handle:

struct nv_cxr_service* ServiceHandle = nullptr;
nv_cxr_result_t result = pfn_ServiceCreate(&ServiceHandle);
if (result != NV_CXR_SUCCESS)
{
    UE_LOG(LogYourPlugin, Error, TEXT("nv_cxr_service_create failed with code %d"), result);
    return false;
}

You need this handle for all subsequent CloudXR API calls. Store it in your module class.

Configuring Service Properties#

Before you start the service, configure any runtime properties using:

  • nv_cxr_service_set_string_property(ServiceHandle, property_name, property_name_length, value, value_length)

  • nv_cxr_service_set_boolean_property(ServiceHandle, property_name, property_name_length, value)

  • nv_cxr_service_set_int64_property(ServiceHandle, property_name, property_name_length, value)

All three functions return nv_cxr_result_t, and require external synchronization.

Convert Unreal FString arguments to UTF-8 using FTCHARToUTF8 before passing them to the C API.

Example:

FString PropertyName = TEXT("server-port");
FString PropertyValue = TEXT("7000");

FTCHARToUTF8 PropertyNameUTF8(*PropertyName);
FTCHARToUTF8 PropertyValueUTF8(*PropertyValue);

nv_cxr_result_t result = pfn_ServiceSetStringProperty(
    ServiceHandle,
    PropertyNameUTF8.Get(),
    PropertyNameUTF8.Length(),
    PropertyValueUTF8.Get(),
    PropertyValueUTF8.Length()
);

if (result != NV_CXR_SUCCESS)
{
    UE_LOG(LogYourPlugin, Error, TEXT("Failed to set property '%s'"), *PropertyName);
}

You can only modify properties after nv_cxr_service_create succeeds and before calling nv_cxr_service_start. An attempt to set properties after the service starts returns NV_CXR_SERVICE_ALREADY_STARTED.

Starting the Service#

Call nv_cxr_service_start to launch the CloudXR streaming service:

nv_cxr_result_t result = pfn_ServiceStart(ServiceHandle);
if (result != NV_CXR_SUCCESS)
{
    UE_LOG(LogYourPlugin, Error, TEXT("nv_cxr_service_start failed with code %d"), result);
    return false;
}

This function blocks until the service is ready to accept a client connection and an XR application can create an XrInstance. After the function succeeds, immediately initialize Unreal’s OpenXR HMD (see Initializing Unreal’s OpenXR HMD for CloudXR).

Stopping and Destroying the Service#

To shut down gracefully:

  1. Call nv_cxr_service_stop(ServiceHandle) to signal the service to stop. This disconnects clients and applications.

  2. Call nv_cxr_service_join(ServiceHandle) to block until the service has fully stopped.

  3. Call nv_cxr_service_destroy(ServiceHandle) to release the service object.

  4. Unload the CloudXR library with FPlatformProcess::FreeDllHandle(CloudXRLibraryHandle).

Example:

void ShutdownCloudXR()
{
    if (ServiceHandle)
    {
        pfn_ServiceStop(ServiceHandle);
        pfn_ServiceJoin(ServiceHandle);
        pfn_ServiceDestroy(ServiceHandle);
        ServiceHandle = nullptr;
    }
    if (CloudXRLibraryHandle)
    {
        FPlatformProcess::FreeDllHandle(CloudXRLibraryHandle);
        CloudXRLibraryHandle = nullptr;
    }
}

Critical ordering: Ensure that you shut down CloudXR after you clean up Unreal’s OpenXR modules. In your module’s ShutdownModule, unload OpenXRHMD first, then destroy the CloudXR service.

Initializing Unreal’s OpenXR HMD for CloudXR#

After nv_cxr_service_start succeeds, you must reinitialize Unreal’s OpenXR HMD so it uses the CloudXR runtime (the one pointed to by XR_RUNTIME_JSON). Without this step, Unreal continues rendering to the system’s default OpenXR runtime (e.g., SteamVR, Windows Mixed Reality) and your CloudXR client receives no frames.

Create a function that initializes Unreal’s HMD for CloudXR. You can name this function anything you want.

Call this function immediately after nv_cxr_service_start succeeds, always on the game thread.

The function must perform these tasks:

  1. Verify preconditions:

    • GEngine is valid.

    • GEngine->XRSystem is either null or can be safely replaced.

    • The CloudXR service is running.

  2. Load the OpenXRHMD module and create a new tracking system:

    IOpenXRHMDModule* OpenXRModule = FModuleManager::Get().LoadModulePtr<IOpenXRHMDModule>("OpenXRHMD");
if (!OpenXRModule)
{
    UE_LOG(LogYourPlugin, Error, TEXT("Failed to load OpenXRHMD module"));
    return false;
}

TSharedPtr<IXRTrackingSystem, ESPMode::ThreadSafe> NewXRSystem = OpenXRModule->CreateTrackingSystem();
if (!NewXRSystem.IsValid())
{
    UE_LOG(LogYourPlugin, Error, TEXT("Failed to create OpenXR tracking system"));
    return false;
}

  3. Assign the new XR system and stereo rendering device to the engine:

    GEngine->XRSystem = NewXRSystem;
GEngine->StereoRenderingDevice = GEngine->XRSystem->GetStereoRenderingDevice();
GEngine->StereoRenderingDevice->EnableStereo(true);

  4. Notify the XR system about the active world context:

    for (const FWorldContext& Context : GEngine->GetWorldContexts())
{
    if (Context.WorldType == EWorldType::Game)
    {
        GEngine->XRSystem->OnBeginPlay(const_cast<FWorldContext&>(Context));
        break;
    }
}

  5. Restart the OpenXR input plugin to register motion controllers for the new runtime:

    OpenXRModule->StartupModule();

IOpenXRInputPlugin* OpenXRInputPlugin = FModuleManager::Get().LoadModulePtr<IOpenXRInputPlugin>("OpenXRInput");
if (OpenXRInputPlugin)
{
    // Unregister stale motion controller modular features
    TArray<IMotionController*> MotionControllers = IModularFeatures::Get().GetModularFeatureImplementations<IMotionController>(IMotionController::GetModularFeatureName());
    for (IMotionController* MC : MotionControllers)
    {
        if (MC->GetMotionControllerDeviceTypeName() == FName(TEXT("OpenXR")))
        {
            IModularFeatures::Get().UnregisterModularFeature(IMotionController::GetModularFeatureName(), MC);
            break;
        }
    }
    OpenXRInputPlugin->StartupModule();
}

Why this is critical:

  • CloudXR provides its own OpenXR runtime JSON manifest. Setting XR_RUNTIME_JSON is not enough; you must tell Unreal’s OpenXR plugin to reload and create a new XrInstance against that runtime.

  • This function essentially forces Unreal to “rebind” to CloudXR’s OpenXR implementation instead of the system default.

Complete Lifecycle Summary#

Integrate these steps into a custom UGameInstanceSubsystem or module class:

  1. On startup: Load the CloudXR library, resolve function pointers, and call nv_cxr_service_create.

  2. Configure properties using nv_cxr_service_set_*_property calls.

  3. Start the service with nv_cxr_service_start.

  4. Initialize the OpenXR HMD to make Unreal render through CloudXR.

  5. Use OpenXR APIs normally: Once the CloudXR runtime is active, all standard OpenXR functionality works transparently; hand tracking joints, motion controller poses and button states, head tracking, etc. are automatically streamed from the client. From your application’s perspective, the client device appears as a local XR device.

  6. On shutdown: Stop the service with nv_cxr_service_stop, join with nv_cxr_service_join, destroy with nv_cxr_service_destroy, and unload the library.

All CloudXR API calls require external synchronization. Keep them on the game thread.

CloudXR Opaque Data Channel Module#

This module implements NVIDIA’s XR_NV_opaque_data_channel extension, which provides a bidirectional side channel for custom data exchange between your Unreal server and the CloudXR client. The extension API is defined in XR_NV_opaque_data_channel.h.

Important

Opaque data channels require a valid, active OpenXR session. You must create the channel after the CloudXR service has started and the OpenXR session is established.

Implementing the OpenXR Extension Plugin#

Following the established pattern used by Unreal’s built-in OpenXR extensions (hand tracking, eye tracking, Vive trackers), your module should inherit from IInputDeviceModule and your extension class should implement both IOpenXRExtensionPlugin and IInputDevice.

Most of the IInputDevice interface methods (SendControllerEvents, SetChannelValue, etc.) can be left as empty stubs. You only need to implement Tick() for polling xrReceiveOpaqueDataChannelNV.

Requesting the Extension#

Override GetRequiredExtensions to request the opaque data channel extension:

bool GetRequiredExtensions(TArray<const ANSICHAR*>& OutExtensions) override
{
    OutExtensions.Add(XR_NV_OPAQUE_DATA_CHANNEL_EXTENSION_NAME);
    return true;
}

Resolving Extension Functions#

In PostCreateInstance, use xrGetInstanceProcAddr to resolve the six extension functions defined in XR_NV_opaque_data_channel.h:

void PostCreateInstance(XrInstance InInstance) override
{
    xrGetInstanceProcAddr(InInstance, "xrCreateOpaqueDataChannelNV", (PFN_xrVoidFunction*)&xrCreateOpaqueDataChannelNV);
    xrGetInstanceProcAddr(InInstance, "xrDestroyOpaqueDataChannelNV", (PFN_xrVoidFunction*)&xrDestroyOpaqueDataChannelNV);
    xrGetInstanceProcAddr(InInstance, "xrGetOpaqueDataChannelStateNV", (PFN_xrVoidFunction*)&xrGetOpaqueDataChannelStateNV);
    xrGetInstanceProcAddr(InInstance, "xrSendOpaqueDataChannelNV", (PFN_xrVoidFunction*)&xrSendOpaqueDataChannelNV);
    xrGetInstanceProcAddr(InInstance, "xrReceiveOpaqueDataChannelNV", (PFN_xrVoidFunction*)&xrReceiveOpaqueDataChannelNV);
    xrGetInstanceProcAddr(InInstance, "xrShutdownOpaqueDataChannelNV", (PFN_xrVoidFunction*)&xrShutdownOpaqueDataChannelNV);
}

Store these as member variables:

PFN_xrCreateOpaqueDataChannelNV xrCreateOpaqueDataChannelNV = nullptr;
PFN_xrDestroyOpaqueDataChannelNV xrDestroyOpaqueDataChannelNV = nullptr;
PFN_xrGetOpaqueDataChannelStateNV xrGetOpaqueDataChannelStateNV = nullptr;
PFN_xrSendOpaqueDataChannelNV xrSendOpaqueDataChannelNV = nullptr;
PFN_xrReceiveOpaqueDataChannelNV xrReceiveOpaqueDataChannelNV = nullptr;
PFN_xrShutdownOpaqueDataChannelNV xrShutdownOpaqueDataChannelNV = nullptr;

Accessing the OpenXR Instance and System#

In OnCreateSession, store the XrInstance and XrSystemId for later use when creating the channel:

const void* OnCreateSession(XrInstance InInstance, XrSystemId InSystem, const void* InNext) override
{
    OpenXRInstance = InInstance;
    OpenXRSystemId = InSystem;
    return InNext;
}

Store these as member variables. In OnDestroySession, null them out.

Creating the Opaque Data Channel#

After the OpenXR session is established, create a channel with a unique UUID. You must call this after OnCreateSession has been invoked:

bool CreateOpaqueDataChannel()
{
    if (!xrCreateOpaqueDataChannelNV)
    {
        UE_LOG(LogYourPlugin, Error, TEXT("Extension functions not resolved"));
        return false;
    }

    if (OpaqueDataChannelHandle)
    {
        UE_LOG(LogYourPlugin, Warning, TEXT("Channel already created"));
        return false;
    }

    // Generate or use a predefined UUID (16 bytes)
    // The UUID must match what your client expects
    XrUuidEXT ChannelUUID;
    // Example: hardcoded UUID or generate from a known value
    for (int i = 0; i < 16; i++)
    {
        ChannelUUID.data[i] = (uint8_t)(0x01 + i); // Replace with your UUID
    }

    XrOpaqueDataChannelCreateInfoNV CreateInfo = {};
    CreateInfo.type = XR_TYPE_OPAQUE_DATA_CHANNEL_CREATE_INFO_NV;
    CreateInfo.next = nullptr;
    CreateInfo.systemId = OpenXRSystemId;
    CreateInfo.uuid = ChannelUUID;

    XrResult Result = xrCreateOpaqueDataChannelNV(OpenXRInstance, &CreateInfo, &OpaqueDataChannelHandle);
    if (Result != XR_SUCCESS)
    {
        UE_LOG(LogYourPlugin, Error, TEXT("Failed to create opaque data channel: %d"), Result);
        return false;
    }

    UE_LOG(LogYourPlugin, Log, TEXT("Opaque data channel created"));
    return true;
}

Store XrOpaqueDataChannelNV OpaqueDataChannelHandle as a member variable.

Polling Channel State#

Before sending or receiving data, check that the channel is in the CONNECTED state:

XrOpaqueDataChannelStatusNV GetChannelState()
{
    if (!xrGetOpaqueDataChannelStateNV || !OpaqueDataChannelHandle)
        return XR_OPAQUE_DATA_CHANNEL_STATUS_MAX_ENUM;

    XrOpaqueDataChannelStateNV State = {};
    State.type = XR_TYPE_OPAQUE_DATA_CHANNEL_STATE_NV;
    State.next = nullptr;

    XrResult Result = xrGetOpaqueDataChannelStateNV(OpaqueDataChannelHandle, &State);
    if (Result != XR_SUCCESS)
        return XR_OPAQUE_DATA_CHANNEL_STATUS_MAX_ENUM;

    return State.state;
}

The channel states, defined in XR_NV_opaque_data_channel.h, are:

  • XR_OPAQUE_DATA_CHANNEL_STATUS_CONNECTING_NV – Channel is being established

  • XR_OPAQUE_DATA_CHANNEL_STATUS_CONNECTED_NV – Channel is ready for send/receive

  • XR_OPAQUE_DATA_CHANNEL_STATUS_SHUTTING_NV – Channel is shutting down

  • XR_OPAQUE_DATA_CHANNEL_STATUS_DISCONNECTED_NV – Channel is disconnected

Receiving Opaque Data#

Implement Tick (from IInputDevice) to poll for incoming data every frame. The extension uses the standard two-call idiom:

  1. First call: Query the size of available data.

  2. Second call: Allocate a buffer and read the actual payload.

void Tick(float DeltaTime) override
{
    if (GetChannelState() != XR_OPAQUE_DATA_CHANNEL_STATUS_CONNECTED_NV)
        return;

    // First call: query size
    uint32_t ReceivedDataCount = 0;
    XrResult Result = xrReceiveOpaqueDataChannelNV(OpaqueDataChannelHandle, 0, &ReceivedDataCount, nullptr);
    if (Result != XR_SUCCESS)
        return;

    if (ReceivedDataCount == 0)
        return;

    // Check buffer size (900 bytes typical limit)
    if (ReceivedDataCount > XR_NV_OPAQUE_BUF_SIZE)
    {
        UE_LOG(LogYourPlugin, Warning, TEXT("Received data exceeds buffer size"));
        return;
    }

    // Second call: read data
    OpaqueDataReceiveBuf.SetNumUninitialized(ReceivedDataCount);
    Result = xrReceiveOpaqueDataChannelNV(OpaqueDataChannelHandle, XR_NV_OPAQUE_BUF_SIZE, &ReceivedDataCount, OpaqueDataReceiveBuf.GetData());
    if (Result != XR_SUCCESS)
        return;

    // Notify gameplay code that new data is available
    BroadcastDataReceivedEvent();
}

Define XR_NV_OPAQUE_BUF_SIZE as 900 bytes (typical limit for opaque data channels).

Sending Opaque Data#

Provide helper methods that validate the channel state, check buffer size, and call xrSendOpaqueDataChannelNV. The function signature is:

typedef XrResult (XRAPI_PTR *PFN_xrSendOpaqueDataChannelNV)(XrOpaqueDataChannelNV opaqueDataChannel, uint32_t opaqueDataInputCount, const uint8_t* opaqueDatas);

Example wrapper for sending a string:

bool SendOpaqueDataByString(const FString& Data)
{
    if (GetChannelState() != XR_OPAQUE_DATA_CHANNEL_STATUS_CONNECTED_NV)
    {
        UE_LOG(LogYourPlugin, Warning, TEXT("Channel not connected"));
        return false;
    }

    auto Src = StringCast<UTF8CHAR>(*Data);
    if (Src.Length() >= XR_NV_OPAQUE_BUF_SIZE)
    {
        UE_LOG(LogYourPlugin, Warning, TEXT("Data exceeds buffer size"));
        return false;
    }

    XrResult Result = xrSendOpaqueDataChannelNV(OpaqueDataChannelHandle, Src.Length(), reinterpret_cast<const uint8_t*>(Src.Get()));
    return (Result == XR_SUCCESS);
}

Example wrapper for sending raw bytes:

bool SendOpaqueDataByBytes(const TArray<uint8>& Data)
{
    if (GetChannelState() != XR_OPAQUE_DATA_CHANNEL_STATUS_CONNECTED_NV)
    {
        UE_LOG(LogYourPlugin, Warning, TEXT("Channel not connected"));
        return false;
    }

    if (Data.Num() >= XR_NV_OPAQUE_BUF_SIZE)
    {
        UE_LOG(LogYourPlugin, Warning, TEXT("Data exceeds buffer size"));
        return false;
    }

    XrResult Result = xrSendOpaqueDataChannelNV(OpaqueDataChannelHandle, Data.Num(), Data.GetData());
    return (Result == XR_SUCCESS);
}

Shutting Down and Destroying the Channel#

When you’re finished with the channel, gracefully shut it down and destroy it:

void DestroyOpaqueDataChannel()
{
    if (!OpaqueDataChannelHandle)
        return;

    // First, shutdown the channel
    if (xrShutdownOpaqueDataChannelNV)
    {
        XrResult Result = xrShutdownOpaqueDataChannelNV(OpaqueDataChannelHandle);
        if (Result != XR_SUCCESS && Result != XR_ERROR_CHANNEL_NOT_CONNECTED_NV)
        {
            UE_LOG(LogYourPlugin, Warning, TEXT("Failed to shutdown channel: %d"), Result);
        }
    }

    // Then destroy the handle
    if (xrDestroyOpaqueDataChannelNV)
    {
        XrResult Result = xrDestroyOpaqueDataChannelNV(OpaqueDataChannelHandle);
        if (Result != XR_SUCCESS)
        {
            UE_LOG(LogYourPlugin, Error, TEXT("Failed to destroy channel: %d"), Result);
        }
    }

    OpaqueDataChannelHandle = nullptr;
}

Call this function in your module’s shutdown or when the OpenXR session ends.

On this page