Controller Input and Haptics

CloudXR Framework can forward spatial controller input from visionOS clients to the CloudXR Runtime. Compatible controllers can also receive haptic feedback from the server application.

Controller Setup

Apps that use spatial controller tracking must declare accessory tracking usage in Info.plist. Apps should also declare the GameController profiles they support. Include SpatialGamepad for spatial controllers, and include other profiles only if your app supports them.

<key>NSAccessoryTrackingUsageDescription</key>
<string>This app uses controller tracking during CloudXR streaming.</string>
<key>GCSupportedGameControllers</key>
<array>
    <dict>
        <key>ProfileName</key>
        <string>SpatialGamepad</string>
    </dict>
    <dict>
        <key>ProfileName</key>
        <string>ExtendedGamepad</string>
    </dict>
</array>

For spatial controller tracking, configure controllerTrackingMode and allow RealityKit to receive game controller events:

var config = CloudXRKit.Config()
config.controllerTrackingMode = .spatialController

RealityView { content in
    // Add CloudXR session content.
}
.handlesGameControllerEvents(
    matching: .gamepad,
    withOptions: .receivesEventsInView(false)
)

The runtime presents controller input to the server application through the standard OpenXR action system. For runtime-side controller profiles and binding fallbacks, see Controller and Hand Tracking.

Controller Haptics

CloudXR Framework forwards controller haptic requests from the runtime to compatible physical controllers. On visionOS, controller haptics require visionOS 26.0 or later and a controller that exposes GameController haptics, such as PlayStation VR2 Sense controllers.

Your OpenXR server application can use standard OpenXR haptic actions. Haptic requests are delivered when the connected client and controller support them. If haptics are unavailable, the request is ignored and controller input continues to work.