cuVSLAM on a VB1940 stereo rig
Tracks the pose of a Leopard Eagle VB1940 stereo camera with cuVSLAM. Frames are captured through SIPL in ISP mode, converted to color on the GPU, and paired by projected HSB first-data time before they reach the tracker, which publishes each estimate into a tree of coordinate frames the application owns for the run.
One SIPLCaptureOp is created per camera the description declares, all sharing a single
SIPLCaptureService, and the tracker takes a frame from every one of them. The rig size comes from
the description rather than from configuration: two cameras up to CuvslamOp::kMaxCameras. Each
camera is identified by its position in the rig, and the window shows them in a row of equal
shares.
SIPL supplies projected HSB first-data time in each Image header and scheduling metadata.
OnAligned pairs the existing image ports using that time; each image’s timestamp also reaches
cuVSLAM. This marks data reaching the HSB FPGA and has not been calibrated to exposure start,
end or midpoint. Host PTP must remain running; see the shared
PTP setup.
Running
common/sipl/config/vb1940_stereo.json is the SIPL platform
description shared with the other SIPL applications. Before running the application,
set up the HSB camera for SIPL.
Press Ctrl+C to stop capture and deinitialize the sensors and sensor bridge.
The tracker prints a progress report every report_period_s: frame sets per second, how far apart
the frames within a set were captured, the range of gaps between sets, and where the rig is.
Camera Calibration
cuVSLAM requires a camera calibration file (config/config.yaml) to run.
Generate the calibration file
Whenever you connect a new VB1940 camera, run this command from the Isaac OS checkout to download
its factory RGB calibration from EEPROM into config/config.yaml:
The generated YAML uses the left camera as the rig origin.
File format
The file contains intrinsics and placement for each camera, plus tracker settings. See the example configuration for the expected format with illustrative values.
Stereo hardware synchronization
[!WARNING] Synchronized stereo capture has a known platform startup issue: both streams can stop delivering frames within the first few seconds or mid-run. If that happens restart the application.
The tracker pairs frames on HSB first-data time. Hardware synchronization remains necessary
for stereo tracking. The shared platform description asks for hardware
synchronization: both sensors share sensorGroup, take a deviceIndex apiece, and the transport
is asked for sync_sensors. The sensor bridge reports what it made of that at start-up, and this
is the line to read first:
Were the sensors free-running instead, it would read:
Two free-running 30 Hz sensors drift through every relative phase, so only a varying fraction of frames land inside a narrow window and the rest are dropped before they reach the tracker.
Widening --alignment-window-ms admits more frame sets but does not increase the tracker’s
supported synchronization tolerance. Use a wider window only to diagnose pairing, and validate
exposure synchronization and tracking accuracy before relying on the poses.