SIPL Stereo Pipeline#
Stereo has been demonstrated with the AR0234-based HAWK stereo camera module. In stereo mode, the left sensor is treated as the primary sensor and the right sensor as the secondary sensor. Both sensors are synchronized through external FSYNC; the secondary sensor reuses the primary sensor’s ISP settings and auto-control output to keep the stereo pair operating with common ISP and auto-control behavior(to maximize the parity between the two images for depth estimation). The nvsipl_camera application can be used to run, test, and monitor the stereo pipeline
JSON Configuration for Stereo#
Stereo is configured entirely through the sensor system JSON config. Two sensors sharing the same non-zero sensorGroup value form a stereo pair. deviceIndex determines left (0) versus right (1).
Example: Stereo Configuration (ar0234_hawk.json)#
fsyncMode is set to external for TSC-driven external FSYNC. Both sensors use the same sensorGroup and distinct deviceIndex values.
{
"cameraConfigs": [
{
"name": "AR0234CS_HAWK",
"type": "GMSL",
"description": "AR0234 HAWK GMSL Stereo Camera Module - 1920x1200 30FPS - CSI-A",
"serInfo": {
"name": "MAX9295",
"i2cAddress": "0x40"
},
"linkMode": "LINK_MODE_GMSL2_6GBPS",
"fsyncMode": "external",
"mipiSettings": {
"dphyRate": 747500,
"phyMode": "dphy"
},
"sensorInfo": [
{
"name": "AR0234",
"id": 0,
"description": "OnSemi AR0234 GRBG Sensor for GMSL - HAWK Stereo Left Camera",
"i2cAddress": "0x18",
"numContext": 1,
"isTriggerModeEnabled": true,
"sensorGroup": 1,
"deviceIndex": 0,
"virtualChannels": [
{
"vcIdSrc": 0,
"vcIdDst": 0,
"cfa": "grbg",
"embeddedTopLines": 2,
"embeddedBottomLines": 0,
"inputFormat": "raw10",
"width": 1920,
"height": 1200,
"fps": 30.0,
"isEmbeddedDataTypeEnabled": true
}
]
},
{
"name": "AR0234",
"id": 1,
"description": "OnSemi AR0234 GRBG Sensor for GMSL - HAWK Stereo Right Camera",
"i2cAddress": "0x10",
"numContext": 1,
"isTriggerModeEnabled": true,
"sensorGroup": 1,
"deviceIndex": 1,
"virtualChannels": [
{
"vcIdSrc": 1,
"vcIdDst": 1,
"cfa": "grbg",
"embeddedTopLines": 2,
"embeddedBottomLines": 0,
"inputFormat": "raw10",
"width": 1920,
"height": 1200,
"fps": 30.0,
"isEmbeddedDataTypeEnabled": true
}
]
}
],
"cryptoConfigName": ""
}
]
}
Example: CoE Stereo Configuration (VB1940_Stereo)#
For CoE stereo, stereo is derived from the sensor and transport settings. Do
not use the removed cameraConfigs[].CoECamera.isStereo field.
{
"cameraConfigs": [
{
"name": "VB1940_Left",
"type": "CoE",
"sensorInfo": {
"name": "VB1940",
"id": 0,
"sensorGroup": 1,
"deviceIndex": 0
}
},
{
"name": "VB1940_Right",
"type": "CoE",
"sensorInfo": {
"name": "VB1940",
"id": 1,
"sensorGroup": 1,
"deviceIndex": 1
}
}
],
"transportSettings": [
{
"name": "VB1940_Stereo",
"type": "CoE",
"sync_sensors": true
}
]
}
The CoE path uses HSB VSYNC for synchronized capture. The bridge supports a
dynamic sensor count and passes the VSYNC rate through
CoEBridgeContext::initConfig.frameRate. Module drivers see the derived
stereo state through CoEModuleContext::config.isStereo.
External FSYNC Signal Chain#
Frame synchronization ensures both stereo sensors are within the threshold of 100 μs delta. The external FSYNC signal propagates through the serializer/deserializer chain as follows:
External FSYNC signal chain in the stereo pipeline.#
Software propagation path for fsyncMode:
JSON "fsyncMode": "external" flows through CameraHAL and serializer setup to
MAX9295::SerFinalizeInit, which selects external vs internal FSYNC sequence.#
Stereo Pipeline Lifecycle (Primary and Secondary Sensors)#
When two sensors share the same non-zero sensorGroup in JSON, SIPL treats them as a stereo pair.
The primary sensor (typically deviceIndex 0) owns the group pipeline; the secondary sensor
(typically deviceIndex 1) is attached to that pipeline. Public SIPL calls that target the secondary
sensor index are routed to the same pipeline instance as the primary so that configuration, buffers,
and sync setup stay consistent.
Creation order, queue pre-allocation, and PreInit / Init routing follow the rules documented in
the NVIDIA SIPL Camera Driver Release Documentation. The following table summarizes logical
behavior without naming internal implementation symbols.
API area |
Primary sensor (left) |
Secondary sensor (right) |
|---|---|---|
Pipeline configuration |
Creates the group pipeline and pre-allocates resources needed for both sensors. |
Reuses pre-allocated resources; does not create a second pipeline instance. |
Initialization |
Standard application |
Additional sensor attach and init are handled inside the group path as documented for stereo. |
Registration / images / sync |
Registers buffers and sync objects for the primary path; may take a sensor index where applicable. |
Equivalent calls are satisfied via the group pipeline and sensor index routing. |
Start / Stop |
Starts or stops streaming for the stereo pair. |
No duplicate start/stop on an isolated second pipeline when the pair is unified. |