Jetson Linux API Reference
39.2 Release
sipl_coe_unit_sample (SIPL CoE + GMSL Multi-Sensor Camera Sample)

Overview

The sipl_coe_unit_sample application demonstrates the use of the SIPL (Safe Image Processing Library) Camera API with both Camera over Ethernet (CoE) and GMSL camera transports. It supports multi-sensor configurations — all discovered sensors are initialized, streamed, and composited into a single EGL preview window.

The application provides:

  • Unified CoE and GMSL support — Same binary handles both transports via SIPL Query API
  • Multi-sensor compositing — Automatically discovers N sensors and renders them in a grid layout using VIC-based NvBufSurfTransformMultiInputBufCompositeBlend
  • Per-sensor callbacks — Each sensor has its own SetReadyFrameCallback for isolated frame delivery
  • JSON and database configuration — Sensors can be configured through JSON files or the built-in SIPL sensor database (UDDF drivers)
  • Zero-copy NvBufSurface data access — Direct access to hardware buffer surfaces
  • Real-time rendering — Optional EGL-based composite preview over HDMI
  • Frame capture — Save sensor 0 frames to a raw YUV output file
  • Robust error handling — Signal handling (SIGINT/SIGTERM), timeout, and cleanup

Supported camera configurations:

  • CoE: VB1940 Camera over Ethernet (single or dual sensor via JSON)
  • GMSL: AR0234-based stereo modules (e.g., NOVA0 Hawk with 2 sensors), SmartLead IMX728/IMX623
  • Multi-resolution and frame rate support (auto-detected from sensor config)

Important Notes:

  • For CoE: MAC addresses and IP addresses in JSON files must match your camera hardware setup
  • For GMSL: the -m (link-mask) option is required to select active deserializer links
  • Ensure network connectivity to CoE camera before running
  • Default JSON files (vb1940_*.json) are provided in /var/nvidia/nvcam/settings/sipl/
  • NITO calibration files are loaded from /var/nvidia/nvcam/settings/sipl/ (or NITO_PATH env)

For comprehensive SIPL framework documentation, architecture details, UDDF driver development, and JSON configuration guides, see the Camera Development section of the Jetson Linux Developer Guide.

Building and Running

Prerequisites

  • You have followed steps 1-3 in Building and Running.
  • If you are building from your host Linux PC (x86), you have followed step 4 in Building and Running.
  • For CoE: a compatible camera (e.g., VB1940) with network connectivity established.
  • For GMSL: a compatible camera module (e.g., Hawk AR0234 stereo) connected via deserializer.

To build

  • Enter: 
     $ cd /usr/src/jetson_multimedia_api/samples/unittest_samples/sipl_coe_unit_sample
 $ make

To run

  • Enter: 
    $ ./sipl_coe_unit_sample [OPTIONS]

The captured preview stream displays over HDMI when rendering is enabled. For multi-sensor configurations, all sensors are composited into a single grid-layout window.

Examples

CoE (single sensor): 

   $ ./sipl_coe_unit_sample -c VB1940 -json /var/nvidia/nvcam/settings/sipl/vb1940_1.json

CoE (dual sensor with file output): 

   $ ./sipl_coe_unit_sample -c VB1940 -json /var/nvidia/nvcam/settings/sipl/vb1940_0.json -o capture.yuv

GMSL (Hawk stereo — 2 sensors): 

   $ ./sipl_coe_unit_sample -c NOVA0 -m 0001

GMSL with timeout and headless mode: 

   $ ./sipl_coe_unit_sample -c NOVA0 -m 0001 -t 5 --disable-rendering

GMSL with JSON override: 

   $ ./sipl_coe_unit_sample -c NOVA0 -json /path/to/nova0.json -m 0001

Command Line Options

Platform config (required): 

   -c <name>           Config name for GetSensorSystemConfig (e.g., 'NOVA0', 'VB1940')

Config source (at least one required): 

   -json <path>        JSON config file (CoE or GMSL). Optional if database has the config.

GMSL options: 

   -m <mask>           Link enable mask in hex (e.g., '0001'). Required for GMSL.
                       Space-separated for multiple transports (e.g., '0001 0001').

General options: 

   -o <filename>       Output file for captured frames from sensor 0 (optional)
   -t, --timeout <s>   Capture timeout in seconds (optional)
   --disable-rendering Disable EGL rendering (optional)
   --window-x <x>      Window X position (optional)
   --window-y <y>      Window Y position (optional)
   --window-width <w>  Window width (optional)
   --window-height <h> Window height (optional)
   -h, --help          Display help message

To view supported options

   $ ./sipl_coe_unit_sample -h

Flow

The following steps show the flow through this sample:

  1. Parse command-line arguments for config name, JSON path, link mask, and options.
  2. Create NvSiplCameraInterface and configure via SetSiplCameraConfig().
  3. Sensor discovery: SetupCameraSystemConfig() calls ParseDatabase() → ParseJsonFile() → GetSensorSystemConfig() → ApplyMask() (GMSL only).
  4. Initialize all sensors: Initialize() calls SetPlatformCfg() → SetPipelineCfg() per sensor → Init().
  5. Per-sensor setup loop:
    • SetupICPBuffers(i, 4) — ICP buffer allocation
    • SetupISPBuffers(i, 4, true) — ISP buffer allocation with Block Linear output
    • RegisterAutoControlPlugin(i) — NITO-based AE/AWB
    • SetReadyFrameCallback(i, ...) — per-sensor frame callback
    • SetEventCallback(i, ...) — per-sensor event callback
  6. Composite renderer initialization (if rendering enabled):
    • Compute grid layout based on sensor count (1x1, 2x1, 2x2, etc.)
    • Allocate per-sensor staging NvBufSurfaces and a composite output surface
    • Configure NvBufSurfTransformMultiInputBufCompositeBlend parameters
    • Create EGL renderer at composite resolution
  7. StartCapture() — all sensors start streaming together.
  8. Per-sensor frame processing via ProcessReadyFrame() callback:
    • VIC transform: ISP0 output (Block Linear) → per-sensor staging buffer (Pitch Linear NV12)
    • Track per-sensor readiness via bitmask
    • When all sensors have a fresh frame: composite blend → EGL render
    • Optional: save sensor 0 frames to output file via SaveFrameToFile()
  9. Wait for SIGINT/SIGTERM or timeout.
  10. StopCapture() → PrintStatistics() → cleanup all resources.

The application supports preview mode (EGL composite rendering), headless mode (--disable-rendering), and file capture (-o) operations.

Key Structure and Classes

The sample uses the following key classes and structures:

Element Description
NvSiplFrame Structure containing frame data with NvBufSurface, dimensions, and metadata.
NvSiplCameraInterface High-level interface for SIPL camera operations (CoE + GMSL, multi-sensor).
NvSiplCameraConfig Configuration structure with platform name, JSON path, link masks, and pipeline options.
Classes Description
class NvSiplCameraInterface Main camera interface class providing unified CoE/GMSL multi-sensor abstraction.
class NvEglRenderer EGL-based rendering class for composite frame preview.
class NvNvSciBufHelper Helper class for NvSci buffer management and synchronization.

NvSiplCameraInterface key methods used in the sample:

NvSiplCameraInterface Description
SetSiplCameraConfig Set camera configuration before initialization.
SetupCameraSystemConfig Discover sensors via SIPL Query API (ParseDatabase → ParseJsonFile → GetSensorSystemConfig → ApplyMask).
Initialize Initialize all discovered sensors (SetPlatformCfg → SetPipelineCfg → Init).
GetNumSensors Get the number of discovered sensors.
SetupICPBuffers Configure Image Capture Processor buffers for a sensor.
SetupISPBuffers Configure Image Signal Processor buffers for a sensor.
RegisterAutoControlPlugin Register AE/AWB plugin with NITO calibration for a sensor.
SetReadyFrameCallback Register per-sensor callback for processed ISP0 frames.
SetEventCallback Register per-sensor callback for pipeline events.
StartCapture Start capture for all sensors.
StopCapture Stop capture for all sensors.
SaveFrameToFile Save captured frame data to file.
PrintStatistics Display capture performance statistics.
GetSensorSystemConfig Get the discovered sensor system config (for resolution/layout).

Key application functions:

Function Description
ProcessReadyFrame Per-sensor callback: VIC transform → composite blend → EGL render.
OnEventReceived Per-sensor callback for pipeline notifications.
ComputeGridLayout Compute grid columns/rows for multi-sensor composite display.
parse_link_mask Parse hex link-mask string into vector of uint32_t masks.
SignalHandler Signal handler for graceful application termination.
TimeoutHandler Timer-based automatic capture termination.
ShowHelp Display application usage information.

JSON Configuration Elements (CoE):

JSON Section Description
cameraConfigs Array of camera configuration objects (name, type, MAC/IP, HSB ID).
transportSettings Network transport configuration (interface name, IP, VLAN, HSB ID).
sensorInfo Sensor-specific information including resolution and frame rate.
CoECamera Camera over Ethernet settings including MAC/IP addresses.

Frame Processing Flow (multi-sensor):

  1. SIPL ISP0 threads deliver frames per sensor via SetReadyFrameCallback
  2. ProcessReadyFrame performs VIC transform: SIPL buffer → per-sensor staging NvBufSurface
  3. A bitmask tracks which sensors have a fresh frame
  4. When all sensors are ready: NvBufSurfTransformMultiInputBufCompositeBlend composites all staging buffers into a single output surface
  5. The composited surface is rendered via NvEglRenderer::render()
  6. Optional: sensor 0 frames saved to file via SaveFrameToFile()
  7. Buffer lifecycle managed automatically by SIPL framework