SIPL Camera Application Developer Guide#

NVIDIA SIPL Camera Driver Package#

Camera driver source package for Jetson L4T platforms. It contains UDDF device drivers for GMSL and Camera-over-Ethernet (CoE) camera modules, plus SIPL sample applications to capture and display camera frames.

Note

This package is a companion to the NVIDIA SIPL Camera Driver Release Documentation, which covers architecture, APIs, platform configuration, and advanced topics in detail. Refer to it for anything beyond build-and-run basics.

Repository Layout#

Top-level layout of sipl:

sipl/
├── uddf/                   # Device drivers (see uddf/README.md)
│   ├── drivers/            #   Production GMSL component drivers
│   ├── coSerDes/modules/   #   Production CoSerDes camera-module drivers
│   ├── libraries/          #   Shared-library wrappers (libnvuddf_*.so)
│   ├── samples/            #   Reference/sample drivers (GMSL, CoE, Eagle)
│   ├── include/            #   UDDF CDI & DDI public headers
│   └── common/             #   Shared driver utilities
├── samples/                # SIPL sample applications
│   ├── camera/             #   nvsipl_camera — main capture & display app
│   └── query/              #   nvsipl_query — camera config query tool (GMSL + CoE)
├── hsl/                    # Hardware Sequencing Language (compiler & headers)
├── hololink/               # Hololink source
├── include/                # SIPL, NvMedia, and NvSci public headers
├── capture/                # Capture subsystem headers
└── CMakeLists.txt          # Top-level build

Build Prerequisites#

Requirement	Details
CMake	3.15 or later.
Python	3.10 to 3.11 (used by the HSL compiler at build time).
Cross-compiler	`aarch64-linux-gnu-g++` (for Jetson target builds from a host).

Install on Ubuntu/Debian:

sudo apt update
sudo apt-get install -y cmake python3 build-essential
# For cross-compilation to aarch64 (host PC):
sudo apt install -y gcc-aarch64-linux-gnu g++-aarch64-linux-gnu

Building the Package#

mkdir build && cd build
# Cross-compile for Jetson (aarch64)
cmake .. -DCMAKE_TOOLCHAIN_FILE=<path-to-aarch64-toolchain.cmake>
make -j$(nproc)

Note

For native builds on the Jetson device, use cmake .. -DCMAKE_BUILD_TYPE=Release && make -j$(nproc)

CMake Options for Applications#

These options are set in samples/camera/CMakeLists.txt and match the flags used by the internal tmake build.

Option	Default	Description
`ENABLE_CAMERA_HAL`	`ON`	Enable Camera HAL query path (`--enable-camera-hal` flag).
`NV_EMBEDDED_L4T`	`ON`	Enable L4T embedded code paths (CoE support).
`EMU_ON_DESKTOP`	`ON`	Skip `nvpkcs11` header; set `OFF` when cross-compiling with a full sysroot.

Key Build Outputs#

Output	Description
`libnvuddf_max96712_library.so`	MAX96712 deserializer driver
`libnvuddf_max96724_library.so`	MAX96724 deserializer driver
`libnvuddf_ar0234hawkcameramodule_library.so`	AR0234 HAWK camera module driver
`libnvuddf_r0sim623cameramodule_library.so`	IMX623 camera module driver
`libnvuddf_r0sim728cameramodule_library.so`	IMX728 camera module driver
`libnvuddf_max20087_library.so`	MAX20087 power controller driver
`libnvuddf_eagle_library.so`	Eagle AIO (CoE) module driver
`libnvuddf_tegra_deser_power_library.so`	Tegra deserializer power management
`nvsipl_camera`	SIPL camera sample application
`nvsipl_query`	Camera configuration query tool (GMSL + CoE)
`libsample_coe_driver.so`	CoE sample/reference driver

Installing on the Target#

cd build && sudo make install

This installs artifacts to the Jetson target as follows:

Target	Install path	Source
`nvsipl_camera`	`/usr/sbin/`	`samples/camera/`
`nvsipl_query`	`/usr/sbin/`	`samples/query/`
`libnvuddf_*.so`	`/usr/lib/nvsipl_drv/`	`uddf/libraries/`

The SIPL runtime discovers and loads drivers from /usr/lib/nvsipl_drv/ at startup.

Note

Each sample’s CMakeLists.txt defines its own install() rules.

Running Sample Applications#

`nvsipl_camera`#

The main sample application: it initializes the camera pipeline, captures frames, and optionally displays them.

# Basic camera test (AR0234CS_HAWK platform, link 0 only, RAW output, 5 seconds):
sudo nvsipl_camera -c AR0234CS_HAWK -m '0001' --enable-camera-hal -s -Z -012R -r 5

# With verbose logging:
sudo nvsipl_camera -c AR0234CS_HAWK -m '0001' --enable-camera-hal -s -Z -012R -r 5 -v 4

Flag	Purpose
`-c AR0234CS_HAWK`	Platform configuration name
`-m '0001'`	Link enable mask (hex; link 0 on first deserializer)
`--enable-camera-hal`	Use Camera HAL query path (required for newer configs)
`-s`	Show FPS every 2 seconds
`-Z`	Disable sensor authentication
`-012R`	Disable ISP0/1/2; enable RAW output
`-r 5`	Run for 5 seconds, then exit
`--egl-display`	Enable on-screen preview via `NvEglRenderer` with zero-copy `NvSciBuf` to `NvBufSurface` (alternative to the WFD display path). Takes no argument. When enabled, ISP0 output is rendered to a grid window on the connected EGL display.

Refer to the release documentation and samples/camera/ source for the full set of command-line options and supported platform configurations. The sections below document CoE-focused usage patterns in more detail.

For the nvsipl_query sample (INvSIPLCameraQuery and SensorSystemConfig), see SIPL Query Sample.

The nvsipl_camera application demonstrates how to use the NVIDIA Sensor Input Processing Library (SIPL) with Camera Over Ethernet (CoE) cameras. CoE cameras transmit image data over Ethernet networks, providing flexibility in camera placement and simplified wiring compared to traditional camera interfaces.

Note

The older CoE-only sample that came in JetPack 7.1 and before nvsipl_coe_camera is no longer a separate binary. Its merged into nvsipl_camera, the unified SIPL reference application that supports both GMSL and CoE. Use nvsipl_camera with the camera configuration or JSON that matches your camera transport (GMSL or CoE).

Key Features#

CoE Camera Support: Direct integration with CoE cameras via Ethernet.
Multiple Output Formats: Support for ICP (raw) and ISP0/ISP1/ISP2 (processed YUV) outputs.
Dynamic Buffer Management: Automatic buffer size calculation for various sensor configurations.
Multi-threaded Processing: A separate thread for each pipeline stage.
Auto Control Plug-in: Support for NITO-based image processing tuning.

System Components#

+-----------------+    +--------------+    +-----------------+
| CoE Camera      |--->| Ethernet     |--->| Jetson Thor     |
|                 |    | Network      |    | Platform        |
| • VB1940 Sensor |    |              |    |                 |
| • AR0234 Sensor |    |              |    | • SIPL Library  |
| • Image Proc.   |    |              |    | • Application   |
| • Network Stack |    |              |    | • Buffer Mgmt.  |
+-----------------+    +--------------+    +-----------------+

Pipeline Overview#

CoE Camera → Network → SIPL ICP → ISP Pipeline → Application Consumers
                         │
                         ├── RAW Consumer (ICP output)
                         ├── ISP0 Consumer (YUV output)
                         ├── ISP1 Consumer (YUV output)
                         └── ISP2 Consumer (YUV output)

Application Setup and Configuration#

Prerequisites#

Hardware
- NVIDIA Jetson Thor platform with CoE camera support.
- CoE camera (such as VB1940-based).
- Ethernet network connection.
Software
- NVIDIA JetPack 7.2.
- SIPL libraries.
- NvSci libraries.

Required System Libraries

The following libraries must be present on the system for proper operation.

/usr/lib/aarch64-linux-gnu/nvidia/libnvsipl_control.so
/usr/lib/aarch64-linux-gnu/nvidia/libnvsipl_devblk_cdi.so
/usr/lib/aarch64-linux-gnu/nvidia/libnvsipl_devblk_crypto.so
/usr/lib/aarch64-linux-gnu/nvidia/libnvsipl_devblk_ddi.so
/usr/lib/aarch64-linux-gnu/nvidia/libnvsipl_devblk.so
/usr/lib/aarch64-linux-gnu/nvidia/libnvsipl_query.so
/usr/lib/aarch64-linux-gnu/nvidia/libnvsipl.so
/usr/lib/aarch64-linux-gnu/nvidia/libnvsipl_pipeline.so
/usr/lib/aarch64-linux-gnu/nvidia/libnvcamerahal.so
/usr/lib/aarch64-linux-gnu/nvidia/libnvfusacap.so
/usr/lib/nvsipl_drv/libnvuddf_eagle_library.so
/usr/lib/nvsipl_drv/libnvsipl_qry_vb1940.so

These libraries are typically installed with NVIDIA JetPack.

Note

The /usr/lib/nvsipl_drv/ path contains UDDF driver libraries (such as the Eagle CoE driver) and query database libraries.

Installation#

Download the SDK and install JetPack.

If JetPack 7.2 with Jetson Linux 39.2 is installed on NVIDIA Jetson Thor, skip this step.

Download SDK Manager from NVIDIA JetPack SDK Downloads and Notes and follow the provided steps to install JetPack.

SDK Manager copies SIPL package files to /usr/src/jetson_sipl_api/sipl.

Note

Back up existing driver libraries and applications before building and installing the new ones.

Note

In JetPack 7.2, driver build target names changed. Use all_drivers instead of the older samples target. For Eagle-specific builds, use the public CMake targets eagle_driver and eagle_library instead of older eagleAIO references.

On the Jetson device, install the required build tools:

ubuntu@jetson:~$ sudo apt-get update && sudo apt-get install build-essential
ubuntu@jetson:~$ sudo apt-get install cmake

Install build dependencies:

ubuntu@jetson:~$ sudo apt install libfmt-dev
ubuntu@jetson:~$ sudo apt-get install libjsoncpp-dev

Note

The Eagle camera module library depends on hololink source code, which requires fmtlib and CUDA headers.

Install CUDA Headers

The build requires CUDA headers (for example, cuda_runtime.h). Install them using one of the following methods.

Option 1 — Install the CUDA Toolkit via the NVIDIA apt repository (Ubuntu 24.04, sbsa):

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/sbsa/cuda-ubuntu2404.pin
sudo mv cuda-ubuntu2404.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/13.2.1/local_installers/cuda-repo-ubuntu2404-13-2-local_13.2.1-595.58.03-1_arm64.deb
sudo dpkg -i cuda-repo-ubuntu2404-13-2-local_13.2.1-595.58.03-1_arm64.deb
sudo cp /var/cuda-repo-ubuntu2404-13-2-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda-toolkit-13-2

Option 2 — Install the CUDA Toolkit via the NVIDIA apt repository keyring:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/sbsa/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install cuda-toolkit-13-2

Build the application:

ubuntu@jetson:~$ cd /usr/src/jetson_sipl_api/sipl
ubuntu@jetson:/usr/src/jetson_sipl_api/sipl$ sudo mkdir build
ubuntu@jetson:/usr/src/jetson_sipl_api/sipl$ cd build/
ubuntu@jetson:/usr/src/jetson_sipl_api/sipl$ sudo cmake ../
ubuntu@jetson:/usr/src/jetson_sipl_api/sipl$ sudo make
ubuntu@jetson:/usr/src/jetson_sipl_api/sipl$ sudo make install

After successful installation, the application is available at the following location:
```
/usr/sbin/nvsipl_camera
```

Configuration Methods#

The application supports three configuration methods:

Platform configuration name: nvsipl_camera -H -c <platform-config-name> (-H required for CoE; see below).
JSON configuration file: nvsipl_camera -H -t <path-to-config.json> when using a CoE JSON/platform flow that goes through Camera HAL.
CoE override file (optional): nvsipl_camera -H -c <config> --coeConfigOverridePath <override.csv>.

Important

For Camera over Ethernet (CoE), you must pass -H / --enable-camera-hal on the command line whenever you run the camera (any combination of -c, -t, and optional CoE overrides). CoE is brought up through the Camera HAL path, not the legacy DevBlk path that nvsipl_camera uses when -H is omitted. Omitting -H for CoE is unsupported for normal streaming and configuration workflows described in this guide.

Other camera transports (for example GMSL) may use different defaults; confirm with your platform’s SIPL or GMSL guide and ./nvsipl_camera --help when not using CoE.

Usage and Command-Line Options#

Basic Usage Examples#

Examples for various camera modules and configurations:

# Run with default settings (5 seconds, no file output)
sudo nvsipl_camera -c AR0234CS_HAWK -m "1000" -H -012 -R

# Run with default settings (5 seconds, no file output, -K 30 to skip first 30 frames)
sudo nvsipl_camera -c R0SIM623S5RU1197NB3 -m "0010" -H -012 -R -W 1 -K 30 -f test_imx623 --verbosity 4

# Run with default settings (5 seconds, no file output and verbosity 4 for debug output, mask 1 for link 0)
sudo nvsipl_camera -c R0SIM728S3RU2120NB2 -m "1" -H -12 -W 1 -K 30 -f test --verbosity 4

# Run with default settings (5 seconds, no file output)
sudo nvsipl_camera -c VB1940_Camera -H

# Run with custom duration 10 seconds
sudo nvsipl_camera -c VB1940_Camera -r 10 -H

# Enable RAW output capture and write 10 frames to file test_capture
sudo nvsipl_camera -c VB1940_Camera -R -W 10 -f test_capture -H

# Enable multiple outputs with custom NITO path
sudo nvsipl_camera -c VB1940_Camera -R -0 -1 -W 5 -f capture -N /custom/nito/path -H

# Use JSON configuration file
sudo nvsipl_camera -t custom_config.json -0 -W 3 -v 2 -H

# Use CoE override file for network configuration
sudo nvsipl_camera -c VB1940_Camera --coeConfigOverridePath override.csv -R -W 5 -H

Command-Line Options#

Run ./nvsipl_camera --help on the device for the authoritative option list. The following matches the application’s built-in usage text (option names and behavior may vary slightly by release).

Usage:
-h or --help                               :Prints this help
-c or --platform-config 'name'             :Platform configuration. Supported values
-m or --link-enable-masks 'masks'          :Enable masks for links on each deserializer connected to CSI
                                           :masks is a list of masks for each deserializer.
                                           :Eg: '0x0000 0x1101 0x0000 0x0000' disables all but links 0, 2 and 3 on CSI-CD interface
-r or --runfor <seconds>                   :Exit application after n seconds (also disables runtime menu)
-v or --verbosity <level>                  :Set verbosity
                                           :Supported values (default: 1)
                                           : 0 (None)
                                           : 1 (Errors)
                                           : 2 (Warnings and above)
                                           : 3 (Infos and above)
                                           : 4 (Debug and above)
-t or --test-config-file <file>            :Set custom platform config json file
-l or --list-configs                       :List configs from file specified via -t or --test-config-file
-R or --enableRawOutput                    :Enable the Raw output
-0 or --disableISP0Output                  :Disable the ISP0 output
-1 or --disableISP1Output                  :Disable the ISP1 output
-2 or --disableISP2Output                  :Disable the ISP2 output
-s or --showfps                            :Show FPS (frames per second) every 2 seconds
-M or --showmetadata                       :Show Metadata when RAW output is enabled
-p or --plugin <type>                      :Auto Control Plugin. Supported types (default: If NITO available 0 else 1)
                                           :  Nvidia AE/AWB Plugin
                                           :  Custom Plugin
-a or --autorecovery                       :Recover deserializer link failure automatically
-k or --profile <file-prefix>              :Dump profiling timestamps in file-prefix_cam_x_out_y.csv file
-N or --nito <folder>                      :Path to folder containing NITO files
-O or --icrop 'y+h'                        :Specifies the cropping at the input in the format 'y+h'
-e or --showEEPROM                         :Show EEPROM data
-I or --ignoreError                        :Ignore the fatal error
-o or --enableStatsOverrideTest            :Enable ISP statistics settings override
-u or --disableSubframe                    :Disable Subframe feature
-S or --enablePassive                      :Enable passive mode
-f or --filedump-prefix 'str'              :Dump RAW file with filename prefix 'str'  when RAW output is enabled.
-K or --skipFrames <val>                   :Number of frames to skip before writing to file
-W or --writeFrames <val>                  :Number of frames to write to file
-Z or --disableAuth                        :Forcefully disable authentication for all camera sensors.
-E or --setSensorCharMode <expNo>          :Set sensor in characterization mode with exposure number.
-i or --input-raw-files <file1[,file2]...> :Set input RAW files for simulator mode testing.
-F or --fsyncGroupStartTime 'group ticks'  :Group ID and start time in tsc ticks to be programmed.
                                           :All fsync signal generator in group will start at specified start time.
                                           :Eg: 'group_id start_time_in_tsc_ticks'.
-x or --getTimeTSC                         :Get the current TSC ticks value.
-b or --fsyncGroupStartNow group           :Start the fsync group now.
                                           :All fsync signal generator in group will start at current time.
-B or --bufferAlloc 'Buffers'              :Set number of buffers to allocate and register for each channel (default: 6 buffers for capture, 4 for each ISP channel)
                                           :Eg: '40 64 64 64' selects 40 buffers for the capture (VI) channel and 64 for ISP0/1/2
-H or --enable-camera-hal                  :Enable Camera HAL path instead of legacy DevBlk path
--coeConfigOverridePath <file>             :Path to CoE config override CSV file (auto-detected for CoE platforms)
--egl-display                              :Enable EGL display rendering (alternative to WFD). L4T only.

Configuration Sources#

The application accepts configuration from two sources:

Built-in Platform Configurations (-c option):

Use predefined platform configurations from the SIPL database:
```
./nvsipl_camera -c VB1940_Camera -H
```
To print built-in usage and options (no camera session; -H is not required):
```
./nvsipl_camera -h
```
To list configuration entries from a JSON file you pass with -t / --test-config-file:
```
./nvsipl_camera -t /path/to/config.json -l -H
```
JSON Configuration Files (-t option):

Use custom JSON configuration files:
```
./nvsipl_camera -t /path/to/config.json -H
```

Verbosity Levels#

The application supports five verbosity levels:

Level	Description	Output
0	No logging	None.
1	Error only	Error messages (default).
2	Warning	Error and warning messages.
3	Info	Error, warning, and info messages.
4	Debug	All messages including detailed debug info.

Example with debug output:

./nvsipl_camera -c VB1940_Camera -v 4 -H

Normal Streaming with CoE Cameras#

The following steps outline the normal streaming use case:

Call INvSIPLCamera::GetInstance: Use this pointer to call into all the SIPL APIs.
Call SetPlatformCfg: Set valid platform configuration.
For each device block and sensor call SetPipelineCfg: Set valid pipeline configuration and NvSIPLPipelineQueue objects.
Call Init: Initialize the SIPL camera system.
Call SetSensorCharMode (optional, non-safety only): If sensor characterization is required, call this for each sensor that needs characterization. Must be called after Init() and before Start(). See 5. Sensor Characterization Mode (optional, non-safety only) for details.
Image Creation:
1. Create NvSciBufModule using NvSciBufModuleOpen.
2. For each ISP output:
  1. Create NvSciBufAttrList using NvSciBufAttrListCreate.
  2. Set any general attributes required using NvSciBufAttrListSetAttrs. Include attributes to set image formats for ISP.
  3. Call GetImageAttributes to set default attributes or validate the attributes previously set.
  4. Create a Reconciled Attribute List and a Conflict Attribute List.
  5. Create and fill any other attribute list for consumers of the ISP outputs.
  6. Reconcile all lists using NvSciBufAttrListReconcile.
  7. Create as many buffer objects as required to create a buffer pool using NvSciBufObjAlloc.
  8. After the object is allocated, the attribute lists are no longer needed. You can destroy them by calling NvSciBufAttrListFree.
  9. Register the buffer objects with SIPL by calling RegisterImages.
Auto Control: For each sensor on which ISP needs to be enabled:
1. Load the NITO file into system memory.
2. Call RegisterAutoControlPlugin.
Synchronization:
1. Create NvSciSyncModule using NvSciSyncModuleOpen.
2. For each sensor that needs ISP:
  1. Create a NvSciSyncAttrList using NvSciSyncAttrListCreate (producer list).
  2. Call FillNvSciSyncAttrList for this attribute list with valid parameters:
    - Valid pipeline ID.
    - outType: ICP or ISP0. (ISP1 or ISP2 can be used, but only one is required per pipeline.)
    - clientType:
      - SIPL_SIGNALER if it is to be used as an EOFFence.
      - SIPL_WAITER if it is to be used as a PreFence.
  3. Create another NvSciSyncAttrList (consumer list):
    - If the producer list is SIGNALER, the consumer list should be a waiter.
    - If the producer list is WAITER, the consumer list should be a signaler.
  4. Fill the attributes required for the producer list:
    - If the CPU consumer is a waiter, set the following suggested attributes:
      - NvSciSyncAttrKey_NeedCpuAccess: true
      - NvSciSyncAttrKey_RequiredPerm: NvSciSyncAccessPerm_WaitOnly
    - If the consumer is some other engine, call the analogous FillNvSciSyncAttrList API for that engine.
  5. Create a Reconciled Attribute List and a Conflict Attribute List.
  6. Reconcile all the lists using NvSciSyncAttrListReconcile.
  7. Create sync objects using the attribute lists with a call to NvSciSyncObjAlloc.
  8. After the object is allocated, destroy the attribute lists by using NvSciSyncAttrListFree.
  9. Register NvSciSyncObj with SIPL by calling RegisterNvSciSyncObj with valid parameters:
    - Valid pipeline ID.
    - outType: ICP or ISP0. (ISP1 or ISP2 can be used, but only one is required per pipeline.)
    - clientType:
      - NVSIPL_EOFSYNCOBJ if it is to be used as an EOFFence.
      - NVSIPL_PRESYNCOBJ if it is to be used as a PreFence.
Create Threads: Create one thread per sensor for each of the following queues to get the payload when available and perform the relevant actions:
1. Output queues of ICP and ISPs that are enabled:
  1. Action 1: Call GetImageData on the INvSIPLBuffer payload that was obtained.
  2. Action 2: Check frameSeqNumValid and print frameSequenceNumber.
  3. Optional for consumers of SIPL:
    - When a consumer is using the buffer, call AddRef on the buffer from the consumer end.
    - Wait on the EOF fence in the buffer obtained by calling GetEOFNvSciSyncFence using CPU.
    - Ensure the consumer calls Release on the buffer only after it has finished processing.
  4. Action 3: Call Release when done with the buffer to allow SIPL to access it.
    
    Important
    
    Release the buffers in a timely manner to avoid starving SIPL of buffers.
2. Notification queue of each pipeline enabled:
  1. Action 1: Look for NOTIF_INFO* notifications if the required user can keep track; otherwise, they can be ignored.
  2. Action 2: Check for warning notifications for NOTIF_WARN_ICP_FRAME_DROP or NOTIF_WARN_ICP_FRAME_DISCONTINUITY and keep a count.
  3. Action 3: Check for any other notifications and report them to the user.
    
    Important
    
    Be sure to get these notifications because the size of the queue is limited. If the queue is not cleared, SIPL fails to write new content to the queue.
3. Start the threads.
Waiter Context:
- CPU waiter context is helpful when you need to wait on the EOF fences of SIPL.
- You can use NvSciSyncModule to create a context using the NvSciSyncCpuWaitContextAlloc API.
Call Start: Begin streaming.
Queue threads should see incoming frame data.
Sleep for N seconds: N is passed in the command line. During this time, check for any errors in the notification queue and report errors to the user.
Call Stop to stop streaming.
Optional: For each sensor, report the number of frame drops and discontinuities.
Call Deinit: Deinitialize the camera system.
Deinitialize buffer resources:
1. Free the NvSciBufObj objects by calling NvSciBufObjFree.
2. Free NvSciBufModule by calling NvSciBufModuleClose.

Initialization Sequence for CoE-specific Implementation#

The application follows this initialization sequence:

1. SIPL Setup#

// Get SIPL Camera and Query instances
m_upCamera = INvSIPLCamera::GetInstance();
m_upQuery = INvSIPLCameraQuery::GetInstance();

// Parse database and configuration
status = m_upQuery->ParseDatabase();
if (m_cmdline.sTestConfigFile != "") {
    status = m_upQuery->ParseJsonFile(m_cmdline.sTestConfigFile);
} else if (m_cmdline.sConfigName != "") {
    status = m_upQuery->GetSensorSystemConfig(m_cmdline.sConfigName, sensorSystemConfig);
}

2. Platform Configuration#

// Set platform configuration with CoE camera system
status = m_upCamera->SetPlatformCfg(sensorSystemConfig);

// Build module list from sensor system config
m_coeModules.clear();
for (size_t i = 0; i < sensorSystemConfig.modules.size(); i++) {
    const auto& moduleConfig = sensorSystemConfig.modules[i];
    if (!std::holds_alternative<nvsipl::sensorconfig::CoEModule>(
            moduleConfig.moduleType)) {
        continue;
    }

    const auto& coeModule =
        std::get<nvsipl::sensorconfig::CoEModule>(moduleConfig.moduleType);
    for (const auto& sensorConfig : coeModule.sensorConfigs) {
        if (!std::holds_alternative<nvsipl::sensorconfig::CoECameraSensorConfig>(
                sensorConfig)) {
            continue;
        }

        const auto& coeSensor =
            std::get<nvsipl::sensorconfig::CoECameraSensorConfig>(sensorConfig);
        CoeModuleData module;
        module.sensorId = coeSensor.id;
        module.index = static_cast<uint32_t>(i);
        module.name = moduleConfig.name;
        module.platform = moduleConfig.platform;
        module.sensorName = coeSensor.name;
        m_coeModules.push_back(module);
    }
}

3. Pipeline Configuration#

// Global CoE pipeline configuration
NvSIPLPipelineConfiguration g_coePipelineCfg = {
    .captureOutputRequested = true,    // ICP (raw) output
    .isp0OutputRequested = true,       // ISP0 (YUV) output
    .isp1OutputRequested = false,      // ISP1 (optional)
    .isp2OutputRequested = false,      // ISP2 (optional)
    .disableSubframe = true,
    .bufferCfg = {
        .maxCaptureBufferCount = 4U,   // Minimum required for CoE mode
        .maxIsp0BufferCount = 64U,     // Standard ISP buffer count
        .maxIsp1BufferCount = 64U,     // Standard ISP buffer count
        .maxIsp2BufferCount = 64U,     // Standard ISP buffer count
    }
};

// Configure pipeline for each CoE module
for (const auto& module : m_coeModules) {
    uint32_t sensorId = module.sensorId;
    uint32_t index = module.index;

    // Initialize pipeline queues for this module
    m_queues[index] = {};

    status = m_upCamera->SetPipelineCfg(sensorId, g_coePipelineCfg, m_queues[index]);
}

4. SIPL Initialization#

status = m_upCamera->Init();

5. Sensor Characterization Mode (optional, non-safety only)#

After Init() and before Start(), you can optionally place one or more sensors into characterization mode. In this mode the sensor streams a single exposure channel with gain clamping disabled and unity white-balance gains, which is required for sensor characterization workflows such as optical calibration.

#if !NV_IS_SAFETY
// Characterize sensor 0 using exposure channel 0.
// For single-exposure sensors (VB1940, AR0234), expNo must always be 0.
// For HDR sensors (IMX728), expNo selects the channel: 0=SP1H, 1=SP2, 2=SP1VS.
status = m_upCamera->SetSensorCharMode(/*index=*/0U, /*expNo=*/0U);
if (status != NVSIPL_STATUS_OK) {
    // NVSIPL_STATUS_NOT_SUPPORTED is returned when the sensor driver
    // does not implement ISensorCharMode.
    SIPL_LOG_ERR_STR_INT("SetSensorCharMode failed", static_cast<int32_t>(status));
}
#endif // !NV_IS_SAFETY

SetSensorCharMode signature:

SIPLStatus SetSensorCharMode(uint32_t const index, uint8_t expNo);

Parameter	Direction	Description
`index`	in	Zero-based sensor ID as used in `SetPipelineCfg`.
`expNo`	in	Zero-based exposure number. For single-exposure sensors this must be `0`. For the IMX728 HDR sensor: `0` = SP1H, `1` = SP2, `2` = SP1VS.

Returns NVSIPL_STATUS_OK on success, NVSIPL_STATUS_NOT_SUPPORTED if the sensor driver does not implement ISensorCharMode, or NVSIPL_STATUS_ERROR if the driver reports an internal failure.

Note

This API is only available in non-safety builds (#if !NV_IS_SAFETY). It must not be called in ASIL-B safety contexts.

Important

SetSensorCharMode must be called after Init() and before Start(). Calling it after Start() has undefined behavior.

6. Auto Control / ISP Frame Processing Rate (optional)#

By default, Auto Control (AE-AWB) and ISP run on every captured frame. To reduce CPU and ISP load on lower-power configurations or high-frame-rate sensors, you can request that the auto-control plugin and ISP run at a fraction of the capture rate by passing a FrameProcessingRate value to RegisterAutoControlPlugin.

The enum is defined in NvSIPLCamera.hpp:

enum FrameProcessingRate : std::uint8_t {
    FULL_RATE      = 0U,  // AE-AWB and ISP every frame (default)
    HALF_RATE,            // AE-AWB and ISP every 2nd frame
    THIRD_RATE,           // AE-AWB and ISP every 3rd frame
    AWB_HALF_RATE,        // AE and ISP every frame, AWB every 2nd frame
    AWB_THIRD_RATE,       // AE and ISP every frame, AWB every 3rd frame
};

Value	Description
`FULL_RATE`	Process Auto Control (AE-AWB) and ISP every frame. Default.
`HALF_RATE`	Process Auto Control (AE-AWB) and ISP every second frame.
`THIRD_RATE`	Process Auto Control (AE-AWB) and ISP every third frame.
`AWB_HALF_RATE`	Process AE and ISP every frame; process AWB every second frame.
`AWB_THIRD_RATE`	Process AE and ISP every frame; process AWB every third frame.

Pass the value as the optional last argument of RegisterAutoControlPlugin. The single-plugin overload accepts one nvsipl::FrameProcessingRate per call; the multi-plugin overload accepts a vector with one entry per registered plugin.

// Single plugin: process AE-AWB and ISP every 2nd frame.
status = m_upCamera->RegisterAutoControlPlugin(/*index=*/0U,
                                               NV_PLUGIN,
                                               autoControl,
                                               nitoBlob,
                                               /*vcIdx=*/0U,
                                               nvsipl::HALF_RATE);

// Multi-context pipeline: one rate per plugin.
std::vector<nvsipl::FrameProcessingRate> rates {
    nvsipl::FULL_RATE,
    nvsipl::AWB_HALF_RATE,
};
status = m_upCamera->RegisterAutoControlPlugin(/*index=*/0U,
                                               NV_PLUGIN,
                                               autoControls,
                                               nitoBlobs,
                                               /*vcIdx=*/0U,
                                               rates);

When the rate vector is empty (default), every plugin runs at FULL_RATE. The rate applies for the lifetime of the registered plugin and cannot be changed without re-registering after Stop().

Note

FrameProcessingRate controls how often the auto-control algorithms and ISP processing run; it does not change the sensor capture rate. Frames that are not processed by ISP are still delivered to the ICP (raw) output queue.

Auto-control ISP Override Settings#

JetPack 7.2 adds the optional ISiplControlAuto::GetIspOverrideSettings() method. An auto-control plugin can implement this method to return ISP override settings for the current frame when the default auto-control path is not sufficient.

The GTM plugin override control-point limits are defined by ISP_GTM_PLUGIN_OVERRIDE_MIN_CTRLPOINTS and ISP_GTM_PLUGIN_OVERRIDE_MAX_CTRLPOINTS. Use these constants when validating plugin-provided GTM override curves before passing them to SIPL.

Image Buffer Management#

Buffer Object Management#

The application uses a dedicated CoeBufObjManager class for managing NvSciBuf objects:

class CoeBufObjManager {
public:
    std::vector<NvSciBufObj> m_sciBufObjs;

    CoeBufObjManager(NvSciBufModule sciBufModule) : m_sciBufModule(sciBufModule) { }

    SIPLStatus AllocateBufObjs(INvSIPLCamera *siplCamera,
                               uint32_t uSensor,
                               INvSIPLClient::ConsumerDesc::OutputType output,
                               uint32_t numObjects);
    ~CoeBufObjManager();

private:
    NvSciBufModule m_sciBufModule {};
};

Buffer Registration Process#

Create buffer attributes:

NvSciBufAttrList attrList;
NvSciBufAttrListCreate(m_sciBufModule, &attrList);

// Set general attributes
NvSciBufAttrKeyValuePair attrKvp[] = {
    { NvSciBufGeneralAttrKey_Types, &bufType, sizeof(bufType) },
    { NvSciBufGeneralAttrKey_RequiredPerm, &accessPerm, sizeof(accessPerm) },
    { NvSciBufGeneralAttrKey_NeedCpuAccess, &isCpuAcccessReq, sizeof(isCpuAcccessReq) },
    { NvSciBufGeneralAttrKey_EnableCpuCache, &isCpuCacheEnabled, sizeof(isCpuCacheEnabled) }
};

Get image attributes from SIPL:

// Set general attributes first
NvSciBufAttrKeyValuePair attrKvp[] = {
    { NvSciBufGeneralAttrKey_Types, &bufType, sizeof(bufType) },
    { NvSciBufGeneralAttrKey_RequiredPerm, &accessPerm, sizeof(accessPerm) },
    { NvSciBufGeneralAttrKey_NeedCpuAccess, &isCpuAcccessReq, sizeof(isCpuAcccessReq) },
    { NvSciBufGeneralAttrKey_EnableCpuCache, &isCpuCacheEnabled, sizeof(isCpuCacheEnabled) }
};

size_t uNumAttrs = (output == INvSIPLClient::ConsumerDesc::OutputType::ICP) ? 2U : 4U;
err = NvSciBufAttrListSetAttrs(*(attrList.get()), attrKvp, uNumAttrs);

// Get image attributes from SIPL
status = siplCamera->GetImageAttributes(uSensor, output, *(attrList.get()));

Reconcile attributes:

std::unique_ptr<NvSciBufAttrList, CloseNvSciBufAttrList> reconciledAttrList;
std::unique_ptr<NvSciBufAttrList, CloseNvSciBufAttrList> conflictAttrList;
reconciledAttrList.reset(new NvSciBufAttrList());
conflictAttrList.reset(new NvSciBufAttrList());

err = NvSciBufAttrListReconcile(attrList.get(), 1U,
                                reconciledAttrList.get(), conflictAttrList.get());

Allocate buffer objects:

// Pre-allocate vector capacity to avoid reallocations during frame capture
m_sciBufObjs.reserve(numObjects);

for (size_t i = 0U; i < numObjects; i++) {
    NvSciBufObj bufObj {};
    err = NvSciBufObjAlloc(*(reconciledAttrList.get()), &bufObj);
    CHK_NVSCISTATUS_AND_RETURN(err, "NvSciBufObjAlloc()");
    CHK_PTR_AND_RETURN(bufObj, "NvSciBufObjAlloc()");

    m_sciBufObjs.push_back(bufObj);
}

Register with SIPL:

status = m_upCamera->RegisterImages(uSensor, outputType, m_sciBufObjs);

Auto Control Plug-in Registration#

For ISP processing, register the auto control plug-in:

// Check whether any ISP outputs are enabled
bool hasISPOutputs = false;
for (auto& moduleInfo : m_coeModules) {
    uint32_t index = moduleInfo.index;
    if (m_queues[index].isp0CompletionQueue != nullptr ||
        m_queues[index].isp1CompletionQueue != nullptr ||
        m_queues[index].isp2CompletionQueue != nullptr) {
        hasISPOutputs = true;
        break;
    }
}

if (!hasISPOutputs) {
    LOG_INFO("No ISP outputs enabled (ICP-only configuration)");
    return NVSIPL_STATUS_OK;
}

// Load NITO file for each sensor
struct CoeModuleData *module = CoeFirstModule();
while (module != nullptr) {
    uint32_t uSensor = module->sensorId;

    // Load NITO file
    std::vector<uint8_t> blob;
    SIPLStatus loadStatus = LoadNITOFile(m_cmdline.sNitoFolderPath, module->sensorName, blob);

    // Register plug-in
    ISiplControlAuto* autoControl = nullptr; // For NV_PLUGIN
    status = m_upCamera->RegisterAutoControlPlugin(uSensor, NV_PLUGIN, autoControl, blob);

    module = CoeNextModule();
}

Frame Processing and Consumer Threads#

Thread Architecture#

The application creates multiple threads per camera module:

ICP Thread: Processes raw image data from Image Capture Pipeline.
ISP0/ISP1/ISP2 Threads: Process YUV data from Image Signal Processor.
Event Thread: Handles pipeline notifications.
CPU Signal Thread: Manages CPU signaling.

Consumer Classes#

Base Consumer Class#

class CoeConsumerBase {
public:
    virtual SIPLStatus ProcessBuffer(INvSIPLClient::INvSIPLNvMBuffer* pBuffer,
                                     uint32_t frameNum, uint32_t cameraModule) = 0;
    virtual ConsumerType GetType() const = 0;
    virtual const char* GetTypeName() const = 0;
};

RAW Consumer (ICP Output)#

class CoeRawConsumer : public CoeConsumerBase {
private:
    uint32_t GetDynamicRawBufferSize(INvSIPLClient::INvSIPLNvMBuffer* pBuffer);
    SIPLStatus SaveRawFrame(INvSIPLClient::INvSIPLNvMBuffer* pBuffer,
                            uint32_t frameNum, uint32_t cameraModule);
};

YUV Consumer (ISP Output)#

class CoeYuvConsumer : public CoeConsumerBase {
private:
    INvSIPLClient::ConsumerDesc::OutputType m_outputType;
    NvSciSyncCpuWaitContext m_cpuWaitContext;

    SIPLStatus SaveYuvFrame(INvSIPLClient::INvSIPLNvMBuffer* pBuffer,
                            uint32_t frameNum, uint32_t cameraModule);
};

Thread Processing Loop#

void SIPLCoeCamera::CoeImageThread(uint32_t const cameraModule, uint32_t const threadIndex) {
    LOG_DBG("CoE Image Thread %u for module %u started", threadIndex, cameraModule);

    // Create appropriate consumer using factory
    std::unique_ptr<CoeConsumerBase> consumer = CoeConsumerFactory::CreateConsumer(
        threadIndex, cameraModule, m_cpuWaitContext,
        m_cmdline.bEnableRaw, m_cmdline.bDisableISP0,
        m_cmdline.bDisableISP1, m_cmdline.bDisableISP2,
        m_cmdline.uNumWriteFrames);

    if (!consumer) {
        LOG_ERR("Failed to create consumer for thread index: %u", threadIndex);
        return;
    }

    SIPLStatus status = NVSIPL_STATUS_OK;
    INvSIPLClient::INvSIPLBuffer *pBuffer = nullptr;
    ThreadData *threadData = &m_threadData[cameraModule][threadIndex];
    m_threadReady[cameraModule][threadIndex] = true;

    while (!m_exitAllThreads && (status != NVSIPL_STATUS_EOF)) {
        if (threadData->imageQueue) {
            status = threadData->imageQueue->Get(pBuffer, 1000000); // 1 second timeout

            if (status == NVSIPL_STATUS_OK && pBuffer != nullptr) {
                m_queueCounts[cameraModule][threadIndex]++;

                // Cast to NvM buffer for processing
                INvSIPLClient::INvSIPLNvMBuffer *pNvMBuffer =
                    dynamic_cast<INvSIPLClient::INvSIPLNvMBuffer *>(pBuffer);

                if (pNvMBuffer != nullptr) {
                    uint32_t frameNumber = m_queueCounts[cameraModule][threadIndex];

                    // Process buffer based on consumer type
                    if (m_cmdline.bEnableRaw && consumer->GetType() == ConsumerType::RAW_CONSUMER) {
                        SIPLStatus processStatus = consumer->ProcessBuffer(pNvMBuffer, frameNumber, cameraModule);
                    } else if (threadIndex >= THREAD_INDEX_ISP0 && threadIndex <= THREAD_INDEX_ISP2) {
                        SIPLStatus status = WriteISPBufferToFile(pNvMBuffer, frameNumber, cameraModule, threadIndex);
                    }
                }

                // Return buffer to SIPL
                SIPLStatus releaseStatus = pBuffer->Release();
                pBuffer = nullptr;
            }
        } else {
            usleep(10000); // 10ms
        }
    }
}

File Output and Frame Dumping#

RAW Frame Saving#

The application automatically calculates RAW buffer sizes based on sensor configuration:

uint32_t GetDynamicRawBufferSize(INvSIPLClient::INvSIPLNvMBuffer* pBuffer) {
    NvSciBufObj rawBufObj = pBuffer->GetNvSciBufImage();

    // Get buffer attributes
    NvSciBufAttrKeyValuePair imgAttrs[] = {
        { NvSciBufImageAttrKey_PlanePitch, NULL, 0 },
        { NvSciBufImageAttrKey_PlaneHeight, NULL, 0 },
    };

    NvSciBufAttrListGetAttrs(bufAttrList, imgAttrs, sizeof(imgAttrs) / sizeof(imgAttrs[0]));

    uint32_t pitch = *(static_cast<const uint32_t*>(imgAttrs[0].value));
    uint32_t height = *(static_cast<const uint32_t*>(imgAttrs[1].value));

    return pitch * height;
}

Frame File Naming Convention#

RAW files: /tmp/coe_sensor<N>_raw_frame_<N>.raw
YUV files: /tmp/<prefix>_sensor<N>_<output>_frame_<N>.yuv

<N> is the sensor or frame number. <prefix> is a configurable prefix (default: nvsipl_camera). <output> is ISP0, ISP1, or ISP2.

File Writer Implementation#

Steps required to write image data to files, building on the normal streaming process:

Obtain the buffer object:

NvSciBufObj bufObj = pBuffer->GetNvSciBufImage();

Get the buffer properties:

NvSciBufAttrList bufAttrList;
NvSciBufObjGetAttrList(bufObj, &bufAttrList);

NvSciBufAttrKeyValuePair imgAttrs[] = {
    { NvSciBufImageAttrKey_PlaneCount, NULL, 0 },
    { NvSciBufImageAttrKey_PlaneColorFormat, NULL, 0 },
    { NvSciBufImageAttrKey_PlaneHeight, NULL, 0 },
    { NvSciBufImageAttrKey_PlaneWidth, NULL, 0 },
    { NvSciBufImageAttrKey_PlanePitch, NULL, 0 },
    { NvSciBufImageAttrKey_PlaneAlignedSize, NULL, 0 },
    { NvSciBufImageAttrKey_PlaneBitsPerPixel, NULL, 0 },
    { NvSciBufGeneralAttrKey_CpuNeedSwCacheCoherency, NULL, 0 }
};

NvSciBufAttrListGetAttrs(bufAttrList, imgAttrs, sizeof(imgAttrs) / sizeof(imgAttrs[0]));

Determine the file extension by format:
- raw for Bayer images.
- yuv for YUV images.

Handle ISP output synchronization:

if (isISPOutput) {
    NvSciSyncFence fence = NvSciSyncFenceInitializer;
    status = pBuffer->GetEOFNvSciSyncFence(&fence);
    NvSciSyncFenceWait(&fence, m_cpuWaitContext, FENCE_FRAME_TIMEOUT_MS);
    NvSciSyncFenceClear(&fence);
}

Format Override Support#

ICP Output Formats

Based on sensor configuration, set the correct input format type and pixel order in platform configuration.
CoE sensors typically do not allow format overriding because they have fixed output formats.

ISP Output Formats

Users can specify different ISP output formats as long as they are supported.
Before calling GetImageAttributes, override ISP output formats to apply user preferences.
The default format is YUV 420 SemiPlanar UINT 8 BlockLinear.

Refer to SIPL documentation for supported format combinations.

// Example: Override ISP format before GetImageAttributes
NvSciBufAttrKeyValuePair formatOverride[] = {
    { NvSciBufImageAttrKey_PlaneColorFormat, &desiredFormat, sizeof(desiredFormat) }
};
NvSciBufAttrListSetAttrs(attrList, formatOverride, 1);

NvSciStream Integration#

The NvSciStream library provides utilities to construct streaming applications for use cases that have complex buffer handling and synchronization. It also helps simplify IPC and IVC use cases.

High-level steps for setup (analogous to steps 5 and 7 of normal streaming):

Create a static pool of packets for a producer.
Create a producer for the static pool.
Connect the producer to the downstream consumer.
Ensure that the static pool, producer, queue, and downstream consumer are connected.
Create and reconcile buffer attributes across producer and consumer, ensuring that they are present in the stream.
Set up and reconcile the synchronization attributes across producer and consumer to ensure that they are present in the stream.
Allocate NvSciSyncObj for producer, register with producer, and inform the stream that the sync object is set.
Allocate NvSciSyncObj for consumer, register with consumer, and inform the stream that the sync object is set.
Get the consumer sync object and register with producer and vice versa.
Allocate buffers.
Create PoolPackets and insert buffers into the packets.
Replace the buffers and insert cookies as identifiers.
Ensure that the PacketsComplete event is observed.
Set status to PacketImport.
Check whether setup is complete.
For each packet, ensure that it is ready.

High-level steps for streaming on the producer side (SIPL):

Have a Post function to send buffers to consumers. In normal streaming, this function is called from the pipeline queue and performs the following actions:
1. Calls GetEOFNvSciSyncFence and GetNvSciBufImage.
2. Finds the cookie of the buffer.
3. Calls AddRef on the buffer.
4. Informs the stream that the fence for the packet with the cookie ID has been set.
5. Informs the stream that the packet is present.
6. Increments the number of buffers with consumers and clears the fence.
Have a thread running to receive buffers from consumers. This thread needs to perform the following actions as long as the number of buffers with consumers is greater than 0:
1. Query events.
2. When the packet is ready, get the fence and packet.
3. Decrement the number of buffers with consumers.
4. Call AddNvSciSyncPrefence on the buffer.
5. Call Release for the buffer.
6. Clear the fence.
Implement a similar function and thread on the consumer.

For complex multi-consumer scenarios, NvSciStream can simplify buffer management.

Key Benefits

Simplified IPC and IVC use cases.
Complex buffer handling automation.
Streamlined synchronization management.

Setup Overview

Create a static pool of packets for producer.
Create a producer for the static pool.
Connect the producer to downstream consumers.
Reconcile buffer and sync attributes across the stream.
Allocate and register NvSciSyncObj for producer and consumer.
Exchange sync objects between producer and consumer.
Allocate buffers and create pool packets.
Complete setup and verify readiness.

Streaming Process

// Producer side (SIPL)
void PostBuffer(INvSIPLClient::INvSIPLBuffer* pBuffer) {
    NvSciSyncFence fence = NvSciSyncFenceInitializer;
    NvSciBufObj bufObj = pBuffer->GetNvSciBufImage();

    // Find cookie and notify stream
    uint32_t cookie = FindBufferCookie(bufObj);
    pBuffer->AddRef();

    // Set fence and present packet
    SetPacketFence(cookie, fence);
    PresentPacket(cookie);

    NvSciSyncFenceClear(&fence);
}

// Consumer thread
void ConsumerThread() {
    while (numBuffersWithConsumers > 0) {
        QueryEvents();
        if (PacketReady()) {
            auto [fence, packet] = GetPacketAndFence();
            numBuffersWithConsumers--;

            // Process and return buffer
            AddNvSciSyncPrefence(buffer, fence);
            ReleaseBuffer(buffer);
            NvSciSyncFenceClear(&fence);
        }
    }
}

Error Handling and Notifications#

Event Processing#

The application processes various pipeline notifications:

void SIPLCoeCamera::OnEvent(NvSIPLPipelineNotifier::NotificationData &oNotificationData) {
    switch (oNotificationData.eNotifType) {
        case NvSIPLPipelineNotifier::NOTIF_INFO_ICP_PROCESSING_DONE:
            m_uNumFrameCaptured++;
            break;
        case NvSIPLPipelineNotifier::NOTIF_WARN_ICP_FRAME_DROP:
            m_uNumFrameDrops++;
            break;
        case NvSIPLPipelineNotifier::NOTIF_ERROR_ICP_CAPTURE_FAILURE:
            m_bInError = true;
            break;
        // ... handle other notification types
    }
}

Common Error Types#

Frame Drops: Network congestion or processing delays.
Capture Failures: Hardware or communication issues.
Processing Failures: ISP processing errors.

Configuration Files#

JSON Configuration File#

The following is an example of a JSON configuration file for a CoE camera. For more details, refer to SIPL Query JSON Guide.

{
    "cameraConfigs": [
        {
            "name": "VB1940_Camera",
            "type": "CoE",
            "platform": "VB1940",
            "sensorInfo": {
                "id": 0,
                "name": "VB1940",
                "sensorGroup": 0,
                "deviceIndex": 0
            },
            "mac_address": "8c:1f:64:6d:70:03",
            "ip_address": "192.168.1.2",
            "hsb_id": 0
        }
    ],
    "transportSettings": [
        {
            "name": "VB1940_Camera",
            "type": "CoE",
            "if_name": "mgbe0_0",
            "ip_address": "192.168.1.2",
            "hsb_id": 0,
            "sync_sensors": false
        }
    ]
}

For stereo CoE configurations, do not use the removed cameraConfigs[].CoECamera.isStereo field. Set both camera entries to the same non-zero sensorInfo.sensorGroup value, assign deviceIndex values 0 and 1, and set transportSettings[].sync_sensors to true. The shipped stereo configuration name is VB1940_Stereo.

CoE Override CSV Format#

# Format: hsb_id, <HSB-ID>, <interface-name>, <MAC-address>, <IP-address>
hsb_id,0,mgbe0_0,8c:1f:64:6d:70:03,192.168.1.2

Troubleshooting#

Common Issues#

Camera not detected
- Verify that the mgbe0_0 interface is up and the IP address is set.
- Ping the sensor.
- If you are using an override file, ensure that the MAC and IP addresses in the CoE override file are correct.
NITO file errors
- Verify the NITO file path.
- Check file permissions.
- Ensure that the sensor name matches the NITO filename. For example, vb1940.nito.

Debug Commands#

# Enable maximum verbosity
nvsipl_camera -c VB1940_Camera -v 4 -H

# List configs from a custom JSON (requires -t)
nvsipl_camera -t /path/to/config.json -l -H

Log Analysis#

Monitor application logs for the following:

Buffer allocation failures
Network timeout errors
Frame sequence discontinuities
Pipeline queue overflows

SIPL Camera Application Developer Guide#

NVIDIA SIPL Camera Driver Package#

Repository Layout#

Build Prerequisites#

Building the Package#

CMake Options for Applications#

Key Build Outputs#

Installing on the Target#

Running Sample Applications#

nvsipl_camera#

Key Features#

System Components#

Pipeline Overview#

Application Setup and Configuration#

Prerequisites#

Installation#

Configuration Methods#

Usage and Command-Line Options#

Basic Usage Examples#

Command-Line Options#

Configuration Sources#

Verbosity Levels#

Normal Streaming with CoE Cameras#

Initialization Sequence for CoE-specific Implementation#

1. SIPL Setup#

2. Platform Configuration#

3. Pipeline Configuration#

4. SIPL Initialization#

5. Sensor Characterization Mode (optional, non-safety only)#

6. Auto Control / ISP Frame Processing Rate (optional)#

Auto-control ISP Override Settings#

Image Buffer Management#

Buffer Object Management#

Buffer Registration Process#

Auto Control Plug-in Registration#

Frame Processing and Consumer Threads#

Thread Architecture#

Consumer Classes#

Base Consumer Class#

RAW Consumer (ICP Output)#

YUV Consumer (ISP Output)#

Thread Processing Loop#

File Output and Frame Dumping#

RAW Frame Saving#

Frame File Naming Convention#

File Writer Implementation#

Format Override Support#

NvSciStream Integration#

Error Handling and Notifications#

Event Processing#

Common Error Types#

Configuration Files#

JSON Configuration File#

CoE Override CSV Format#

Troubleshooting#

Common Issues#

Debug Commands#

Log Analysis#

Additional Resources#

`nvsipl_camera`#