Camera Software Development Solution for Jetson Thor

This documentation provides a comprehensive guide to NVIDIA^® Jetson™ camera software solution on the Jetson Thor platform. It outlines and explains the new Safe Image Processing Library (SIPL) framework and the development options for camera applications and sensor drivers for Camera over Ethernet (CoE) solutions.

The content is organized to take you from understanding the basic concepts to implementing custom camera solutions using the SIPL framework and CoE architecture on the Thor platform.

Getting Started

For developers new to the NVIDIA camera software stack, we recommend following this sequence:

1. SIPL Framework Introduction

   Learn about the core imaging pipeline framework.

   Introduction to SIPL

   Introduction to the Safe Image Processing Library (SIPL), NVIDIA’s modular, extensible framework for camera and image sensor integration, image signal processing, and control. Covers fundamental concepts, architecture, and basic usage patterns for high reliability and safety.

2. CoE Architecture Overview

   Start here to understand the overall architecture and components of the Camera-over-Ethernet (CoE) software solution on the Jetson Thor platform.

   Camera-over-Ethernet Overview

   Comprehensive overview of the CoE solution architecture, NVIDIA’s next-generation camera connectivity solution enabling high-performance, flexible, and scalable camera integration using standard Ethernet networks via Holoscan Sensor Bridge (HSB, or CSI2Eth bridge).

3. Hardware Sequence Language and UDDF Overview

   Understand the hardware abstraction layer and driver framework for camera devices.

   HSL and UDDF Overview

   Overview of Hardware Sequence Language (HSL), a framework for specifying sequences of hardware accesses, and Unified Device Driver Framework (UDDF), NVIDIA’s standardized approach for developing device drivers in the camera and multimedia domain.

4. Holoscan Sensor Bridge (HSB)

   The Holoscan Sensor Bridge (HSB) is an FPGA-based interface developed by NVIDIA to enable real-time, low-latency streaming of sensor data over Ethernet directly into GPU memory for processing. Peripheral device data is acquired by the HSB device FPGA and sent over Ethernet to the host system (the Jetson Thor platform).

   For more details on HSB, refer to Holoscan Sensor Bridge

Development Guides

After you understand the basic concepts, proceed with the following development guides:

Configuration and Tools

SIPL Query JSON Guide for CoE Camera Development

Comprehensive guide to SIPL query configuration using JSON:

• JSON configuration syntax and structure.

• Query parameter reference.

• Configuration examples and best practices.

Application Development

NvSIPL CoE Camera Application Developer Guide

Comprehensive guide for developing camera applications using SIPL with CoE cameras:

• Setting up the development environment for CoE camera systems.

• Creating camera applications with multiple output formats (ICP raw, ISP processed YUV).

• Dynamic buffer management and multi-threaded processing.

• Auto Control Plug-in integration and debugging techniques.

Sensor Programming with Python Development Tools

PyHSL

Guide for using Python-based Hardware Sequence Language (PyHSL) tools:

• Python API reference and usage for hardware control sequences.

• Rapid prototyping techniques for sensor programming.

• Debugging and testing with Python tools.

Sensor Driver Development

Guide to Writing UDDF Drivers

Detailed guide for writing Unified Device Driver Framework (UDDF) drivers for CoE cameras via HSB:

• UDDF driver architecture and components for safety-certifiable drivers.

• Implementation guidelines focusing on hardware operation sequences.

• Testing and validation procedures for driver development.

Integrating UDDF Drivers with SIPL

Guide for integrating UDDF drivers with the SIPL framework:

• Making UDDF drivers discoverable to SIPL.

• Integration patterns and workflows for camera and transport drivers.

• Configuration management and performance optimization.

Reference Documentation

SIPL Notifications

Camera SIPL Notifications

Reference documentation for the SIPL notification system:

• Notification types and formats for camera events.

• Event handling mechanisms and error reporting.

• Error codes and troubleshooting guidelines.

Supported Image Formats

SIPL Guidance on Output Image Formats

Complete reference for supported image formats in NvMedia and NvSIPL:

• Supported pixel formats and encodings for CoE cameras.

• Format conversion capabilities and pipeline outputs.

• Performance characteristics for various image formats.

Support and Additional Resources

For additional support and the latest updates to this documentation:

• Contact your NVIDIA support representative.

• Refer to the NVIDIA Developer documentation portal.

• Join the NVIDIA Developer forums for community support.

• For details on building a custom camera solution:

  • Consult a member of the NVIDIA Preferred Partners Community.

  • Refer to Jetson Partner Supported Cameras.

Note

This documentation is specifically tailored for the Jetson Thor platform with CoE architecture. For other NVIDIA platforms, refer to the platform-specific documentation.

Important

Always ensure you are using the correct version of this documentation that matches your Jetson Thor platform software release.

Glossary

SIPL Glossary

Term	Description
API	Application programming interface.
CDI	Common Driver Infrastructure (hardware abstraction in UDDF).
CoE	Camera-over-Ethernet (Ethernet-based camera transport).
CSI	Camera Serial Interface.
CSI2Eth	Camera Serial Interface to Ethernet bridge.
DDI	Device Driver Interface (public driver interface in UDDF).
GMSL	Gigabit Multimedia Serial Link (high-speed serial camera interface).
GPIO	General-purpose input/output.
HAL	Hardware Abstraction Layer (CameraHAL-UDDF).
HSL	Hardware Sequence Language (hardware access scripting for drivers).
HSB	Holoscan Sensor Bridge.
I2C	Inter-Integrated Circuit (hardware communication protocol).
ICP	Image Capture Pipeline (ISP-processed YUV).
ISP	Image signal processor.
JSON	JavaScript Object Notation (used for configuration and database).
MGBE	Multi-Gigabit Ethernet (MGBE controller).
MIPI	Mobile Industry Processor Interface.
PHY	Physical Layer (network hardware interface).
PyHSL	Python-based Hardware Sequence Language.
RCE	Real-Time Camera Engine (camera firmware).
SIPL	Safe Image Processing Library.
UDDF	Unified Device Driver Framework (standardized driver model).