Camera Software Development Solution for Jetson Thor

This documentation provides a comprehensive guide to NVIDIA^® Jetson™ camera software solution on the Jetson Thor platform. It outlines and explains the new Safe Image Processing Library (SIPL) framework and the development options for camera applications and sensor drivers for Camera over Ethernet (CoE) solutions.

The content is organized to take you from understanding the basic concepts to implementing custom camera solutions using the SIPL framework and CoE architecture on the Thor platform.

Getting Started

For developers new to the NVIDIA camera software stack, we recommend following this sequence:

1. Introduction to SIPL (single entry point)

   Read the full introduction first; it includes the SIPL Architecture Overview section (SIPL Architecture Overview) with the CoE control/data diagram and a pointer to the GMSL architecture guide for CSI/GMSL-specific diagrams.

   Introduction to SIPL

   Introduction to the Safe Image Processing Library (SIPL), NVIDIA’s modular, extensible framework for camera and image sensor integration, image signal processing, and control. Covers fundamental concepts, architecture, and basic usage patterns for high reliability and safety.

2. Camera-over-Ethernet (CoE) solution deep dive

   After the introduction, continue here for CoE-specific components, HSB (CSI2Eth bridge), and Ethernet-based integration on Jetson Thor.

   Camera-over-Ethernet Overview

   Comprehensive overview of the CoE solution architecture, NVIDIA’s next-generation camera connectivity solution enabling high-performance, flexible, and scalable camera integration using standard Ethernet networks via Holoscan Sensor Bridge (HSB, or CSI2Eth bridge).

3. Hardware Sequence Language and UDDF Overview

   Understand the hardware abstraction layer and driver framework for camera devices.

   HSL and UDDF Overview

   Overview of Hardware Sequence Language (HSL), a framework for specifying sequences of hardware accesses, and Unified Device Driver Framework (UDDF), NVIDIA’s standardized approach for developing device drivers in the camera and multimedia domain.

4. Holoscan Sensor Bridge (HSB)

   Holoscan Sensor Bridge (HSB)

   HSB overview for CoE on Jetson Thor: FPGA bridge from sensor to Ethernet, role in CoE bring-up, and link to the Holoscan Sensor Bridge product documentation.

Development Guides

After you understand the basic concepts, proceed with the following development guides:

Configuration and Tools

SIPL Query JSON Guide

Comprehensive guide to SIPL query configuration using JSON:

• JSON configuration syntax and structure.

• Query parameter reference.

• Configuration examples and best practices.

Application Development

SIPL Camera Application Developer Guide

Comprehensive guide for developing camera applications using SIPL with CoE cameras:

• Setting up the development environment for CoE camera systems.

• Creating camera applications with multiple output formats (ICP raw, ISP processed YUV).

• Dynamic buffer management and multi-threaded processing.

• Auto Control Plug-in integration and debugging techniques.

Sensor Programming with Python Development Tools

PyHSL

Guide for using Python-based Hardware Sequence Language (PyHSL) tools:

• Python API reference and usage for hardware control sequences.

• Rapid prototyping techniques for sensor programming.

• Debugging and testing with Python tools.

Sensor Driver Development

Guide to Writing UDDF Drivers

Detailed guide for writing Unified Device Driver Framework (UDDF) drivers for CoE cameras via HSB:

• UDDF driver architecture and components for safety-certifiable drivers.

• Implementation guidelines focusing on hardware operation sequences.

• Testing and validation procedures for driver development.

Integrating UDDF Drivers with SIPL

Guide for integrating UDDF drivers with the SIPL framework:

• Making UDDF drivers discoverable to SIPL.

• Integration patterns and workflows for camera and transport drivers.

• Configuration management and performance optimization.

Reference Documentation

SIPL Notifications

Camera SIPL Notifications

Reference documentation for the SIPL notification system:

• Notification types and formats for camera events.

• Event handling mechanisms and error reporting.

• Error codes and troubleshooting guidelines.

Supported Image Formats

SIPL Image Formats

Complete reference for supported image formats in NvMedia and NvSIPL:

• Supported pixel formats and encodings for CoE cameras.

• Format conversion capabilities and pipeline outputs.

• Performance characteristics for various image formats.

Support and Additional Resources

For additional support and the latest updates to this documentation:

• Contact your NVIDIA support representative.

• Refer to the NVIDIA Developer documentation portal.

• Join the NVIDIA Developer forums for community support.

• For details on building a custom camera solution:

  • Consult a member of the NVIDIA Preferred Partners Community.

  • Refer to Jetson Partner Supported Cameras.

GMSL Camera Development

For developers building GMSL (Gigabit Multimedia Serial Link) camera solutions on supported NVIDIA Jetson platforms:

GMSL Camera Development Guide

Complete guide to GMSL camera development using SIPL and UDDF:

• GMSL hardware topology and the five UDDF driver types.

• Writing and integrating deserializer, module, serializer, and power drivers.

• Sensor System Config JSON schema for GMSL cameras.

• Driver name matching and installation.

Note

This documentation is specifically tailored for the Jetson Thor platform with GMSL and CoE architecture.

For other NVIDIA platforms, refer to the platform-specific documentation.

Important

Always ensure you are using the correct version of this documentation that matches your Jetson Thor platform software release.

Glossary

SIPL Glossary

Term	Description
API	Application programming interface.
CDI	Common Driver Infrastructure (hardware abstraction in UDDF).
CoE	Camera-over-Ethernet (Ethernet-based camera transport).
CSI	Camera Serial Interface.
CSI2Eth	Camera Serial Interface to Ethernet bridge.
DDI	Device Driver Interface (public driver interface in UDDF).
GMSL	Gigabit Multimedia Serial Link (high-speed serial camera interface).
GPIO	General-purpose input/output.
HAL	Hardware Abstraction Layer (CameraHAL-UDDF).
HSL	Hardware Sequence Language (hardware access scripting for drivers).
HSB	Holoscan Sensor Bridge.
I2C	Inter-Integrated Circuit (hardware communication protocol).
ICP	Image Capture Pipeline (ISP-processed YUV).
ISP	Image signal processor.
JSON	JavaScript Object Notation (used for configuration and database).
MGBE	Multi-Gigabit Ethernet (MGBE controller).
MIPI	Mobile Industry Processor Interface.
PHY	Physical Layer (network hardware interface).
PyHSL	Python-based Hardware Sequence Language.
RCE	Real-Time Camera Engine (camera firmware).
SIPL	Safe Image Processing Library.
UDDF	Unified Device Driver Framework (standardized driver model).