Introduction

Getting Started

View as Markdown

What You’ll Need

Essential Hardware Components

  • HSB Development Kit - Lattice CPNX100-ETH-SENSOR-BRIDGE
  • Camera - HSB supported sensors include Leopard Imaging VB1940 Eagle Camera, Leopard Imaging IMX274
  • Photodetector - Silicon photodiode with broad spectrum response (400-1000nm) and high-frequency response (>1MHz)
  • LED - High-brightness white LED (12,000-14,000 mcd, 20mA, 3.0-3.2V) for visual timing reference

Hardware Component Schematics

LED and Photodetector Circuit Configuration

LED and Photodetector Schematic

HSB GPIO Connections

HSB GPIO Connections

The schematics provide the essential wiring diagrams for setting up the HSB Native Latency Measurement Tool. Please review these diagrams carefully before assembling your hardware setup to ensure proper connections and avoid potential damage to components.

Each schematic shows the specific pin assignments, voltage levels, and connection requirements for the measurement system components.

Bill of Materials (BOM)

Refer to the Bill of Materials (BOM) - LED/Photodetector Circuit for the complete list of components and their specifications.

Camera-to-Display Setup Diagram

The diagram below shows the complete system configuration for HSB Native Latency measurement. Use this as your reference guide when connecting all components together to ensure proper signal flow and timing synchronization throughout the measurement pipeline.

Camera-to-Display Setup Diagram

Complete system overview showing camera, HSB, host system, and measurement components

Key Timing Events

The diagram illustrates three critical timestamps that enable precise latency measurement:

  • FSR_T (Frame Start Received) - Timestamp when the camera frame start signal is received at the HSB FPGA, triggering the LED activation sequence
  • LED_T (LED Triggered) - Timestamp when the LED_ON event occurs at GPIO pin 0, marking the visual reference point
  • PD_T (Photodetector Triggered) - Timestamp when the PD_TRIGGERED event is detected, indicating the LED signal has been captured from the display

The camera-to-display latency is calculated as: Latency = PD_T - LED_T

Rolling Shutter vs Global Shutter Considerations

Shutter Type Differences

Understanding your camera’s shutter type is crucial for accurate latency measurement, as each type requires different timing strategies and has distinct latency characteristics.

Shutter Type Comparison

Rolling Shutter Behavior

Capture Process:

  • Pixels are captured row-by-row sequentially
  • Each row has a different capture timestamp
  • Frame readout occurs over time (typically 16-33ms)

Latency Implications:

  • Variable Latency: Different rows have different capture times
  • Jitter Source: Row-to-row timing variations contribute to measurement uncertainty
  • Measurement Challenge: Need to account for exposure timing across the frame

Software Event Setup:

  • FSR Event Timing: FSR event triggers LED activation during frame readout
  • Readout Time Window: LED can be captured in the current frame being read
  • Timing Strategy: LED activated during the frame readout period
  • Current Frame Capture: LED light appears in the same frame as the trigger event

Global Shutter Behavior

Capture Process:

  • All pixels captured simultaneously at a single moment
  • Uniform exposure timing across entire frame
  • Readout occurs after capture is complete

Latency Implications:

  • Consistent Latency: All pixels captured at the same time point
  • Reduced Jitter: Uniform timing across the entire frame
  • Simpler Measurement: Single, well-defined capture time point

Software Event Setup:

  • FSR Event Timing: FSR event occurs after frame capture is complete
  • No Current Frame Window: Frame already captured when FSR is received
  • Next Frame Strategy: LED must be delayed for the next frame
  • Delayed Capture: LED light appears in the subsequent frame

Setup Configuration Differences

The key difference between shutter types is the LED trigger delay configuration:

Rolling Shutter Configuration

In Rolling shutter, the LED triggers on the current frame - the LED can be captured within the same frame that generated the FSR event.

$# Rolling shutter - immediate LED trigger
$# No delay, LED triggers during frame readout
$led_trigger_delay = 0

Global Shutter Configuration

In Global shutter, the LED triggers for the next frame - we must wait for the subsequent frame since the current frame capture is already complete when FSR is received

$# Global shutter - delayed LED trigger based on sensor readout time
$# Delay based on specific sensor mode readout timing
$led_trigger_delay = sensor_readout_time

Important: The LED trigger delay for global shutter cameras must be configured based on the specific sensor mode’s readout time to ensure proper next-frame synchronization.

Timing Sequence Diagrams

Rolling Shutter Sequence

The following sequence diagram illustrates the timing flow for rolling shutter cameras, showing how events are synchronized within the current frame:

Rolling Shutter Sequence Diagram

Event Sequence:

  1. Frame Start - Camera begins row-by-row capture
  2. FSR_T - HSB FPGA receives frame start signal
  3. LED_T - LED immediately triggers (delay = 0) during frame readout
  4. Frame Processing - Current frame continues through pipeline
  5. Display Output - Frame appears on display with LED visible
  6. PD_T - Photodetector captures LED from display

Key Timing Characteristics:

  • LED activation occurs during the current frame readout period
  • Zero delay between FSR and LED trigger
  • LED appears in the same frame that generated the FSR event
  • Latency calculation: PD_T - LED_T

System Requirements

Supported Platforms

  • NVIDIA Jetson AGX Orin
  • NVIDIA Jetson IGX Orin with NVIDIA RTX A6000 dGPU (used for testing and performance results)

Software Requirements

  • NVIDIA Jetson AGX Orin: JetPack 6.2 or later with Ubuntu 22.04 - Verify with lsb_release -a
  • NVIDIA IGX Orin: IGX Software (BaseOS) 1.1.2 or later - IGX Orin does not use JetPack; refer to the IGX Software documentation for setup
  • Holoscan Sensor Bridge Software - Clone from GitHub: git clone https://github.com/nvidia-holoscan/holoscan-sensor-bridge

Camera Compatibility

The HSB Native Latency Measurement Tool works with all sensors supported by the Holoscan Sensor Bridge. Example applications are included for the following sensors: