Examples

Holoscan sensor bridge Python example applications are located under the examples directory.

Below are instructions for running the applications on the AGX Thor, DGX Spark, IGX and the Jetson AGX Orin platforms.

Choosing platform-appropriate examples

Holoscan Sensor Bridge supports different hardware accelerated transport strategies compatible with a wide range of hosts. Each example is hard-coded to a specific receiver type; examples are prefixed to indicate the receiver operator used.

RoceReceiver (ConnectX Host)

• Examples without any prefix in the filename use the RoCEv2-accelerated network receiver operator and require a ConnectX-enabled platform like IGX Orin, IGX T7000, or DGX Spark. The sensor module must be connected to the ConnectX-enabled port on these systems, not the RJ45 connector. AGX Orin, AGX Thor, and IGX T5000 systems cannot run these examples.
• For accelerated networking examples on IGX T7000, CUDA_VISIBLE_DEVICES environment variable will have to be set to the matching graphics GPU configuration, which defaults to iGPU. For example, if dGPU graphics are configured, imx274_player must have CUDA_VISIBLE_DEVICES set to 0 to select RDMA and graphics to the dGPU.
• These examples work on both iGPU and dGPU configurations, provided that the container was built with the appropriate iGPU or dGPU setting.

LinuxReceiver (Unaccelerated)

• Examples starting with the word linux use the unaccelerated Linux Sockets API network receiver operator. These examples work on all systems.

FusaCoeCapture (Thor MGBE only)

• Examples starting with fusa_coe use the hardware-accelerated CoE capability found on Thor’s MGBE interface, and require an AGX Thor or IGX T5000. The sensor module must be connected to the host’s QSFP MGBE port, not the RJ45 port.

SiplCameraOutput (Thor MGBE only)

• Examples starting with “sipl_” use the SIPL camera processing stack, which requires AGX Thor or IGX T5000. The sensor module must have a UDDF driver and be connected to the host’s QSFP MGBE port, not the RJ45 port.

IMX274 player example

To run the video player IMX274 example on a ConnectX-enabled host, in the demo container, run

1
  $ python3 examples/imx274_player.py

1
  $ python3 examples/linux_imx274_player.py

Documentation breaking down the source code for the IMX274 player application is available here; this example illustrates the basic sensor bridge workflow which is described in the architecture documentation. Press Control/C to stop the video player.

Leopard imaging VB1940 Eagle player example

This example is similar to the IMX274 player example above, using an LI VB1940 Eagle camera instead of IMX274. To run the high-speed video player with LI VB1940 Eagle, in the demo container with a ConnectX accelerated host, run

1
$ python3 examples/vb1940_player.py

for unaccelerated hosts,

1
$ python3 examples/linux_vb1940_player.py

lastly, running SIPL accelerated network python example on AGX Thor or IGX T5000:

1
$ python3 ./examples/sipl_player.py --json-config ./examples/sipl_config/vb1940_single.json

MAX96716A deserializer player example

The TauroTech DA326 sensor bridge board has an on-board Analog Devices MAX96716A GMSL deserializer, which includes a built-in Video Pattern Generator (VPG).

The max96716a_player examples drive the VPG to emit a synthetic stream out of the deserializer’s MIPI CSI-2 output without any additional hardware.

````{tab-item} Python
For ConnectX-enabled hosts:
```sh
python3 examples/max96716a_player.py
```
For unaccelerated configurations:
```sh
python3 examples/linux_max96716a_player.py
```
For AGX Thor and T5000 using MGBE with FuSa CoE accelerated receiver:
```sh
python3 examples/fusa_coe_max96716a_player.py
```
````
````{tab-item} C++
The C++ examples need to be built first. From the repository root:
```sh
mkdir -p build && cd build
cmake ..
make -j
```
This leaves the resulting executables in `build/examples/`. From the `build/`
directory, you can then run the `max96716a_player` variants.
For ConnectX accelerated hosts:
```sh
./examples/max96716a_player
```
For unaccelerated configurations:
```sh
./examples/linux_max96716a_player
```
For AGX Thor or IGX T5000 with FuSa CoE accelerated receiver:
```sh
./examples/fusa_coe_max96716a_player
```
````

Hawk camera + Tauro Tech DA326 player example

This example is similar to the IMX274 and VB1940 player examples above, using a TauroTech DA326 sensor bridge board with one or more Hawk camera modules connected.

Please note that the following arguments are required:

• Single-link applications: --channel A|B to select the GMSL link to use, and --sensor left|right|both to select which sensor(s) to enable receiving from.
• Dual-link applications: --sensor left|right to select which sensor side to enable receiving from (both GMSL links are used).

Run with --help for the full list of options.

````{tab-item} Python
For ConnectX-accelerated platforms:
```sh
python3 examples/hawk_single_link_player.py
```
For unaccelerated configurations:
```sh
python3 examples/linux_hawk_single_link_player.py
```
For AGX Thor or T5000 MGBE with FuSa CoE accelerated receiver:
```sh
python3 examples/fusa_coe_hawk_single_link_player.py
```
For two Hawk modules attached over both DA326 GMSL connectors, dual-link variants
are also available.
For ConnectX-accelerated platforms:
```sh
python3 examples/hawk_dual_link_player.py
```
For unaccelerated configurations:
```sh
python3 examples/linux_hawk_dual_link_player.py
```
For AGX Thor or IGX T5000 with FuSa CoE accelerated receiver:
```sh
python3 examples/fusa_coe_hawk_dual_link_player.py
```
````
````{tab-item} C++
The C++ examples need to be built first. From the repository root:
```sh
mkdir -p build && cd build
cmake ..
make -j
```
This leaves the resulting executables in `build/examples/`. From the `build/`
directory, you can then run the Hawk player variants.
For ConnectX-accelerated platforms:
```sh
./examples/hawk_single_link_player
```
For unaccelerated hosts:
```sh
./examples/linux_hawk_single_link_player
```
For AGX Thor or T5000 with FuSa CoE accelerated receiver:
```sh
./examples/fusa_coe_hawk_single_link_player
```
For two Hawk modules attached over both DA326 GMSL connectors, dual-link variants
are also available.
For ConnectX-accelerated platforms:
```sh
./examples/hawk_dual_link_player
```
For unaccelerated hosts:
```sh
./examples/linux_hawk_dual_link_player
```
For AGX Thor T5000 with FuSa CoE accelerated receiver:
```sh
./examples/fusa_coe_hawk_dual_link_player
```
````

Running the TAO PeopleNet example

The tao-peoplenet example demonstrates running inference on a live video feed. Tao PeopleNet provides a model that given an image can detect persons, bags, and faces. In this example, when those items are detected, bounding boxes are shown as an overlay over the live video.

Note that metadata passing has an open issue when used with InferenceOp and is explicitly disabled in the Application in these examples.

Prerequisite: Download the PeopleNet ONNX model from the NGC website:

1
wget --content-disposition 'https://api.ngc.nvidia.com/v2/models/org/nvidia/team/tao/peoplenet/pruned_quantized_decrypted_v2.3.3/files?redirect=true&path=resnet34_peoplenet_int8.onnx' -O examples/resnet34_peoplenet_int8.onnx

For systems with a ConnectX accelerated network controller interfaces with IMX274 camera,

1
$ python3 examples/tao_peoplenet.py

for unaccelerated configurations with IMX274 camera,

1
$ python3 examples/linux_tao_peoplenet.py

lastly, running SIPL accelerated network python example on AGX Thor with LI VB1940 camera:

1
$ python3 ./examples/sipl_tao_peoplenet.py --json-config ./examples/sipl_config/vb1940_single.json

This will bring up the Holoscan visualizer on the GUI showing the live video feed from the IMX274/Li VB1940 device as well as red/green box overlays when a person image is captured. Press Ctrl/C to exit. More information about this application can be found here.

Running the body pose example

Prerequisite: Download the YOLOv8 ONNX model from the YOLOv8 website and generate the body pose ONNX model. Within the Holoscan sensor bridge demo container:

NOTE: The stock AGX Orin Development Kits running Jetpack 7.2 may not have sufficient disk space to install dependencies. Prior to installing the python dependencies below, ensure you have plenty of disk space by clearing out unused packages or the docker builder cache

From the repo base directory holoscan-sensor-bridge:

1
apt-get update && apt-get install -y ffmpeg
2
pip3 install ultralytics onnx
3
cd examples
4
yolo export model=yolov8n-pose.pt format=onnx
5
trtexec --onnx=yolov8n-pose.onnx --saveEngine=yolov8n-pose.engine.fp32
6
cd -

Note that this conversion step only needs to be executed once; the yolov8n-pose.engine.fp32 file contains the converted model and is all that’s needed for the demo to run. The installed components will be forgotten when the container is exited; those do not need to be present in future runs of the demo.

For ConnectX-accelerated hosts, within the sensor bridge demo container, launch the Body Pose estimation with IMX274 camera:

1
$ python3 examples/body_pose_estimation.py

for unaccelerated hosts, launch the Body Pose estimation example within the demo container this way:

1
$ python3 examples/linux_body_pose_estimation.py

lastly, running SIPL accelerated network python example on AGX Thor with Li VB1940 camera:

1
$ python3 ./examples/sipl_body_pose_estimation.py --json-config ./examples/sipl_config/vb1940_single.json

This will bring up the Holoscan visualizer on the GUI showing the live video feed from the IMX274/Li VB1940 device, along with a green overlay showing keypoints found by the body pose net model. For more information about this application, look here.

Press Ctrl/C to exit.

Running the Stereo IMX274, Leopard imaging Li VB1940 Eagle, and TauroTech DA326 Hawk examples

The stereo_imx274_player.py example shows two independent pipelines, one for each camera on the dual-camera module. Make sure that both network ports are connected between the host and the Holoscan sensor bridge unit.

1
$ python3 examples/stereo_imx274_player.py

This brings up a visualizer display with two frames, one for the left channel and the other for the right.

For the purpose of aggregating lower bandwidth streams, you can observe the following examples aggregating both cameras to a single network port:

ConnectX-accelerated hosts with IMX274:

1
$ python3 examples/single_network_stereo_imx274_player.py

Unaccelerated hosts with IMX274:

1
$ python3 examples/linux_single_network_stereo_imx274_player.py

ConnectX-accelerated hosts with Li VB1940 Eagle:

1
$ python3 examples/single_network_stereo_vb1940_player.py

Unaccelerated hosts with Li VB1940 Eagle:

1
$ python3 examples/linux_single_network_stereo_vb1940_player.py

SIPL python example on AGX Thor or IGX T5000 with Li VB1940 camera:

1
$ python3 ./examples/sipl_player.py --json-config ./examples/sipl_config/vb1940_dual.json

When using the TauroTech DA326 sensor bridge and a Hawk camera module, both AR0234 sensors on the Hawk use a single GMSL link to the DA326’s on-board deserializer; the single_network_stereo_hawk_player examples aggregate them to the DA326’s SFP+ port:

ConnectX-accelerated host with TauroTech DA326 Hawk:

1
python3 examples/single_network_stereo_hawk_player.py --channel A|B

Unaccelerated host with TauroTech DA326 Hawk:

1
python3 examples/linux_single_network_stereo_hawk_player.py --channel A|B

AGX Thor with TauroTech DA326 Hawk using the FuSa CoE accelerated receiver:

1
python3 examples/fusa_coe_single_network_stereo_hawk_player.py --channel A|B

Applications wishing to map sensors to specific data channels can do so using the use_sensor API, which is demonstrated in these examples. Many hosts limit the network bandwidth to 10Gbps, so support is only provided for observing stereo video in 1080p mode.

TSN IMX274 player example

examples/linux_tsn_imx274_player.py extends the Linux IMX274 player with Time-Sensitive Networking (TSN) support. It configures 802.1Q VLAN tagging on the sensor virtual port and the EVT channel, and programs the FPGA PTP engine before starting the pipeline. This example works on all platforms that support the Linux Receiver (AGX Orin, AGX Thor, DGX Spark, and IGX).

Host network prerequisites

Before running the player, a VLAN subinterface must exist on the host NIC and the sensor bridge IP must be assigned to it so that the kernel delivers VLAN-tagged frames to the UDP socket:

1
# Create the VLAN subinterface (replace <iface> and <vlan-id> to match your setup)
2
sudo ip link add link <iface> name <iface>.<vlan-id> type vlan id <vlan-id>
3
sudo ip link set <iface>.<vlan-id> up
4
5
# Assign the sensor bridge IP to the subinterface
6
sudo ip addr add 192.168.0.101/32 dev <iface>.<vlan-id>

These commands are not persistent across reboots. Add them to a startup script or network manager configuration to make them permanent.

Host PTP service

The --ptp-profile argument controls which PTP profile the FPGA is programmed with. The host PTP daemon must match. Two systemd services are used — only one should be running at a time:

The default Host setup creates and starts ptp4l-<iface>.service (IEEE 1588 E2E). For gPTP, create the service once as follows (replace <iface> with your NIC name, e.g. enP5p3s0f0np0):

1
# Create the gPTP configuration file
2
cat <<EOF | sudo tee /etc/linuxptp/gptp.conf >/dev/null
3
[global]
4
gmCapable               1
5
priority1               100
6
priority2               100
7
domainNumber            0
8
logAnnounceInterval     0
9
logSyncInterval         -3
10
syncReceiptTimeout      3
11
assume_two_step         1
12
path_trace_enabled      1
13
follow_up_info          1
14
transportSpecific       0x1
15
ptp_dst_mac             01:80:C2:00:00:0E
16
network_transport       L2
17
delay_mechanism         P2P
18
EOF
19
20
# Create the systemd service unit
21
GPTP4L_SERVICE=/etc/systemd/system/gptp4l-<iface>.service
22
cat <<EOF | sudo tee $GPTP4L_SERVICE >/dev/null
23
[Unit]
24
Description=Send gPTP (802.1AS) on <iface>
25
After=phc2sys-<iface>.service
26
27
[Service]
28
Type=simple
29
ExecStart=/usr/sbin/ptp4l -i <iface> -f /etc/linuxptp/gptp.conf
30
31
[Install]
32
WantedBy=multi-user.target
33
EOF
34
35
sudo chmod u+x $GPTP4L_SERVICE
36
sudo systemctl daemon-reload
37
sudo systemctl enable gptp4l-<iface>.service

Switch to gPTP:

1
sudo systemctl stop ptp4l-<iface>.service
2
sudo systemctl start gptp4l-<iface>.service
3
journalctl -u gptp4l-<iface>.service -f   # wait for LOCKED

Switch to IEEE 1588 E2E:

1
sudo systemctl stop gptp4l-<iface>.service
2
sudo systemctl start ptp4l-<iface>.service
3
journalctl -u ptp4l-<iface>.service -f   # wait for grandmaster or LOCKED

Note that gPTP packets are not VLAN-tagged per the 802.1AS specification, so the gPTP service runs on the physical interface regardless of the VLAN configuration.

Running the example

1
python3 examples/linux_tsn_imx274_player.py

By default the player uses gPTP (IEEE 802.1AS, profile 1), VLAN ID 2, sensor PCP 6, and EVT PCP 4. All of these can be overridden on the command line:

1
# Use IEEE 1588 E2E profile instead of gPTP
2
python3 examples/linux_tsn_imx274_player.py --ptp-profile 0
3
4
# Override VLAN ID and PCP values
5
python3 examples/linux_tsn_imx274_player.py --vlan-id 10 --sensor-pcp 5 --evt-pcp 3
6
7
# Override the PTP domain number
8
python3 examples/linux_tsn_imx274_player.py --ptp-domain 1

Full option reference:

See the TSN IMX274 player application section for a detailed breakdown of the source code.

RoCE TSN IMX274 player example

examples/tsn_imx274_player.py is the RoCE variant of the TSN IMX274 player. It uses the same VLAN and PTP configuration but sends data over RoCE v2 instead of Linux sockets. Because the ConnectX NIC strips VLAN tags in hardware, no host VLAN subinterface is needed — the --vlan-id, --sensor-pcp, --evt-pcp, --ptp-profile, and --ptp-domain options work the same as in the Linux TSN player.

1
python3 examples/tsn_imx274_player.py

Running the GPIO example

examples/gpio_example_app.py is a simple example of using the GPIO interface of the sensor bridge to set GPIO directions, read input values from GPIO pins and write output values to GPIO pins. To run the application:

1
$ python3 examples/gpio_example_app.py

This brings up a textual display which cycles over different pre-set pin configurations and allows time between different settings of the pins to measure or readback pins values. Please refer to the application structure section to read more about the GPIO example application.

Using the Jetson Hardware ISP example

examples/linux_hwisp_player.py shows an example of the Jetson hardware ISP unit processing the Bayer frame captured live using IMX274. This example supports Jetson Orin AGX and IGX Orin 700 in iGPU configuration. This example does not support AGX Thor; refer to the SIPL examples for Hardware ISP usage.

Before starting the docker run, setup the nvargus-daemon with the flag enableRawReprocessing=1 and rawReprocessModulePartName="A6V26". This enables us to run the ISP with the Bayer frame capture from imx274 using Holoscan sensor bridge unit and this change persists through even restart. In the host system:

1
sudo su
2
pkill nvargus-daemon
3
export enableRawReprocessing=1
4
export rawReprocessModulePartName="A6V26"
5
nvargus-daemon
6
exit

To run the example, within the demo container:

1
$ python3 examples/linux_hwisp_player.py

This will run the application with visualizer display showing the live capture. Note that the default camera mode has changed since Jetpack 6 and now defaults to 4k frames. This example is compatible with Jetpack 7.2+.

Note if user wishes to undo running the nvargus-daemon with flag enableRawReprocessing=1, then please execute the following command.

1
sudo su
2
pkill nvargus-daemon
3
unset enableRawReprocessing
4
unset rawReprocessModulePartName
5
nvargus-daemon
6
exit

Running the Latency Application for IMX274 example

For IGX systems, examples/imx274_latency.py shows an example of how to use timestamp to profile hardware and software pipeline. This example demonstrates recording timestamps received from the FPGA when data is acquired and timestamps measured in the host at various points in frame reception and pipeline execution. At the end of the run, the application will provide a duration and latency report with average, minimum, and maximum values.

Before running the app, make sure the PTP sync has been enabled on the setup and then use the following commands to run the example.

1
$ python3 examples/imx274_latency.py

Running the latency example application on unaccelerated hosts:

1
$ python3 examples/linux_imx274_latency.py

This example will not run on AGX Thor

Running the ECam0M30ToF Player Application

The ecam0m30tof_player.py application demonstrates how to capture and display depth and/or IR data from the ECam0M30ToF time-of-flight camera using RoCE (RDMA over Converged Ethernet) for high-performance data transmission, utilizing Holoviz operator’s DEPTH_MAP rendering for enhanced depth visualization. This application can be modified to run on Jetson AGX by changing the receiver operator from RoceReceiverOp to LinuxReceiverOperator.

Prerequisites: ECam0M30ToF camera connected to the Hololink device

To run the application use following command.

$
python3 examples/ecam0m30tof_player.py --hololink 192.168.0.2 --camera-mode=<0|1|2>

Camera Configuration:

• --camera-mode: Select camera mode (0: DEPTH_IR, 1: DEPTH, 2: IR)

This example will not run on AGX Thor

Running CPU and GPU frame validation examples

Frame validation examples demonstrate how to access frame metadata in order to detect missing frames, frame timestamp misalignment and frame CRC errors. These examples record timestamps, frame numbers and CRC32 data received from the FPGA when data is acquired. During the run, missing frames, timestamp misalignment and CRC32 errors are detected and reported. At the end of the run, the application provides a duration and latency report with average, minimum, and maximum values. These values are collected during the application run to assess the impact of various detection mechanisms on the latency of the pipeline.

IMX274

Linux Receiver

For unaccelerated configurations, examples/linux_imx274_frame_validation.py uses standard Linux sockets for network communication with CPU-based CRC validation.

Before running the app, enable PTP sync on your setup, then use the following commands to run the example. Running the frame validation example on AGX Orin systems:

1
$ python3 examples/linux_imx274_frame_validation.py

Since the CRC32 calculation in this example is done by CPU, trying to detect CRC32 error using the example as is will trigger frame loss errors. For that reason CRC32 error detection is not enabled by default. To enable CRC32 detection every N frames use the --crc-frame-check option:

1
$ python3 examples/linux_imx274_frame_validation.py --crc-frame-check 50

In this example, the application will check for CRC32 frame errors every 50 frames.

RoCE Receiver

For ConnectX-accelerated platforms, examples/imx274_frame_validation.py provides high-performance frame validation with GPU-accelerated CRC checking using nvCOMP 5.0. This example uses the accelerated network receiver operator and requires ConnectX SmartNIC controllers.

Before running the app, enable PTP sync on your setup, then use the following command:

1
$ python3 examples/imx274_frame_validation.py

Unlike the CPU-based CRC validation in the Linux version, GPU-based CRC using nvCOMP 5.0 is fast enough to validate every frame by default. CRC validation is enabled by default with --crc-frame-check 1. To disable CRC validation entirely, use --crc-frame-check 0.

At the end of execution, the application provides a CRC validation report showing total frames processed, CRC errors detected, and success rate, followed by detailed performance metrics including frame time, transfer latency, operator latency, and processing time.

To validate the stereo camera configuration:

1
$ python3 examples/stereo_imx274_frame_validation.py

Performance

The nvCOMP CRC calculation performance on IGX-dGPU for single-camera configuration with 4K resolution (measured over 1000 frames):

Minimum: 275 us
Maximum: 390 us
Average: 295 us

Note on Startup Performance: Runtime performance of an HSDK pipeline at startup can be unpredictable, usually due to GPU kernel initialization. This is likely to lead to CRC failures: when the pipeline is slower than the camera frame rate, the receiver buffer can be overwritten with new data, which triggers the failure that CRC checking is looking for. Once the pipeline is fully initialized and can keep up with the received data, these errors would no longer be expected. We have primarily observed this issue in the stereo camera case. For this reason, our testing (see tests/test_module_imx274_pattern.py) skips CRC checking at the beginning, e.g. only after 15 frames have been received. User applications would likely use a similar startup state to avoid misleading errors occurring due to this known condition.

This example will not run on AGX Thor

Li VB1940 Eagle

FUSA CoE Receiver (nvCOMP)

For AGX Thor and T5000, examples/vb1940_fusa_nvcomp_crc_validation.py provides high-performance frame validation with GPU-accelerated CRC checking using nvCOMP 5.0. This example uses the FUSA CoE capture operator for accelerated network data transfer.

Before running the app, enable PTP sync on your setup, then use the following command:

1
$ python3 examples/vb1940_fusa_nvcomp_crc_validation.py

GPU-based CRC using nvCOMP 5.0 is fast enough to validate every frame by default. The application computes CRC on the full CSI frame (including CSI header and trailing bytes) to match the camera’s FPGA CRC computation.

At the end of execution, the application provides a CRC validation report showing total frames processed, CRC errors detected, and success rate, followed by detailed performance metrics including frame time, FUSA capture latency, operator latency, and processing time.

Performance

The nvCOMP CRC calculation performance on AGX Thor for Li VB1940 Eagle camera configuration (measured over 1000 frames):

Minimum: 554.4 us
Maximum: 631.0 us
Average: 614.3 us

Running PVA frame validation examples

PVA CRC validation examples demonstrate hardware-accelerated CRC computation using NVIDIA PVA (Programmable Vision Accelerator) to validate camera frames. These examples compare PVA-computed CRC values against camera FPGA-embedded CRC values to detect data corruption.

For requirements and build instructions, see the PVA CRC README.

To run the PVA CRC validation examples, first set the LD_LIBRARY_PATH environment variable (required before running any application). Within the demo container:

1
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/aarch64-linux-gnu/tegra/:/opt/nvidia/pva-sdk-2.9/lib/aarch64-linux-gnu/:/usr/lib/aarch64-linux-gnu/nvidia

To run the IMX274 PVA CRC validation example on ConnectX-accelerated systems:

1
$ python3 examples/imx274_pva_crc_validation.py --frame-limit 100

To run the Li VB1940 Eagle PVA CRC validation example on AGX Thor or IGX T5000:

1
$ python3 examples/vb1940_fusa_pva_crc_validation.py --frame-limit 100

Performance

IMX274

The PVA CRC calculation performance on IGX-dGPU for single-camera configuration with 4K resolution (measured over 1000 frames):

Minimum: 412 us
Maximum: 442 us
Average: 414 us

Li VB1940 Eagle

The PVA CRC calculation performance on AGX Thor for single-camera configuration with 1080p resolution (measured over 1000 frames):

Minimum: 106 us
Maximum: 137 us
Average: 111 us

Running the UART dual-board example

examples/uart_dual_board_loopback.py tests UART between two boards.

Hardware setup: Connect Board 1 GPIO 10 (TX) to Board 2 GPIO 11 (RX) and Board 2 GPIO 10 (TX) to Board 1 GPIO 11 (RX). See Lattice Bajoran Board GPIO Pin Locations to identify the pins.

Two modes are supported: one-way (tx/rx), with one board sending and the other receiving, and dual, where both boards send and receive.

One-way (tx/rx) — one board transmits, one receives:

1
# Terminal 1 (receiver):
2
$ python3 examples/uart_dual_board_loopback.py --mode rx --hololink 192.168.2.2
3
 
4
# Terminal 2 (transmitter):
5
$ python3 examples/uart_dual_board_loopback.py --mode tx --hololink 192.168.0.2

Dual mode — both boards transmit and receive:

1
# Terminal 1 (board 1):
2
$ python3 examples/uart_dual_board_loopback.py --mode dual --hololink 192.168.0.2 --test-string "HELLO" --expected-rx-string "WORLD"
3
 
4
# Terminal 2 (board 2):
5
$ python3 examples/uart_dual_board_loopback.py --mode dual --hololink 192.168.2.2 --test-string "WORLD" --expected-rx-string "HELLO"

Pass --mode (tx, rx, or dual), --hololink for each board’s IP, and optionally --test-string, --expected-rx-string, and --flow-control in dual mode. Run with --help for all options.

The UART FIFO size is 256 bytes. Both applications chunk data accordingly and verify transmitted vs. received data.

Running the I2S audio capture example

Both examples capture I2S audio from the board and play it through ALSA (optionally writing a WAV file). They exercise the I2S receive (RX) path; the board’s on-board digital microphone (DMIC) drives the I2S slots.

Setup: Run on the host using the demo launcher script, which starts the demo container with PulseAudio mounted so the decoded audio plays out on the host’s speakers or headphones.

RoCE receiver

examples/audio_recorder.py streams audio to a ConnectX-accelerated host.

1
# Live playback:
2
$ sh docker/demo.sh python3 examples/audio_recorder.py
3
4
# Record to a WAV file while playing, stopping after 2000 frames:
5
$ sh docker/demo.sh python3 examples/audio_recorder.py --output capture.wav --frame-limit 2000
6
7
# Capture-only (no playback):
8
$ sh docker/demo.sh python3 examples/audio_recorder.py --no-playback --output capture.wav

Pass --hololink for the board’s IP, --sensor to select the I2S sensor index (default 2 on hololink-lite; may differ on other boards), --output to write a little-endian stereo WAV, --frame-limit to stop after N frames (omit to run until interrupted with Ctrl-C), --no-playback for capture-only, and --alsa-device to select the output device. Run with --help for all options.

The audio is delivered as 48kHz stereo 32-bit PCM. Both I2S slots are forwarded as a stereo pair (slot 0 -> left, slot 1 -> right); when the DMIC drives only one slot, the other carries whatever is on the bus and appears in the corresponding channel.

FuSa CoE (MGBE) receiver

examples/audio_recorder_fusa.py streams audio to AGX Thor or IGX T5000.

1
# Live playback:
2
$ sh docker/demo.sh python3 examples/audio_recorder_fusa.py
3
4
# Record to a WAV file while playing, stopping after 2000 frames:
5
$ sh docker/demo.sh python3 examples/audio_recorder_fusa.py --output capture.wav --frame-limit 2000
6
7
# Capture-only (no playback):
8
$ sh docker/demo.sh python3 examples/audio_recorder_fusa.py --no-playback --output capture.wav

Pass --hololink for the board’s IP, --sensor to select the I2S sensor index (default 2 on hololink-lite; may differ on other boards), --output to write a little-endian stereo WAV, --frame-limit to stop after N frames (omit to run until interrupted with Ctrl-C), --no-playback for capture-only, and --alsa-device to select the output device. Run with --help for all options.

The audio is delivered as 48kHz stereo 32-bit PCM. Both I2S slots are forwarded as a stereo pair (slot 0 -> left, slot 1 -> right); when the DMIC drives only one slot, the other carries whatever is on the bus and appears in the corresponding channel.

Sub-Frame Processing Examples

Sub-frame processing allows high-resolution sensor data frames to be processed and displayed incrementally as they arrive, reducing latency and enabling progressive display. See the Sub-Frame Processing Applications section for detailed information about how sub-frame processing works.

IMX274 Sub-Frame Player Example

The sub-frame IMX274 player example demonstrates processing and displaying camera frames as sub-frames rather than complete frames. This enables lower latency visualization for high-resolution cameras.

1
  # With ConnectX accelerated network controller
2
  $ python3 examples/sub_frame_imx274_player.py --sub-frame-rows 540
3
   
4
  # For unaccelerated configurations (e.g. AGX Orin, AGX Thor)
5
  $ python3 examples/linux_sub_frame_imx274_player.py --sub-frame-rows 540

VB1940 Sub-Frame Visualizer Example

The sub-frame VB1940 player example demonstrates sub-frame processing with display-synchronized capture using the SubFrameVisualizerOp. Sub-frames are visualized incrementally as they arrive. Each sub-frame is composited into its position in the display frame without waiting for the full frame to complete. Capture timing is synchronized to the display refresh using the FPGA PTP/PPS output, minimizing end-to-end latency.

1
$ $BUILD_DIR/examples/sub_frame_vb1940_player --sub-frame-rows 248

The --sub-frame-rows parameter specifies the number of rows per sub-frame. The VB1940 default mode is 2560×1984 at 60fps.

NOTE: sub_frame_rows must evenly divide the frame height. For 1984 rows, valid values include 248 (8 sub-frames), 496 (4 sub-frames), and 992 (2 sub-frames).

Additional options:

• --fullscreen: Run in fullscreen mode (default: true)
• --exclusive-display: Use exclusive display mode for lower latency
• --camera-mode: Select VB1940 sensor mode (default: VB1940_MODE_2560X1984_60FPS)
• --use-sensor: Select sensor index when two sensors are connected (0 or 1, default: 0)
• --frame-limit: Exit after receiving this many frames (default: run indefinitely)

See SubFrameVisualizerOp in the applications guide for a detailed description of how display-synchronized capture works.

Sub-Frame Processing Modes

Sub-frame examples support two display modes:

• Sub-Frame Combiner Mode (IMX274 examples): Sub-frames are accumulated by SubFrameCombinerOp into a complete frame buffer before being passed to HolovizOp for display. Capture timing is independent of the display refresh rate.

• Sub-Frame Visualizer Mode (VB1940 example): Sub-frames are composited directly onto the display by SubFrameVisualizerOp as they arrive. Capture is synchronized to the display FPO (First Pixel Out) event via PTP/PPS, so each display refresh shows the latest available data with minimum latency.

Sub-Frame Configuration Parameters

• sub_frame_rows: Number of rows per sub-frame. Set to 0 to disable sub-frame processing (full-frame mode). Must evenly divide the frame height.

Performance Considerations

• Memory: Smaller sub-frames reduce memory requirements but increase processing overhead. The SubFrameCombinerOp waits for all sub-frames to arrive before emitting a complete frame, or emits an incomplete frame if a sub-frame is dropped and the next frame starts.

• Network: Sub-frame size should align with network packet sizes to minimize partial sub-frames and improve efficiency.