> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/holoscan/sensor-bridge/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/holoscan/sensor-bridge/_mcp/server. * [Overview](#overview) * [Features](#features) * [Environment Compatibility](#environment-compatibility) * [Building HSB Emulator](#building-hsb-emulator) * [Configuring the Emulator-only build](#configuring-the-emulator-only-build) * [Build Instructions](#build-instructions) * [Examples](#examples) * [Testing](#testing) * [Developing with HSB Emulator](#developing-with-hsb-emulator) * [Making your own HSB MCU](#making-your-own-hsb-mcu) * [Primary Classes](#primary-classes) * `HSBEmulator` * `DataPlane` * `BaseTransmitter` * `I2CPeripheral` * `Vb1940Emulator` * [Structure of an HSB Emulator application](#structure-of-an-hsb-emulator-application) * [Troubleshooting](#troubleshooting) * [Clean-Up](#clean-up) ## Overview The holoscan-sensor-bridge (HSB) repository includes a minimally featured emulation of the HSB framework's IP for use in testing and development within a hosted or embedded environment - the "HSB Emulator". Though it is called like a single object, it is a collection of objects that can be used to configure a real endpoint and data source for HSB host-side applications. This allows for simulation, testing, or development of a host application when the hardware is not fully developed, in testing, or a full software-in-loop/CI environment. An outline of the typical communication pipelines between and HSB Host application and an HSB Emulator application is shown in the diagram below. This graphic is but one particular topology and user client code may piece together the components so that an HSB Emulator application may meet the needs of their particular Host application(s)'s code. Note also that the two applications may exist either on different device, within the same device in different processes, or even within different threads of the same process (through transport options may be limited). The details of the specific objects in the diagram are described in [Developing with HSB Emulator](#developing-with-hsb-emulator). These objects in the header files in `src/hololink/emulation` comprise the HSB hardware abstraction layer ("HAL") API that each target HSB Emulator environment is minimally expected to implement. Communication Pipelines between HSB Host and Emulator Applications The HSB Emulator as provided is *not* meant to be a full implementation of the FPGA HSB device nor provide the full performance benefits of those implementations. Many features of the HSB IP are not fully implemented in that the existing emulator will accept and reply to requests, e.g. ptp, but may be "no-ops" internally so that apps that currently depend on that interface being present can proceed but leave the implementation detail to applications or expected behavior. The features that are implemented are only the ones necessary for a host application to configure and accept sensor data through the HSB transport pipelines. ### Features The features that are supported in provided code are 1. Sensor agnostic * HSB Emulator itself has no built-in knowledge of any given sensor. * Sensors and drivers that do not depend on state change queries, e.g. the HSB example "stateless" imx drivers, are compatible by default 2. `RoCEv2Transmitter` for RoCEv2 UDP-based transport * Name is to be consistent with the rest of HSB where the implementation is via Linux interface access and sockets 3. `COETransmitter` for IEEE 1722B link layer transport (Camera-over-Ethernet, CoE) 4. `BaseTransmitter` and `DataPlane` interfaces for adding experimental transport protocols 5. Underlying C++ implementation + Python API 6. CUDA buffer support * `DLPack` array interface format - compatible with contiguous arrays from any Python package that supports `DLPack` * sensor data buffers may be provided on host or CUDA GPU device 7. Configuration of "stateful" sensors via an `I2CPeripheral` interface * SPI and GPIO are not fully implemented 8. Isolated build from host * HSB Emulator may be built without linking to either HSB or Holoscan SDK to minimize environmental and package dependencies 9. Loopback testing ability * The HSB Emulator can be used to test both accelerated and unaccelerated transport on the same device the host is running on though for accelerated transport, e.g. `RoceReceiverOp` and `SIPLCaptureService / SIPLCameraOutputOp`, a physical connection and network isolation would be needed ### Limitations The HSB Emulator is meant to be as hardware agnostic as possible and may run on the same device as host applications where it cannot change many settings. To that end, it is not fully emulating any particular FPGA or MCU implementation of HSB IP and therefore some features are not or will not be implemented. 1. Only one instance of HSBEmulator is likely to work on any single device: * This is a current limitation in the APIs to be consistent with real HSB IP. While only one HSBEmulator may run, the number of sensor `DataPlane`s that may attach to the instance is limited only by the number of IP addresses that can be assigned and configurable sensor interfaces (SIFs) for the emulated device (\<= 32). * Running in isolated network namespaces will free up this restriction for connections other than a SW loopback, but requires application & system maintenance 2. The HSB Emulator may not be reprogrammed via the host side IP tools like a real HSB: * set\_ip * p2p * reset * any reprogramming ### Environment Compatibility In Linux target environments, Python and C++ APIs are provided. They can be extended to other operating systems or environments (e.g. 32-bit) by creating a new target in the `src/hololink/emulation/src` directory, implementing the HSB "HAL" layer represented by the headers within the `src/hololink/emulation` directory, and creating a suitable, target-specific cmake file in `src/hololink/emulation/cmake/targets/` to configure the build system for that target. Other than OS facilities, requirements have been kept to a minimum and are described in more detail in the [Building HSB Emulator](#building-hsb-emulator) section. HSB Emulator has been tested with host applications in the following environments: 1. x86 Linux Desktop - Ubuntu 22.04+ - Emulator applications only 2. Jetson Orin Nano Developer Kit - Jetpack 6.1+ - Emulator applications only 3. AGX Orin Developer Kit - Jetpack 6.1+ - Host or Emulator applications 4. IGX Orin Developer Kit - IGX BaseOS 1.0+ - Host or Emulator applications 5. AGX Thor Developer Kit - Jetpack 7.1+ - Host or Emulator applications As an example of extending the environment compatibility, an implementation of the HSB IP through the HSBEmulator HAL APIs is provided for STM's STM32f767zi Cortex-M7 processor that is suitable for deployment or development on the [NUCLEO-STM32f767zi](https://www.st.com/en/evaluation-tools/nucleo-f767zi.html) development board. This implementation currently includes only the control plane communication, but has an extensible callback feature where additional "registers" from control plane commands can invoke custom callbacks to be defined by the user. To demonstrate the cross-platform API, a simple example in `examples/hsb_control` is provided that may be compiled for both linux and STM32F767ZI targets without any changes and respond to host control plane commands. For a Linux target, the [Examples](#examples) provided are tested mostly with imx274- (where applicable) and the vb1940-based HSB camera applications. The `serve_linux_file` and `serve_coe_file` have been used with the HSB-provided imx477 and imx715 sensor drivers as well. ## Building HSB Emulator There are 2 ways to build the HSB Emulator * Full * This is a normal build of the HSB repository that will build the emulator and include it within the normal hololink Python module * Emulator-only * Targets a linux environment or MCU (currently only STM32f767zi Cortex-M7 devices, but extensible) * Minimal build dependencies. An optional hololink Python module is provided within a virtual environment for the Python API on linux targets ### Build instructions For all build types, make sure the CUDA compiler and libraries are on your `PATH` ```shell export PATH="$PATH:/usr/local/cuda/bin" export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64" ``` clone and enter the repository ``` git clone https://github.com/nvidia-holoscan/holoscan-sensor-bridge.git cd holoscan-sensor-bridge ``` #### Full Follow the appropriate [HostSetupTarget](/holoscan/sensor-bridge/getting-started/host-setup#HostSetupTarget) instructions first. ``` mkdir build cmake -S . -B build cmake --build build -j ``` #### Emulator-only for Linux Install build dependencies. It's assumed an appropriate [CUDA toolkit](https://developer.nvidia.com/cuda-downloads) has been installed for the target system. Follow platform instructions to ensure version, gpu, and driver compatibility: ``` sudo apt-get update sudo apt-get install -y cmake zlib1g-dev ``` If building the Python bindings as well ``` sudo apt-get install -y python3-pybind11 python3-venv python3-pip ``` Additional dependencies for the Python bindings are required for the provided examples (`numpy` v1.23+ and `cupy`), but the virtual environment that is created will automatically populate them. The versions required are not necessarily compatible with the system Python, especially on Ubuntu 22.04 and comparable or earlier ``` mkdir build cmake -S src/hololink/emulation -B build cmake --build build -j ``` #### Emulator-only for STM32F767ZI Install build dependencies. It's assumed an appropriate [STM32CubeF7 MCU FW Package](https://github.com/STMicroelectronics/STM32CubeF7) has been cloned locally for the target board, e.g. ``` git clone --depth 1 --recursive https://github.com/STMicroelectronics/STM32CubeF7.git ~/STM32/STM32CubeF7 ``` Additionally, the gcc arm cross compiling toolchaing must be installed: ``` sudo apt-get install gcc-arm-none-eabi ``` To build: ``` mkdir build cmake -S src/hololink/emulation -B build -DHSB_EMULATOR_TARGET=STM32F767ZI -DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/arm-none-eabi-gcc.cmake cmake --build build -j ``` Use your normal MCU programming tools compatible with the target platform (STM32). For example, using the STM32\_Programmer\_CLI, ``` STM32_Programmer_CLI --connect port=swd reset=HWrst -w 0x08000000 --go ``` #### Emulator-only for TEMPLATE This TEMPLATE target is meant for developing a custom HSB implementation for a real hardware target. The provided implementation is intentionally minimal: every required symbol is stubbed so the tree compiles with the gcc arm cross compiling toolchain, and a target board linker script + peripheral modules (i2c, spi, gpio, etc.) are left for you to fill in. First install the toolchain: ``` sudo apt-get install gcc-arm-none-eabi ``` To build: ``` mkdir build cmake -S src/hololink/emulation -B build -DHSB_EMULATOR_TARGET=TEMPLATE -DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/arm-none-eabi-gcc.cmake cmake --build build -j ``` An `.elf` will be produced but it does nothing useful yet. See [Making your own HSB MCU](#making-your-own-hsb-mcu) for the per-function quick-start. ### Configuring the Emulator-only build Several `cmake` flags are available to configure the "Emulator-only for Linux" build if prerequisites are not met or not in expected locations | dependency | cmake configuration | default setting | availability | | ----------------------------------------------------------------------------- | -------------------------------------------------------- | --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | Python unavailable/not needed | `-DHSB_EMULATOR_BUILD_PYTHON=OFF` | `ON` | Emulator-only for Linux | | Python virtual environment location
(ignored if Python build is `OFF`) | `-DHSB_EMULATOR_PYTHON_VENV=path/to/virtual_environment` | \`\`\$\{CMAKE\_BINARY\_DIR}`/env` | Emulator-only for Linux | | non Linux targets | `-DHSB_EMULATOR_TARGET=target_name` | `linux` | `target_name` must have a configuration in `src/hololink/emulation/cmake/targets/*` and acceptable targets in `src/hololink/emulation/CMakeLists.txt` | | STM32CubeF7 not in `~/Documents/STM32` | `-DSTM32_PATH=/path/to/STM32CubeF7/repo` | `~/STM32` | Emulator-only for STM32 targets | | zlib source available locally | \`-DZLIB\_PATH=/path/to/zlib | downloads required files | Emulator-only for STM32 targets | Note for cmake version \< 3.24 (`apt` default available in Ubuntu 22.04 or earlier), cmake does not allow setting a native architecture and a default `CMAKE_CUDA_ARCHITECTURES` of 80 86 87 compute capability is given to ensure target Orin platform compatibility. For cmake >= 3.24, native is selected by default. ## Examples To facilitate testing and development of HSB Emulator applications, several examples applications in both C++ and Python are provided to demonstrate different possible configurations and implementations. The source code for all the examples are located in `src/hololink/emulation/examples`. The C++ applications by default will be built to `/examples/`. For all examples, `--help` will provide invocation and flag details. All current transport capabilities and therefore the examples will require running as `root` or under `sudo`. When building as "Emulator-only" with Python bindings, the Python examples should be invoked with the Python interpreter in the provided virtual environment. For example, to run the `serve_coe_stereo_vb1940_frames` example in Python, the full invocation would be ``` sudo build/env/bin/python3 src/hololink/emulation/examples/serve_coe_stereo_vb1940_frames.py 192.168.0.2 ``` | example | sample invocation | brief description | HSB sample camera
driver compatibility | Compatible HSB receivers | Notes | | ---------------------------------- | -------------------------------------------------------------- | --------------------------------------------------------- | --------------------------------------------- | ------------------------------------------------------------------------------ | --------------------------------- | | `hsb_control` | `./hsb_control` | send and receive basic control plane commands to/from HSB | None | None | available on all target platforms | | `gpio_send_roce` | N/A | send board-specific gpio pin status over RoCEv2 from MCU | None | `LinuxReceiver`, `RoceReceiver` | STM32 targets only | | `gpio_trigger_send_roce` | N/A | send timestamp triggered by gpio pin over RoCEv2 from MCU | None | `LinuxReceiver`, `RoceReceiver` | STM32 targets only | | `serve_linux_file` | `./serve_linux_file 192.168.0.2 imx_single_raw_frame.dat` | serve frames from a file | imx cameras | `LinuxReceiver`, `RoceReceiver` | | | `serve_coe_file` | `./serve_coe_file 192.168.0.2 imx_single_raw_frame.dat` | serve frames from a file | imx cameras | `LinuxCoeReceiver` | | | `serve_coe_vb1940_file` | `./serve_coe_vb1940_file 192.168.0.2 imx_single_raw_frame.dat` | serve frames from a file | VB1940 | `LinuxCoeReceiver`, `SIPLCaptureService / SIPLCameraOutputOp`, `FusaCaptureOp` | Compatible with AGX Thor platform | | `serve_linux_single_vb1940_frames` | `./serve_linux_single_vb1940_frames 192.168.0.2` | serve test RGB frames from single VB1940 camera | VB1940 | `LinuxReceiver`, `RoceReceiver` | | | `serve_linux_stereo_vb1940_frames` | `./serve_linux_stereo_vb1940_frames 192.168.0.2` | serve test RGB frames from stereo VB1940 camera | VB1940 | `LinuxReceiver`, `RoceReceiver` | | | `serve_coe_single_vb1940_frames` | `./serve_coe_single_vb1940_frames 192.168.0.2` | serve test RGB frames from single VB1940 camera | VB1940 | `LinuxCoeReceiver`, `SIPLCaptureService / SIPLCameraOutputOp`, `FusaCaptureOp` | Compatible with AGX Thor platform | | `serve_coe_stereo_vb1940_frames` | `./serve_coe_stereo_vb1940_frames 192.168.0.2` | serve test RGB frames from stereo VB1940 camera | VB1940 | `LinuxCoeReceiver`, `SIPLCaptureService / SIPLCameraOutputOp`, `FusaCaptureOp` | Compatible with AGX Thor platform | For the `*_file` examples, it is crucial that image frame sizes are set up appropriately, especially on Jetson AGX Thor platform. You will need to know the correct CSI image frame size that needs to be sent. These are based on Host application driver/operator requirements which are in turn dependent on the camera and HSB hardware. As examples, * For imx274 capture in mode 1 (1920X1080 RAW10 - 4 pixels/5 bytes, 175 header bytes, 8 lines of optical black, 8-byte line alignment and 8-byte overall alignment) the CSI frame size is (175 + (((1920 + 3) / 4 \* 5 + 7) / 8 \* 8) \* (1080 + 8) + 7) / 8 \* 8 = 2611376 bytes, so the input file should be at least 2611376 bytes in size and each slice of that size will be transmitted as one frame from 192.168.0.2 to an `*imx274_player` application with the following call: * `serve_linux_file -s 2611376 192.168.0.2 my_file.dat` * For vb1940 capture in mode 0 (2560X1984 RAW10 - 4 pixels/5 bytes, 1 line of prefix status, 2 lines of postfix status, 8-byte line alignment and 8-byte overall alignment) the CSI frame size is ((((2560 + 3) / 4 \* 5 + 7) / 8 \* 8) \* (1984 + 3) + 7) / 8 \* 8 = 6458400 bytes, but on Jetson AGX Thor, it additionally goes through a swizzle/packetizer step to a raw10 format (3 pixels/4 bytes, 64-byte line alignment) such that the CSI frame size is effectively ((((2560 + 2) / 3 \* 4 + 63) / 64 \* 64) \* (1984 + 3) + 7) / 8 \* 8 = 6867072 bytes. Therefore on Jetson AGX Thor, a similar call would be: * `serve_coe_vb1940_file -s 6867072 192.168.0.2 my_file.dat` For further details on other modes see the provided drivers in `python/hololink/sensors/`. By default the `*file` examples will assume the whole file is one image frame. To set the image frame size, the `-s` flag is available in each of the examples. When the file size is greater than the value supplied to the `-s` option, it will assume multiple frames are in the file and cycle through them, advancing by `-s` bytes after each image frame and reset to the start of the file when a full frame can no longer be sent. For example, if `X` is provided to the `-s` option, `-s X`, and the file size is `N * X + Y` bytes where `N > 0`, and `0 <= Y < X`, the examples will send `N` image frames, reset to the beginning of the file, and then continue. ## Testing Full-loop testing of the HSB Emulator (on linux platforms) is provided within the pytest framework in the docker container. To run the test, launch the docker container and run: ``` pytest ``` the same as described in [Run tests in the demo container](/holoscan/sensor-bridge/getting-started/build#RunningTestsTarget). The default pytest configuration will test vb1940 camera modes with linux sockets in roce and coe packet formats for gpu or cpu inputs as well as linux roce with imx477 camera modes. To run an extended sweep of parameters including also imx715 and imx274 camera mode, run: ``` pytest --emulator ``` to get the more comprehensive testing parameters. ### Hardware Loopback The tests above for vb1940 are all executed by default over software loopback. To run them over-the-wire using available network interfaces, they may be run in hardware loopback mode by attaching an Ethernet cable between 2 Ethernet-compatible ports on the target test device. Note the interfaces `IF1` and `IF2` (from e.g. `ip addr sh`) that correspond to the attached ports. Then run the tests with the following option: ``` pytest --hw-loopback IF1 IF2 ``` This will run the same tests for the HSB Emulator, but isolate `IF1` from `IF2` and launch the relevant the emulator process on `IF1` and the test host application process targeting `IF2`. The scripts `nsisolate.sh`, `nsexec.sh`, `nspid.sh`, and `nsjoin.sh` in the `scripts/` directory are provided to facilitate development of additional hardware loopback tests. The `--hw-loopback IF1 IF2` switch in `pytest` is roughly equivalent to: ``` # setup environment scripts/nsisolate.sh IF1 192.168.0.2/24 # for each test scripts/nsexec.sh IF1 # teardown environment scripts/nsjoin.sh IF1 ``` The `nspid.sh` script facilitates finding the relevant `pid` with which to control a process within the network namespace created by `nsisolate.sh` since you cannot simply control/kill the `nsexec.sh` command without creating an orphaned process. ### Accelerated Networking on Hardware Loopback On platforms that offer accelerated networking such as Jetson AGX Thor (SIPL CoE over MGBE), IGX Orin (RoCEv2 over CX-7), and DGX Spark (RoCEv2 over CX-7), the network namespace scripts above also provide a simple way to test accelerated networking HSB applications. Under normal conditions, the Linux operating systems used on those target devices will route connections between two interfaces through an internal software/loopback interface. Isolating one of the connected interfaces in a network namespace will prevent the software loopback rerouting and force use of the accelerating hardware. The HSB Emulator tests for RoCE will already handle this and on relevant platforms will automatically configure a network namespace for testing when provided with the `--hw-loopback` parameters. On AGX Thor with SIPL, the MGBE interface needs to be managed independently but the tests will still run over the isolated interface if provided. As examples: ``` # interface names may vary pytest --hw-loopback enP5p3s0f1np1 enP5p3s0f0np0 ``` will run the linux, coe transport examples over standard linux sockets and also run the RoCE examples for vb1940 through the CX-7 ports. ``` scripts/nsisolate.sh enP2p1s0 192.168.0.2/24 # this next line run outside the container if mgbe0_0 is "DOWN" after isolating sudo ip link set mgbe0_0 down && sudo ip link set mgbe0_0 up # back in the container pytest --hw-loopback enP2p1s0 mgbe0_0 --json-config my_single_vb1940_sensor_config.json scripts/nsjoin.sh enP2p1s0 ``` will run the linux, coe transport examples over standard linux sockets and also run the SIPL example for vb1940 through the MGBE port with CoE offloading. ## Developing with HSB Emulator ### Making your own HSB MCU The `TEMPLATE` target is the starting point for porting HSB to a new MCU. It builds, but every function is a stub — your job is to plug in the platform-specific logic. 1. **Source layout** — three files under `src/hololink/emulation/src/Template/` provide everything you need to extend: * `HSBTemplate.cpp` — control-plane handling, BootP broadcasting, basic networking. * `HSBTemplate_rocev2.cpp` — RoCEv2 transmit path (ConnectX-compatible). * `HSBTemplate_coe.cpp` — IEEE 1722B Camera-over-Ethernet transmit path (SIPL / FuSa on AGX Thor and IGX Thor Mini with mGbE). 2. **Find every `TODO`** in the three files above and fill it in. The functions that actually need platform work are grouped by concern: *Networking / control plane* * `IPAddress_from_string` — populate the `IPAddress` struct (MAC, `if_name`, subnet mask, broadcast) from dotted-quad text. * `control_plane_reply` — put a pre-built scatter/gather ECB reply on the wire. * `HSBEmulator::handle_msgs` — receive and dispatch ECB packets to `handle_control_packet`; optionally also drive other required networking tasks (ICMP / ARP / BootP) if single-threaded and no networking middleware (see STM32's implementation). * `HSBEmulator::add_data_plane` — track a DataPlane so it gets `start()` / `stop()` lifecycle calls. * `HSBEmulator::HSBEmulator` — register per-control plane register callbacks with `HSBEmulator::register_read_callback` and `HSBEmulator::register_write_callback` *Transmit paths* * `send_coe_packet` — emit a 1722B frame from a scatter/gather array. * `send_rocev2_packet` — emit a RoCEv2 frame from a scatter/gather array. *DataPlane BootP and task timing* * `clock_gettime` — used to stamp BootP `secs` and frame metadata timestamps. * `DataPlane::broadcast_bootp` — put one BootP packet on the wire. Wrap in a thread / RTOS task if your runtime is multi-threaded. * `DataPlane::DataPlane` — register per-DataPlane/Sensor register callbacks with `HSBEmulator::register_read_callback` and `HSBEmulator::register_write_callback` *Main-loop timing helpers (only if you run the bundled `serve_*_msg` examples)* * `msleep(unsigned)` — millisecond sleep used by `serve_coe_msg` / `serve_roce_msg` to throttle to `FRAME_RATE`. 3. **Functions you can usually leave alone.** Anything that exists purely to coordinate concurrency (mutexes, condition variables, atomic flags) can stay as the empty stub when your runtime is single-threaded. Threading hooks are there to *accommodate* a multi-threaded port, not to require one. 4. **Carrying your own state.** Each control-plane / data-plane object holds a `*Ctxt` struct (e.g. `HSBEmulatorCtxt`, `DataPlaneCtxt`, `COECtxt`, `RoCEv2Ctxt`). Extend the appropriate one with whatever per-instance fields your platform needs (DMA descriptors, HAL handles, ring buffers). Lifetime is yours to manage — the framework only holds a `Ctxt*`. 5. **implement your own target** and customize the target build. * Copy your new implementation files out/rename the folder and the corresponding target `.cmake` file in `cmake/targets/`. * Add the new target name to the `HSB_EMULATOR_EMBEDDED_TARGETS` list in the `src/hololink/emulation/CMakeLists.txt` file to enable a new target alongside `STM32F767ZI` and `TEMPLATE`. ### Primary Classes For all the classes below, the C++ namespace is assumed `hololink::emulation` and the Python module is loaded as `import hololink.emulation as hemu`. Python-equivalent examples or definitions are given for clarity for non-trivial signatures. For further details including private and protected members of classes for development purposes, see the source code in `src/hololink/emulation`. For detailed API documentation of the HSB Emulator interface, see the [Emulation API reference](/holoscan/sensor-bridge/api-reference/emulation-api). The HSB Emulator class is used for command and control communication between the Emulator device and the Host HSB Application `HSBEmulator` \<--> `Hololink` in host HSB application Callback type used by `register_read_callback` and `register_write_callback`: `I2CController` is a component in `HSBEmulator` that is only needed to develop a sensor emulator/driver bridge for I2C sensors. `DataPlane`s are the object with which client applications of the HSB Emulator send data to the Host application and how receivers identify unique sensor sources `DataPlane` --> appropriate subclass of BaseReceiverOp in host HSB Application The `IPAddress` and the utility function `IPAddress_from_string` are provided to encapsulate and simplify construction of the network interface properties required for the `DataPlane` object in the hosted environment Two implementations of the `DataPlane` interface provided are `RoCEv2DataPlane` and `COEDataPlane`. The `COEDataPlane` is currently only available on linux target platforms. RoCEv2 UDP-based transport implementation. Transmitters represent the actual transport implementation that is used by the DataPlane to send the data. For development of new/modified transport only. HSBEmulator applications currently do not interact directly with these objects Two implementations of the `BaseTransmitter` interface provided are `RoCEv2Transmitter` and `COETransmitter` For development of new sensor bridge drivers (bridge HSBEmulator to a real device) or emulating a sensor. This will only be available on linux target platforms. The primary example of implementing an I2CPeripheral for an HSB Emulator application is the STM VB1940 sensor for emulating the [Leopard Eagle HSB from Leopard Imaging](https://leopardimaging.com/sensor-bridge/). ### Structure of an HSB Emulator application The [Examples](#examples) in the next section and the `main` functions in their source code show typical workflows for starting up an HSBEmulator instance, but the general outline in C++ and Python is given below for the case of RoCEv2 transmission over Linux sockets. #### C++ ```cpp // including the appropriate header for the implementation of the // DataPlane is sufficient for a minimal application #include "hololink/emulation/data_plane.hpp"   // OPTIONAL: import the target sensor emulator #include "hololink/emulation/sensors/vb1940_emulator.hpp"   // Declare a target HSBEmulator instance. In this case, the example is explicitly // emulating a Leopard Eagle HSB configuration (needed for VB1940). It is not yet // ready to receive commands from a host application until a subsequent call to its // `start()` method. // NOTE: If multiple instances of HSBEmulator exist on a single device // (even across // processes), they can receive and emit independent ECB packets, which // may conflict with the host application. HSBEmulator hsb(HSB_LEOPARD_EAGLE_CONFIG);   // Create an implementation of the DataPlane interface which will be used to send data // as if from a *single* sensor. For this, you need to set a sensor_id to identify the // sensor interface the channel will emulate and IPAddress + data_plane_id that identify // NOTE: multiple DataPlanes with IPAddress or data_plane_id will result in multiple // BOOTP broadcast message sources // NOTE: if multiple DataPlanes have the same (data_plane_id, sensor_id) combination, // their configurations may override each other uint8_t data_plane_id = 0; uint8_t sensor_id = 0; RoCEv2DataPlane data_plane(hsb, IPAddress_from_string(ip_address), data_plane_id, sensor_id);   // Optional for stateful sensor emulation or driver bridge declare the sensor instance sensors::Vb1940Emulator vb1940; // Attach it to an appropriate I2CController and bus. On Leopard Eagle HSB, the i2c bus // address is the sensor_id offset from CAM_I2C_BUS vb1940.attach_to_i2c(hsb.get_i2c(hololink::I2C_CTRL), hololink::CAM_I2C_BUS + sensor_id);   // Start the emulator. After this, the HSBEmulator instance can receive and respond // to control commands from the host application. All DataPlane instances which have // been declared will start emitting BOOTP broadcast messages. All I2CPeripherals // will receive a call to their start() methods hsb.start();   // client code in a loop. Passing the DataPlane implementation gives the loop access // to send data to the host application and sending the sensor instance (if needed) // provides access to sensor-specific/state data application_specific_data_loop(RoCEv2DataPlane, vb1940, ...user configuration data)   // OPTIONAL: Stop the emulator if the application needed to be able to restart the // HSBEmulator instance without exiting. All registered elements will receive a call // to their `stop()` methods for cleanup. The emulator and all registered elements // can be re-started with a subsequent call to // `start()`. hsb.stop(); ``` #### Python ```python # importing the appropriate module for the implementation of the DataPlane is sufficient for # a minimal application import hololink.emulation as hemu   # import the hololink module for access to some HSB constants. # NOTE: this is very minimal in the "Emulator-only" build and does not include any of the # other hololink libraries import hololink as hololink_module   # Declare a target HSBEmulator instance. In this case, the example is explicitly emulating a # Leopard Eagle HSB configuration (needed for VB1940). It is not yet ready to receive # commands from a host application until a subsequent call to # its `start()` method. # NOTE: If multiple instances of HSBEmulator exist on a single device (even across # processes), they can receive and emit independent ECB packets, which may conflict with the # host application. hsb = hemu.HSBEmulator(hemu.HSB_LEOPARD_EAGLE_CONFIG)   # Create an implementation of the DataPlane interface which will be used to send data as if # from a *single* sensor. For this, you need to set a sensor_id to identify the sensor # interface the channel will emulate and IPAddress + data_plane_id that identify # NOTE: multiple DataPlanes with IPAddress or data_plane_id will result in multiple BOOTP # broadcast message sources # NOTE: if multiple DataPlanes have the same (data_plane_id, sensor_id) combination, their # configurations may override each other data_plane_id = 0 sensor_id = 0 data_plane = hemu.RoCEv2DataPlane( hsb, hemu.IPAddress(args.ip_address), data_plane_id, sensor_id)   # Optional for stateful sensor emulation or driver bridge declare the sensor instance vb1940 = hemu.sensors.Vb1940Emulator() # Attach it to an appropriate I2CController and bus. On Leopard Eagle HSB, the i2c bus # address is the sensor_id offset from CAM_I2C_BUS vb1940.attach_to_i2c( hsb.get_i2c(hololink_module.I2C_CTRL), hololink_module.CAM_I2C_BUS + sensor_id )   # Start the emulator. # After this, the HSBEmulator instance can receive and respond to control commands from the # host application. All DataPlane instances which have been declared will start emitting # BOOTP broadcast messages. All I2CPeripherals will receive a call to their start() methods hsb.start()   # sensor client code in a loop. Passing the DataPlane implementation gives the loop access to # send data to the host application # and sending the sensor instance (if needed) provides access to sensor-specific/state data application_specific_data_loop(RoCEv2DataPlane, vb1940, ...user configuration data)   # OPTIONAL: Stop the emulator if the application needed to be able to restart the HSBEmulator # instance without exiting. All registered elements will receive a call to their `stop()` # methods for cleanup. The emulator and all registered elements can be re-started with a # subsequent call to `start()`. hsb.stop() ``` For stereo/dual sensor setups, see the `*stereo_vb1940_frames` examples. The specific loops used to stream the files or test frames across all the examples can be found in the `emulator_utils` files. ## Troubleshooting | Problem Description | Root Cause | Corrective Action (s) | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | "Permission Denied"/"Operation not permitted" in socket creation or bind | UDP/port combination is protected by [capabilities](https://man7.org/linux/man-pages/man7/capabilities.7.html) privilege | - Run under `sudo`
- Check firewall status and if necessary ensure the ports are open for UDP.
Example (Ubuntu 24.04 UDP to/from IGX on port 192.168.0.101):
`sudo ufw status # check for UDP access, specifically on port 8192`
`sudo ufw allow from 192.168.0.101 to any port 8192`
- use `sudo setcap cap_net_raw+ep /path/to/executable` to remove raw socket restrictions. WARNING: allowing this on the Python interpreter has severe security implications | | white streaks/lost packets in frame visible on receiver end | on the receiver host device, make sure your have sufficient network receiver buffer size. Common with LinuxReceiverOp and LinuxCoeReceiverOp, esp on AGX devices. | `sudo sysctl -w net.core.rmem_max=16777216 && sudo sysctl -w net.core.rmem_default=16777216` | | running with Python code:
AttributeError: 'numpy.ndarray' object has no attribute '\_\_dlpack\_\_'
or similar with data types other than 'numpy.ndarray' | data type (in this case numpy array) does not support [Python array API](https://data-apis.org/array-api/latest/design_topics/data_interchange.html) DLPack data interchange format | - change to a data type that supports DLPack
- upgrade environment to version of package that does support DLPack (numpy requires 1.23+) | | frequent I2C failures with emulator sending to Thor host device | failed packet receipt on Thor mGbE interface with no retries in JP 7.0 with SIPL rel 38.2.1 | upgrade to JP 7.1 EA rel 38.4 | ## Clean-up `rm -rf build`