Hololink Module, Application Tutorial

Overview

Follow this tutorial to build a player for an IMX274 camera and see how a hololink_module application discovers, configures, and streams from a Holoscan Sensor Bridge board. Applications using the new hololink_module can talk to different HSB devices transparently, unlike those applications using the legacy API.

An application accesses the capabilities it needs as services instead of constructing device classes directly. A service is requested by type through get_service, usually keyed by the enumeration metadata that identifies a specific board:

1
  hololink = hololink_module.HololinkInterfaceV1.get_service(metadata)

This indirection is what lets one application drive different boards — and different revisions of the same board — without recompiling. The Background section explains how; the rest of this page builds a working player first.

IMX274 player tutorial

This tutorial builds a player that receives video from an IMX274 camera and displays it. The complete program ships as imx274_module_tutorial.py (Python) and imx274_module_tutorial (C++); every snippet below is taken from it. Pick your language with the tabs.

Configuration

You need a Holoscan Sensor Bridge board with an IMX274 camera attached, reachable at its default address 192.168.0.2. The IMX274 is a stereo camera pair; at 192.168.0.2 this tutorial drives the first camera on the device. This tutorial assumes you have built and are running the demo container as described in Host Setup and Build. The program hard-codes its settings, so there is nothing to pass on the command line.

Application pipeline

The player is a Holoscan pipeline of five operators. Data flows left to right:

receiver -> csi_to_bayer -> image_processor -> demosaic -> holoviz

While data flow starts with the receiver operator, we can’t construct it without knowing some configuration (specifically the receiver buffer size) that comes from csi_to_bayer. So build CsiToBayerOp, configure it against the camera, and read back the frame size:

1
  csi_op = hololink_module.operators.CsiToBayerOp(
2
      self,
3
      name="csi_to_bayer",
4
      allocator=csi_to_bayer_pool,
5
      cuda_device_ordinal=self._cuda_device_ordinal,
6
  )
7
  # Configuring the converter tells us how big each received frame is.
8
  self._camera.configure_converter(csi_op)
9
  frame_size = csi_op.get_csi_length()

RoceReceiverOp configures itself from the enumeration metadata to listen for data on the enumerated channel. The device_start / device_stop callbacks start and stop the camera around streaming:

1
  receiver = hololink_module.operators.RoceReceiverOp(
2
      self,
3
      name="receiver",
4
      enumeration_metadata=self._metadata,
5
      frame_context=self._cuda_context,
6
      frame_size=frame_size,
7
      device_start=self._camera.start,
8
      device_stop=self._camera.stop,
9
  )

Three operators finish the pipeline:

• ImageProcessorOp is a hololink_module operator that provides a naive ISP of Bayer data (e.g. RGGB) using the GPU.
• BayerDemosaicOp is a standard Holoscan operator that interpolates the Bayer grid into RGBA.
• HolovizOp is a standard Holoscan operator that displays the result.

Wire the operators together:

1
  self.add_flow(receiver, csi_op, {("output", "input")})
2
  self.add_flow(csi_op, image_proc, {("output", "input")})
3
  self.add_flow(image_proc, demosaic, {("output", "receiver")})
4
  self.add_flow(demosaic, visualizer, {("transmitter", "receivers")})

Application initialization

The main program is responsible for finding the board, configuring it, and then running the pipeline described above.

Get the adapter — the object through which an application discovers and reaches boards — and wait for the board to announce itself. wait_for_channel blocks until the board at the given IP appears and returns its enumeration metadata:

1
  adapter = hololink_module.Adapter.get_adapter()
2
  metadata = adapter.wait_for_channel(HOLOLINK_IP, DISCOVERY_TIMEOUT)

Construct the camera driver from that metadata. The sensor driver is not a service: it is linked directly into your application, which is tightly coupled to this sensor. The driver resolves the services it needs from the metadata internally:

1
  camera = hololink_module.sensors.imx274.Imx274Cam(metadata)

Bring up the board. Fetch the control-plane service, start() it to open a socket to the board, and reset() it into a known state. The framework never changes device state on its own, so you must reset() to be sure the board is in a known configuration. Then configure the camera:

1
  hololink = hololink_module.HololinkInterfaceV1.get_service(metadata)
2
  # start() opens the control-plane socket; without it, no device I/O works.
3
  hololink.start()
4
  hololink.reset()
5
  # configure() writes device registers, so it must run after reset().
6
  camera.configure(CAMERA_MODE)
7
  camera.set_digital_gain_reg(0x4)

Run the application. When it exits, stop() the control plane to stop streaming and close the socket:

1
  app = HoloscanApplication(cu_context, cu_device_ordinal, metadata, camera)
2
  app.run()
3
  hololink.stop()

Build and run the application

Run the installed example. With a camera attached and the board reachable at 192.168.0.2, a Holoviz window opens and shows live video:

1
  $ python3 examples/imx274_module_tutorial.py

Background

Services are what let one application work with different boards — and different board revisions — without recompiling:

• Services are versioned: the version is part of the type name, for example HololinkInterfaceV1. Once an interface is published it never changes; new capabilities ship as the next version (for example HololinkInterfaceV2, usually an extension of V1). Modules keep publishing older versions too, so already-deployed applications keep working.
• Each board’s services are implemented in a device-specific shared library.
• Device enumeration (bootp) provides a device UUID. That UUID and a “compat-id” locate the shared object that implements the services for that board.
• The shared object supplies the requested interface by binding its virtual methods at runtime.
• Requesting an interface a board does not implement raises an exception — unless you pass allow_null=true, which tells the framework your application handles the missing-service case itself.