Video Replayer

So far we have been working with simple operators to demonstrate Holoscan SDK concepts. In this example, we look at two built-in Holoscan operators that have many practical applications.

In this example we’ll cover:

• how to load a video file from disk using VideoStreamReplayerOp operator

• how to display video using HolovizOp operator

• how to configure your operator’s parameters using a YAML configuration file

Here is the diagram of the operators and workflow used in this example.

Fig. 8 Workflow to load and display video from a file

We connect the “output” port of the replayer operator to the “receivers” port of the Holoviz operator.

The built-in video stream replayer operator can be used to replay a video stream that has been encoded as gxf entities. You can use the convert_video_to_gxf_entities.py script (installed in /opt/nvidia/holoscan/bin or available on GitHub) to encode a video file as gxf entities for use by this operator.

This operator processes the encoded file sequentially and supports realtime, faster than realtime, or slower than realtime playback of prerecorded data. The input data can optionally be repeated to loop forever or only for a specified count. For more details, see VideoStreamReplayerOp.

We will use the replayer to read gxf entities from disk and send the frames downstream to the Holoviz operator.

The built-in Holoviz operator provides the functionality to composite real time streams of frames with multiple different other layers like segmentation mask layers, geometry layers and GUI layers.

We will use Holoviz to display frames that have been sent by the replayer operator to it’s “receivers” port which can receive any number of inputs. In more intricate workflows, this port can receive multiple streams of input data where, for example, one stream is the original video data while other streams detect objects in the video to create bounding boxes and/or text overlays.

The SDK supports reading an optional YAML configuration file and can be used to customize the application’s workflow and operators. For more complex workflows, it may be helpful to use the application configuration file to help separate operator parameter settings from your code. See Configuring an Application for additional details.

This example uses the following configuration file to configure the parameters for the replayer and Holoviz operators. The full list of parameters can be found at VideoStreamReplayerOp and HolovizOp.

            

            
%YAML 1.2
replayer:
  directory: "../data/racerx"   # Path to gxf entity video data
  basename: "racerx"             # Look for <basename>.gxf_{entities|index}
  frame_rate: 0       # Frame rate to replay.  (default: 0 follow frame rate in timestamps)
  repeat: true        # Loop video?   (default: false)
  realtime: true      # Play in realtime, based on frame_rate/timestamps   (default: true)
  count: 0            # Number of frames to read  (default: 0 for no frame count restriction)

holoviz:
  width: 854          # width of window size
  height: 480         # height of window size
  tensors:
    - name: ""        # name of tensor containing input data to display
      type: color     # input type e.g., color, triangles, text, depth_map
      opacity: 1.0    # layer opacity
      priority: 0     # determines render order, higher priority layers are rendered on top
        

The code below shows our video_replayer example. Operator parameters are configured from a configuration file using from_config() (C++) and self.**kwargs() (Python).

            

            
#include <holoscan/holoscan.hpp>
#include <holoscan/operators/video_stream_replayer/video_stream_replayer.hpp>
#include <holoscan/operators/holoviz/holoviz.hpp>

class VideoReplayerApp : public holoscan::Application {
 public:
  void compose() override {
    using namespace holoscan;

    // Define the replayer and holoviz operators and configure using yaml configuration
    auto replayer = make_operator<ops::VideoStreamReplayerOp>("replayer", from_config("replayer"));
    auto visualizer = make_operator<ops::HolovizOp>("holoviz", from_config("holoviz"));

    // Define the workflow: replayer -> holoviz
    add_flow(replayer, visualizer, {{"output", "receivers"}});
  }
};

int main(int argc, char** argv) {
  // Get the yaml configuration file
  auto config_path = std::filesystem::canonical(argv[0]).parent_path();
  config_path /= std::filesystem::path("video_replayer.yaml");
  if ( argc >= 2 ) {
    config_path = argv[1];
  }

  auto app = holoscan::make_application<VideoReplayerApp>();
  app->config(config_path);
  app->run();

  return 0;
}
        

            

            
import os
import sys

from holoscan.core import Application
from holoscan.operators import HolovizOp, VideoStreamReplayerOp

sample_data_path = os.environ.get("HOLOSCAN_INPUT_PATH", "../data")


class VideoReplayerApp(Application):
    """Example of an application that uses the operators defined above.

This application has the following operators:

- VideoStreamReplayerOp
- HolovizOp

The VideoStreamReplayerOp reads a video file and sends the frames to the HolovizOp.
The HolovizOp displays the frames.
"""

    def compose(self):
        video_dir = os.path.join(sample_data_path, "racerx")
        if not os.path.exists(video_dir):
            raise ValueError(f"Could not find video data:{video_dir=}")

        # Define the replayer and holoviz operators
        replayer = VideoStreamReplayerOp(
            self, name="replayer", directory=video_dir, **self.kwargs("replayer")
        )
        visualizer = HolovizOp(self, name="holoviz", **self.kwargs("holoviz"))

        # Define the workflow
        self.add_flow(replayer, visualizer, {("output", "receivers")})


def main(config_file):
    app = VideoReplayerApp()
    # if the --config command line argument was provided, it will override this config_file
    app.config(config_file)
    app.run()


if __name__ == "__main__":
    config_file = os.path.join(os.path.dirname(__file__), "video_replayer.yaml")
    main(config_file=config_file)
        

Running the application should bring up video playback of the video referenced in the YAML file.