Video Replayer (Distributed)

Holoscan v0.6

In this example, we extend the previous video replayer application into a multi-node distributed application. A distributed application is made up of multiple Fragments (C++/Python), each of which may run on its own node.

In the distributed case we will:

  • create one fragment that loads a video file from disk using VideoStreamReplayerOp operator

  • create a second fragment that will display the video using the HolovizOp operator

These two fragments will be combined into a distributed application such that the display of the video frames could occur on a separate node from the node where the data is read.

Note

The example source code and run instructions can be found in the examples directory on GitHub, or under /opt/nvidia/holoscan/examples in the NGC container and the debian package, alongside their executables.

Here is the diagram of the operators and workflow used in this example.

%%{init: {"theme": "base", "themeVariables": { "fontSize": "16px"}} }%% classDiagram direction LR VideoStreamReplayerOp --|> HolovizOp : output...receivers class VideoStreamReplayerOp { output(out) Tensor } class HolovizOp { [in]receivers : Tensor }

Fig. 9 Workflow to load and display video from a file

This is the same workflow as the single fragment video replayer, each operator is assigned to a separate fragment and there is now a network connection between the fragments.

Distributed applications define Fragments explicitly to isolate the different units of work that could be distributed to different nodes. In this example:

  • We define two classes that inherit from Fragment:

    • Fragment1 contains an instance of VideoStreamReplayerOp named “replayer”.

    • Fragment2 contains an instance of HolovizOp name “holoviz”.

  • We create an application, DistributedVideoReplayerApp. In its compose method:

    • we call make_fragment to initialize both fragments.

    • we then connect the “output” port of “replayer” operator in fragment1 to the “receivers” port of the “holoviz” operator in fragment2 to define the application workflow.

  • The operators instantiated in the fragments can still be configured with parameters initialized from the YAML configuration ingested by the application using from_config() (C++) or kwargs() (Python).

Copy
Copied!
            

#include <holoscan/holoscan.hpp> #include <holoscan/operators/holoviz/holoviz.hpp> #include <holoscan/operators/video_stream_replayer/video_stream_replayer.hpp> class Fragment1 : public holoscan::Fragment { public: void compose() override { using namespace holoscan; auto replayer = make_operator<ops::VideoStreamReplayerOp>("replayer", from_config("replayer")); add_operator(replayer); } }; class Fragment2 : public holoscan::Fragment { public: void compose() override { using namespace holoscan; auto visualizer = make_operator<ops::HolovizOp>("holoviz", from_config("holoviz")); add_operator(visualizer); } }; class DistributedVideoReplayerApp : public holoscan::Application { public: void compose() override { using namespace holoscan; auto fragment1 = make_fragment<Fragment1>("fragment1"); auto fragment2 = make_fragment<Fragment2>("fragment2"); // Define the workflow: replayer -> holoviz add_flow(fragment1, fragment2, {{"replayer.output", "holoviz.receivers"}}); } }; int main(int argc, char** argv) { // Get the yaml configuration file auto config_path = std::filesystem::canonical(argv[0]).parent_path(); config_path /= std::filesystem::path("video_replayer_distributed.yaml"); auto app = holoscan::make_application<DistributedVideoReplayerApp>(); app->config(config_path); app->run(); return 0; }

Copy
Copied!
            

import os from holoscan.core import Application, Fragment from holoscan.operators import HolovizOp, VideoStreamReplayerOp sample_data_path = os.environ.get("HOLOSCAN_INPUT_PATH", "../data") class Fragment1(Fragment): def __init__(self, app, name): super().__init__(app, name) def compose(self): # Set the video source video_path = self._get_input_path() logging.info( f"Using video from{video_path}" ) # Define the replayer and holoviz operators replayer = VideoStreamReplayerOp( self, name="replayer", directory=video_path, **self.kwargs("replayer") ) self.add_operator(replayer) def _get_input_path(self): path = os.environ.get( "HOLOSCAN_INPUT_PATH", os.path.join(os.path.dirname(__file__), "data") ) return os.path.join(path, "endoscopy/video") class Fragment2(Fragment): def compose(self): visualizer = HolovizOp(self, name="holoviz", **self.kwargs("holoviz")) self.add_operator(visualizer) class DistributedVideoReplayerApp(Application): """Example of a distributed application that uses the fragments and operators defined above. This application has the following fragments: - Fragment1 - holding VideoStreamReplayerOp - Fragment2 - holding HolovizOp The VideoStreamReplayerOp reads a video file and sends the frames to the HolovizOp. The HolovizOp displays the frames. """ def compose(self): # Define the fragments fragment1 = Fragment1(self, name="fragment1") fragment2 = Fragment2(self, name="fragment2") # Define the workflow self.add_flow(fragment1, fragment2, {("replayer.output", "holoviz.receivers")}) if __name__ == "__main__": config_file = os.path.join(os.path.dirname(__file__), "video_replayer_distributed.yaml") app = DistributedVideoReplayerApp() app.config(config_file) app.run()

This particular distributed application only has one operator per fragment, so the operators was added via add_operator ( C++/ Python). In general, each fragment may have multiple operators and connections between operators within a fragment would be made using add_flow() (C++/Python) method within the fragment’s compute() (C++/Python) method.

Running the application should bring up video playback of the surgical video referenced in the YAML file.

video_replayer.png

Note

Instructions for running the distributed application involve calling the application from the “driver” node as well as from any worker nodes. For details, see the application run instructions in the examples directory on GitHub, or under /opt/nvidia/holoscan/examples/video_replayer_distributed in the NGC container and the debian package.

Tip

Refer to UCX Network Interface Selection when running a distributed application across multiple nodes.

Previous Video Replayer
Next Bring Your Own Model (BYOM)
© Copyright 2022-2023, NVIDIA. Last updated on Feb 9, 2024.