Application Workflow

This workflow is for application developers to create AI Application using published extensions and deploy those graphs either to containers or on native systems. Application development is supported only on x86, but can be deployed on both Jetson and x86.

Application workflow includes following steps:

  1. Launch Composer

  2. Sync Extensions from NVIDIA Cloud repository

  3. Load Graph and parameters

  4. Deploy AI Application

Launch Composer

There are two options to launch Composer:

  1. Native workstation:

Launch Composer from native workstation using following command:

composer

  1. DeepStream SDK devel container image:

Launch Composer from DeepStream SDK devel container image:

docker pull nvcr.io/nvidia/deepstream:6.1.1-devel
xhost +
docker run -it --entrypoint /bin/bash --gpus all --rm --network=host -e DISPLAY=${DISPLAY} -v /tmp/.X11-unix/:/tmp/.X11-unix --privileged -v /var/run/docker.sock:/var/run/docker.sock nvcr.io/nvidia/deepstream:6.1.1-devel
composer
Graph Composer Default Window

Sync Extensions

Before any graph can be executed or container built, extensions from NGC public repo must be synced. Follow the steps below to sync the extensions:

  1. Close any open graphs

Graph Composer Close Graphs

  1. Open the Registry menu from the menubar at the top and click on Sync Repo

Graph Composer Registry Menu

  1. Select ngc-public from the drop-down list and click on Sync

Graph Composer Select Repo

  1. The composer reports the current status using a progress bar.

Graph Composer Sync Progress

  1. Once the extension sync is complete, the composer displays a success message.

Graph Composer Sync Success

  1. On clicking close, the composer automatically refreshes component list. You can see the refreshed list in the component list window on the right.

Graph Composer Default View

Load Graph

  1. Open the File menu from the menubar at the top and click on Open Graph to launch the file browser. You may alternatively use the Ctrl + O key combination.

Graph Composer File Menu

  1. Browse to a valid graph file, select it and click on Okay to open the graph.

Graph Composer Open Graph

  1. Composer should now show the application graph.

Graph Composer Test1 Graph

  1. To load component parameters from a separate file, right-click on the graph and select Load parameters from the context menu to launch the file browser.

Graph Composer Load Parameters Menu

  1. Browse to an appropriate parameters file for the currently open and visible graph, select it and click on Okay to load parameter values from the file.

Graph Composer Load Parameters

Deploy Graph

You can deploy the graph using one of the following methods:

  1. Execute on x86_64 (host)

  2. Execute on Jetson or another remote system

  3. Execute on remote system (x86_64 Linux, Jetson) through Windows

  4. Build Container Image

  5. Build Container Image through Windows

  6. Execute graph using commandline

Execute on x86_64 (Same Host)

To execute the currently open graph, click on the Run Graph button from the toolbar on the left. This will open the Run Graph dialog.

Graph Composer Run Graph Toolbar button

  • To execute on the same host system, select Local Machine radio button, select appropriate Platform config file using the file browser and click on Run. The graph execution progress will be reported via logs in the console window.

Graph Composer Local Run

Execute on Jetson or another remote system

  1. Launch the Run Graph dialog and select Remote Machine radio button.

  2. Select appropriate Platform config file (aarch64 or x86_64) based on the remote machine configuration.

  3. Set any environment variables required to execute the graph remotely (e.g. DISPLAY=:0) enter the remote system details and click on Run.

  4. The graph execution progress will be reported via logs in the console window.

Graph Composer Remote Run

Please note that this requires Graph Composer package be installed on the remote system.

Execute on Jetson or another remote system through Windows

Executing graph through Windows is very similar to executing graph on Jetson or another remote system. Please note that this requires Graph Composer package be installed on the remote system.

Build Container Image

1. To build a container, first click on the Build Container button from the toolbar on the top.

Graph Composer Container Builder Toolbar Button

  1. This will open the Build Container dialog. Launch the file browser using the button next to the Configuration File input.

Graph Composer Container Builder Launch

  1. Select a container builder configuration file and open it.

Graph Composer Container Builder Config

  1. Click the button next to the Platform config File input to launch the file browser. Select a platform config file and open it.

Graph Composer Container Builder Platform

  1. Click on Build to start the build process. Composer reports the container build status using a progress bar.

Graph Composer Container Builder Progress

On successful completion, composer will show a success message.

Graph Composer Container Builder Success

Build Container Image through Windows

Building container image through Windows is very similar to building container image on linux system. It only needs remote system details. Please note that this requires Graph Composer package be installed on the remote system.

Graph Composer Container Builder On Windows

Execute Graph using commandline (execute_graph.sh script)

The execute_graph.sh script provided with the graph composer helps with graph execution and provides added functionality.

Complete usage reference:

    Usage: /opt/nvidia/graph-composer/execute_graph.sh [options] <graph-file> [additional graph files]

Options:
  -d, --graph-target "<graph-target-file>"    [Required] Graph target config file
  -s, --subgraphs <subgraph1>,<subgraph2>,... [Optional] Paths of subgraphs used by the application, comma-separated list
      --resources <graph-resources-file>      [Optional] Graph resources file
  -f, --fresh-manifest                        [Optional] Re-install graph and generate a new manifest file
  -g, --with-gdb                              [Optional] Execute the graph under gdb
  -m, --use-manifest <existing-manifest>      [Optional] Use an existing manifest file
  -r, --remove-install-dir                    [Optional] Remove graph installation directory during exit
  -t, --target <username@host>                [Optional] Target to execute the graph on. SSH will be used
      --target-env-vars "<env-vars>"          [Optional] Separated list of environment variables to be set before running on target

Note

To execute graphs on a remote target: * Graph Composer package must already be installed on the target * It is recommended that a password-less login method be used for SSH

  • To execute a graph locally, run:

    /opt/nvidia/graph-composer/execute_graph.sh <graph-file> -d <graph-target>

    For example, on dGPU host, run:

    /opt/nvidia/graph-composer/execute_graph.sh <graph-file> -d /opt/nvidia/graph-composer/config/target_x86_64.yaml

To execute on a remote Jetson target, run:

/opt/nvidia/graph-composer/execute_graph.sh <graph-file> -d /opt/nvidia/graph-composer/config/target_aarch64.yaml \
-t <username@host> --target-env-vars "DISPLAY=:0"

Note

If a graph has resources associated with it described in a resources YAML file, an additional argument --resources <resources.yaml> can be passed to the script. The resources would be copied to the remote target before graph execution

Note

When executing a graph that uses subgraphs, you must pass additional argument -s <subgraph1>,<subgraph2>,... containing paths to the subgraph files. You must not pass the subgraphs as graph file arguments without an option.

Note

To run the graph on the remote machine, install the following packages:

  1. openssh-client

  2. sshfs

Use ssh-keygen to generate an ssh key pair. Copy key to target using ssh-copy-id ${TARGET}