Nsight Profiling

Holoscan SDK emits NVTX annotations for runtime profiling. In HSDK 5, operator compute callbacks and message emit/receive events are annotated in the holoscan NVTX domain so Nsight Systems can show where an application spends time and how messages move between operators.

To capture the annotations, run the application under nsys profile with NVTX tracing enabled. For example, after building the SDK examples, this command profiles the HSDK 5 first_pipeline C++ example and writes first_pipeline_compute.nsys-rep:

nsys profile -t nvtx --sample=none --backtrace=none --cpuctxsw=none \
  -o first_pipeline_compute -f true \
  ./build-x86_64/examples/first_pipeline/cpp/first_pipeline

Operator-centric logging should be captured with NVTX tracing only. Avoid CPU sampling, backtraces, and CPU context-switch tracing unless you specifically need them for a separate investigation, because those collection modes add profiler overhead. Use these low-overhead flags whenever capturing operator-centric annotations:

nsys profile -t nvtx --sample=none --backtrace=none --cpuctxsw=none \
  -o my_holoscan_profile -f true ./my_holoscan_app

Open the written report with the Nsight Systems UI:

nsys-ui first_pipeline_compute.nsys-rep

Operator compute ranges

Operator compute annotations appear in the holoscan NVTX domain. Each range is labeled with the operator identity followed by :compute.

For operators in the root graph, the label form is:

<operator>:compute

For the first_pipeline example, the expected root-graph compute labels are:

source:compute
doubler:compute
sink:compute

For operators inside reusable or nested graphs, the label includes the complete root-to-leaf graph instance path before the operator name:

<graph>/<nested-graph>/<operator>:compute

For example, the reusable_graphs example embeds a ScalePairGraph instance named direct-stage; its internal ScaleOp operator is named scale, so its compute range is labeled:

direct-stage/scale:compute

The root graph’s name is omitted. Repeated nested instances remain distinct: an operator named scale inside left/stage has the label left/stage/scale:compute, while the same operator inside right/stage has the label right/stage/scale:compute.

What the compute range measures

A :compute range starts immediately before the runtime invokes the operator callback and ends immediately after that callback returns. It covers the call to the operator’s compute() implementation through the runtime access layer.

The range does not include the runtime work that happens after the callback returns, such as callback-return tracing, CUDA-device restoration, dispatcher completion, or direct-continuation polling.

Message emit and receive markers

Message markers are instantaneous NVTX events. An operator output uses <operator>.<output-port>:emit, and an operator input uses <operator>.<input-port>:receive. Nested operators use the same graph prefix as compute ranges:

<graph>/<nested-graph>/<operator>.<output-port>:emit
<graph>/<nested-graph>/<operator>.<input-port>:receive

A <port>:emit marker means the runtime accepted the message, or delivery may already have started. A cleanly rejected send attempt produces no marker.

Because delivery can start before the emit marker is recorded, a matching <port>:receive marker may appear first. Match the markers using their source identity and sequence, not their timestamp order.

Each marker carries a structured NVTX payload:

To include the structured payload bytes in a SQLite export:

nsys export --type sqlite --include-blobs=true \
  -o first_pipeline_compute.sqlite first_pipeline_compute.nsys-rep

The profiling examples above use -f true to overwrite an existing report. The export command leaves overwrite disabled; choose a new output name or add -f true to replace an existing export.

Label escaping

Graph path segments use /, operator and port names are separated by ., and the event suffix follows :. Within each name component, a literal backslash, dot, colon, or slash is escaped with a preceding backslash.

For example, an operator named camera.front inside an embedded graph named left/right has the compute label left\/right/camera\.front:compute. The escaped slash belongs to one graph name; an unescaped slash separates graph instances.

Finding annotations in Nsight Systems

In Nsight Systems, look for the NVTX domain named holoscan, then expand the runtime thread lanes that executed operators. The compute ranges and message markers are displayed with labels such as:

source:compute
source.out:emit
doubler.in:receive
doubler:compute
sink:compute
direct-stage/scale:compute

These labels are registered NVTX messages, so they may also appear in exported report data as registered-string message references depending on the export format.

Use Show in Events View from the NVTX context menu to inspect the annotations as a table and search for operator names, port names, or the :emit, :receive, and :compute suffixes.