Analysis Guide#

Running analysis recipes on collected profiles.

Conduct Analysis#

What is a Recipe?#

A recipe is a packaged Nsight Systems analysis script that takes one or more .nsys-rep reports as input and produces a human-readable output – most commonly a Jupyter notebook with tables and plots, plus supplementary CSV and log files. Recipes ship with the Nsight Systems CLI and cover common patterns such as CUDA API summaries, memory transfer breakdowns, NCCL timeline analysis, Triton inference latency, and more. The full catalog of recipes and their arguments is documented in the Nsight Systems Analysis Guide.

The NsightAnalysis service runs recipes inside the cluster so that they operate on reports already in your storage backend without requiring you to download them. There are two entry points to the same service:

Output: Jupyter Notebooks and More#

Most recipes produce a .ipynb Jupyter notebook as their primary output, alongside log files and any CSV/JSON artifacts the recipe emits. The analysis service renders notebooks directly via analysis reports open (opens the notebook in your browser via the gateway’s embedded notebook viewer) or analysis reports download (fetches the raw files so you can open them in Jupyter locally or in VSCode).

Session and Collection Scoping#

By default, a recipe runs against all reports in the current active session. Use --session <UUID> to target a specific session and --collection <ID> to target a single collection within that session. Useful for comparing consecutive runs or iterating on a recipe against a stable reference capture.

OTLP Trace Mirroring#

When otlpMirroringEnabled: true is set on a nsight tool config (see NsightOperatorProfileConfig), NVTX ranges captured during profiling are mirrored as OTLP spans into the NsightOtelCollector. From there, you can:

  • Export to an external observability backend (Jaeger, Tempo, Zipkin, Datadog, Prometheus / OpenTelemetry-compatible) via the collector’s standard exporters.

  • Convert the mirrored spans back into .nsys-rep files via the optional OTLP converter sidecar (otlpConverter.enabled: true), so the same data can be analyzed using the native Nsight Systems toolchain.

The OTLP proxy sidecar that mirrors traces is injected into profiled Pods automatically when the OTLPProxyConfig CR is present and enabled.

Prerequisites#

The analysis service is reached through the NsightGateway. Analysis routes are prefixed by the analysisPrefix defined in the routing configuration (by default /analysis/).

Command Line Interface#

Nsight Systems Recipes can also be run with the nsight_operator.py script, found in the NVIDIA Nsight Operator Resources.

Before using analysis commands, you must configure the CLI.

Command

Description

analysis list

List all available recipes.

analysis info <name>

Get detailed information about a recipe.

analysis run [--session SESSION_ID] [--collection COLLECTION_ID] <name> [args...]

Creates a job to run a recipe with optional arguments. The arguments are the same as those provided to nsys recipe. Specify --session to provide profiles from the corresponding session and/or --collection for a specific collection.

analysis reports list [--status STATUS]

List all jobs. If --status is provided, the list is filtered. STATUS can be RUNNING, SUCCESS, NO_DATA, ERROR, CANCELLED, or UNKNOWN.

analysis reports info <id>

Get details of a specific job. The URL containing the job’s report output is shown if the job has finished successfully.

analysis reports logs <id>

View logs for a job.

analysis reports download <id> [--output-dir DIRECTORY]

Download files outputted by a job. If --output-dir is not specified, files are saved to the current directory.

analysis reports open <id>

Open the Jupyter Notebook(s) associated with the job in the default browser.

analysis reports cancel <id>

Cancel a running job.

analysis reports delete <id>

Delete a job.

analysis ui

Open the analysis UI for the current session in the default browser.

Graphical User Interface#

Nsight Systems recipes can also be run through a web-based GUI. By default, the UI is exposed at the gateway root, which by default is on port 8888. For more information on how to connect to the gateway, see Accessing the Gateway.

To open the UI for a specific session use the python3 nsight_operator.py analysis ui command. To view a notebook produced by a job, use the python3 nsight_operator.py analysis reports open <id> command.

Run an analysis#

To run an analysis, first ensure you’re on the “New Analysis” tab. Next, select the analysis you wish to run. You can use the search box to help find the analysis. Fill out the form on the right side of the screen to specify any arguments required by the analysis. Finally, click the “Start” button.

Check the status of an analysis job#

After starting an analysis job, you will be redirected to the “Analyses” tab. The right column contains the status of the job, the log output (if any), the amount of time that the job has been running for, and the cancel button. The cancel button will be replaced with an “Open” notebook button once the job finishes successfully.

Open Jupyter notebooks created by an analysis job#

To open the Jupyter Notebook(s) produced by the session, click on the “Open” button in the bottom right of the completed job.

Open a Nsight Systems streamer for a session#

To open an Nsight Systems streamer for the current session, click the “View Trace” button in the top right corner. All collected files for the session will be available to view in the streamer.