Analysis Guide#
Running analysis recipes on collected profiles.
Conduct Analysis#
What is a Recipe?#
A recipe is a packaged Nsight Systems analysis script that takes one or
more .nsys-rep reports as input and produces a human-readable output –
most commonly a Jupyter notebook with tables and plots, plus supplementary
CSV and log files. Recipes ship with the Nsight Systems CLI and cover
common patterns such as CUDA API summaries, memory transfer breakdowns,
NCCL timeline analysis, Triton inference latency, and more. The full
catalog of recipes and their arguments is documented in the
Nsight Systems Analysis Guide.
The NsightAnalysis service runs recipes inside the cluster so that they operate on reports already in your storage backend without requiring you to download them. There are two entry points to the same service:
Command Line Interface –
analysis list,analysis run,analysis reports ...subcommands.Graphical User Interface – browser UI served at the gateway root (port
8888by default).
Output: Jupyter Notebooks and More#
Most recipes produce a .ipynb Jupyter notebook as their primary output,
alongside log files and any CSV/JSON artifacts the recipe emits. The
analysis service renders notebooks directly via analysis reports open
(opens the notebook in your browser via the gateway’s embedded notebook
viewer) or analysis reports download (fetches the raw files so you can
open them in Jupyter locally or in VSCode).
Session and Collection Scoping#
By default, a recipe runs against all reports in the current active
session. Use --session <UUID> to target a specific session and
--collection <ID> to target a single collection within that session.
Useful for comparing consecutive runs or iterating on a recipe against a
stable reference capture.
OTLP Trace Mirroring#
When otlpMirroringEnabled: true is set on a nsight tool config (see
NsightOperatorProfileConfig), NVTX ranges captured during
profiling are mirrored as OTLP spans into the
NsightOtelCollector. From there, you can:
Export to an external observability backend (Jaeger, Tempo, Zipkin, Datadog, Prometheus / OpenTelemetry-compatible) via the collector’s standard exporters.
Convert the mirrored spans back into
.nsys-repfiles via the optional OTLP converter sidecar (otlpConverter.enabled: true), so the same data can be analyzed using the native Nsight Systems toolchain.
The OTLP proxy sidecar that mirrors traces is injected into profiled Pods automatically when the OTLPProxyConfig CR is present and enabled.
Prerequisites#
The analysis service is reached through the
NsightGateway. Analysis routes are prefixed
by the analysisPrefix defined in the
routing configuration (by default
/analysis/).
Command Line Interface#
Nsight Systems Recipes can also be run with the nsight_operator.py script, found in the NVIDIA Nsight Operator Resources.
Before using analysis commands, you must configure the CLI.
Command |
Description |
|---|---|
|
List all available recipes. |
|
Get detailed information about a recipe. |
|
Creates a job to run a recipe with optional arguments. The arguments are
the same as those provided to |
|
List all jobs. If |
|
Get details of a specific job. The URL containing the job’s report output is shown if the job has finished successfully. |
|
View logs for a job. |
|
Download files outputted by a job. If |
|
Open the Jupyter Notebook(s) associated with the job in the default browser. |
|
Cancel a running job. |
|
Delete a job. |
|
Open the analysis UI for the current session in the default browser. |
Graphical User Interface#
Nsight Systems recipes can also be run through a web-based GUI. By default, the
UI is exposed at the gateway root, which by default is on port 8888. For more
information on how to connect to the gateway, see
Accessing the Gateway.
To open the UI for a specific session use the
python3 nsight_operator.py analysis ui command. To view a notebook produced by
a job, use the python3 nsight_operator.py analysis reports open <id> command.
Run an analysis#
To run an analysis, first ensure you’re on the “New Analysis” tab. Next, select the analysis you wish to run. You can use the search box to help find the analysis. Fill out the form on the right side of the screen to specify any arguments required by the analysis. Finally, click the “Start” button.
Check the status of an analysis job#
After starting an analysis job, you will be redirected to the “Analyses” tab. The right column contains the status of the job, the log output (if any), the amount of time that the job has been running for, and the cancel button. The cancel button will be replaced with an “Open” notebook button once the job finishes successfully.
Open Jupyter notebooks created by an analysis job#
To open the Jupyter Notebook(s) produced by the session, click on the “Open” button in the bottom right of the completed job.
Open a Nsight Systems streamer for a session#
To open an Nsight Systems streamer for the current session, click the “View Trace” button in the top right corner. All collected files for the session will be available to view in the streamer.