Tools#

Nsight Python collects its data through an underlying NVIDIA tool. Only one tool can be active at a time, and nsight.activate() selects and loads it.

nsight.activate(tool: Tool)#

Select and load tool as the active tool for this process.

Only one tool can be active at a time. The first call loads the tool; calling again with the same tool is a no-op. Calling with a different tool while one is already active raises, because the tools are mutually exclusive.

Each tool has its own prerequisites and its own constraints on when it may be activated. See the Tools page for the tool you are selecting.

Parameters:

tool (Tool) – The tool to activate.

Raises:

ValueError – If tool is not a Tool.
RuntimeError – If a different, incompatible tool is already active.
ProfilerException – If the selected tool fails to load (wrong version, injection library missing, etc.). A tool whose prerequisites are missing may raise its own exception type instead.

Return type:

None

nsight.deactivate()#

Stop the currently active tool, if any.

Stops and tears down the tool activate() last activated. It is a no-op if no tool is active.

Not every tool can be torn down. A tool that cannot be stopped stays loaded for the lifetime of the process, so it also stays the active tool and this call is a no-op that warns rather than clearing the selection; no other tool can be activated afterwards. See the Tools page for which tools this applies to.

Raises:: ProfilerException – If tearing down the active tool fails.
Return type:: None

nsight.get_active_tool()#

Return the tool currently activated by activate(), or None.

Returns None if no tool is active (activate() has not been called, or the active tool was cleared by deactivate()). Doubles as an “is anything active?” check and lets callers branch on the active tool – e.g. skipping a test that needs a metric only one tool exposes.

Return type:: Tool | None
Returns:: The active Tool, or None if no tool is active.

class nsight.Tool(value)#

Bases: str, Enum

The profiling/analysis tools Nsight Python supports.

CUPTI = 'cupti'#

NCU = 'ncu'#

Supported tools#

NVIDIA Nsight Compute#

nsight.Tool.NCU profiles individual kernels with the NVIDIA Nsight Compute CLI. It is the default tool for @nsight.analyze.kernel.

Activation#

NVIDIA Nsight Compute’s injection library has to be loaded before CUDA is initialized. It is loaded in one of three ways.

At import, by default. import nsight activates NCU.
Explicitly, for control over the timing. Ensure the environment variable NSPY_NCU_INIT_AT_IMPORT is set to 0 before nsight is imported, then activate NCU where it suits you. Explicit activation also surfaces setup errors at that point.
```
import os

os.environ["NSPY_NCU_INIT_AT_IMPORT"] = "0"

import nsight

nsight.activate(nsight.Tool.NCU)
```
Lazily, on first use. Under NSPY_NCU_INIT_AT_IMPORT=0, the first call to a function decorated with @nsight.analyze.kernel loads the library. Ensure the first decorated call comes before any code that initializes CUDA.

Limitations#

Deactivation. deactivate() is not supported. The injection library cannot be unloaded, so NCU stays loaded for the lifetime of the process and remains the active tool once activated.

NVIDIA CUPTI (experimental)#

nsight.Tool.CUPTI is an opt-in, low-overhead alternative for collecting kernel time durations, measured using CUPTI’s Activity API.

Activation#

To activate CUPTI, ensure the environment variable NSPY_NCU_INIT_AT_IMPORT is set to 0 before nsight is imported:

import os

os.environ["NSPY_NCU_INIT_AT_IMPORT"] = "0"

import torch

import nsight

nsight.activate(nsight.Tool.CUPTI)


@nsight.analyze.kernel
def benchmark_matmul(n):
    a = torch.randn(n, n, device="cuda")
    b = torch.randn(n, n, device="cuda")

    with nsight.annotate("matmul"):
        c = a @ b


if __name__ == "__main__":
    result = benchmark_matmul(1024)
    print(result.to_dataframe())

    nsight.deactivate()

Activating CUPTI updates NVTX_INJECTION64_PATH. Any NVTX usage before nsight.activate(nsight.Tool.CUPTI) will cause NVTX annotation ranges to be missed and profiling results to be incomplete.

CUPTI may be activated either before or after CUDA is initialized.

Limitations#

Platform. Linux only for now. The CUPTI backend is not supported on Windows.

Metrics. Only "gpu__time_duration.sum" is supported. CUPTI measures it by recording kernel start and end timestamps from CUPTI activity records (CONCURRENT_KERNEL) and computing end - start per kernel launch.

Replay mode. Neither kernel replay nor range replay is supported. CUPTI activity records capture one execution of each kernel in a single pass, so no replay mechanism is engaged and replay_mode is ignored.

Clock and cache control. clock_control and cache_control are ignored. Clocks are not locked and caches are not flushed between runs, which may affect measurement stability.

Device synchronization. The whole device is synchronized once after the profiling session, before the activity buffers are flushed, because a kernel’s activity record is only written when the kernel completes. Work that the profiled function left running asynchronously is therefore waited for before profiling returns.