Core Concepts#

Nsight Python requires two components working together to profile your GPU code:

  • @nsight.analyze.kernel - A decorator on your benchmark function that runs the profiler and collects metrics

  • nsight.annotate - A context manager (or decorator) used inside your benchmark function to mark which kernel(s) to measure

Both are required. The @nsight.analyze.kernel decorator controls the profiling session, while nsight.annotate tells the profiler which specific code regions to measure.

@nsight.analyze.kernel  # Controls profiling session
def benchmark(n):
    a = torch.randn(n, n, device="cuda")
    b = torch.randn(n, n, device="cuda")

    with nsight.annotate("matmul"):  # Marks the kernel to measure
        c = a @ b

Nsight Python operates through three key primitives:

1. Annotations An annotation wraps a region of code that launches a GPU kernel and tags it for profiling. By default, each annotation should contain exactly one kernel launch. Annotations can be used as decorators or context managers:

@nsight.annotate("torch")
def torch_kernel():
    ...

# or
with nsight.annotate("cutlass4"):
    cutlass_kernel()

If your annotated region launches multiple kernels, you have two options:

  • Use replay_mode="range" to profile the entire annotated range as a unit, with metrics associated with the range rather than individual kernels

  • Use combine_kernel_metrics to specify how to aggregate metrics across individual kernels (e.g., sum the runtimes)

See the @nsight.analyze.kernel documentation for details on these parameters.

2. Kernel Analysis Decorator Use nsight.analyze.kernel() to annotate a benchmark function. Nsight Python will rerun this function one configuration at a time. You can provide configurations in two ways:

  • At decoration time using the configs parameter.

  • At function call time by passing configs directly as an argument when invoking the decorated function.

@nsight.analyze.kernel
def benchmark(s):
    ...

benchmark(configs=[(1024,), (2048,)])

Config values are mapped to function parameters in declaration order. The following parameter kinds are supported:

  • Regular parameters (POSITIONAL_OR_KEYWORD): fully supported.

  • Keyword-only parameters (after * or *args): fully supported — config values are automatically passed as keyword arguments.

  • Parameters with default values: configs may omit trailing parameters that have defaults. The defaults are filled in automatically.

  • *args and **kwargs: tolerated in the signature but ignored — they will always be empty during profiling and will not appear in the results.

@nsight.analyze.kernel
def benchmark(x, y, *, mode="fast"):
    ...

# mode has a default, so it can be omitted from configs.
# These two calls are equivalent:
benchmark(configs=[(1024, 2048)])
benchmark(configs=[(1024, 2048, "fast")])

3. Plot Decorator Add nsight.analyze.plot() to automatically generate plots from your profiling runs.

@nsight.analyze.plot(filename="plot.png", ylabel="Runtime (ns)")
@nsight.analyze.kernel(configs=[(1024,), (2048,)])
def benchmark(s):
    ...