NVIDIA PyProf - Pytorch Profiler

PyProf is a tool that profiles and analyzes the GPU performance of PyTorch models. PyProf aggregates kernel performance from Nsight Systems or NvProf and provides the following additional features:

What’s New in 3.7.0

  • Monkey patching support for APEX libraries.

Features

  • Identifies the layer that launched a kernel: e.g. the association of ComputeOffsetsKernel with a concrete PyTorch layer or API is not obvious.

  • Identifies the tensor dimensions and precision: without knowing the tensor dimensions and precision, it’s impossible to reason about whether the actual (silicon) kernel time is close to maximum performance of such a kernel on the GPU. Knowing the tensor dimensions and precision, we can figure out the FLOPs and bandwidth required by a layer, and then determine how close to maximum performance the kernel is for that operation.

  • Forward-backward correlation: PyProf determines what the forward pass step is that resulted in the particular weight and data gradients (wgrad, dgrad), which makes it possible to determine the tensor dimensions required by these backprop steps to assess their performance.

  • Determines Tensor Core usage: PyProf can highlight the kernels that use Tensor Cores.

  • Correlate the line in the user’s code that launched a particular kernel (program trace).