Diagnostic Plugins#
This section describes the documented tests that the DCGM diagnostic module
can make available. Each page listed here covers the purpose, target
entities, failure signals, parameters, prerequisites, and examples for one
plugin. The test names are values accepted by dcgmi diag --run.
The Diagnostic Configuration File reference defines the
dcgm-diag-v1 schema, GPU matching, merge precedence, and common fields.
Each plugin page identifies the specific SKU-file map and fields it consumes.
Plugin availability is runtime-dependent. A DCGM release can know a canonical name even when the running host engine cannot load the matching plugin or its external executable on a particular system. The dcgmi diag reference is authoritative for test-selection syntax, suite membership, common options, output, and exit status.
The software result is a special case: it is a collection of built-in
deployment checks rather than a standalone --run software selection.
Run the level-1 suite to request those checks alone.
Interpreting results#
Plugin pages distinguish a diagnostic finding from a test that could not run:
PassThe plugin completed the checks described on its page and did not find a failure condition. A pass applies only to those checks; it is not a general statement that the GPU is healthy.
FailEither the plugin found the named defect or threshold violation, or it could not complete because of a setup or execution error. Use the reported
DCGM_FR_*code and message to distinguish those cases.SkipThe plugin did not produce a health verdict. Common reasons include a disabled test, unsupported hardware, an unavailable prerequisite, an incompatible GPU mode, insufficient free resources, or an interrupted run. Do not treat a skipped result as a pass.
Many workload plugins also monitor common conditions while they run, including uncorrectable memory errors, XIDs, row-remap state, thermal violations, and—when configured—single-bit errors and PCIe replay counts. Those conditions use the corresponding codes in Diagnostic Errors. Each plugin page identifies whether it performs that common monitoring.
Diagnostic |
Purpose |
|---|---|
Check deployment prerequisites and conditions such as libraries, device access, conflicting processes, environment settings, and reported GPU hardware state. |
|
Verify that DCGM can initialize CUDA and create a basic context on the target GPU. |
|
Exercise framebuffer allocation, reads and writes, data patterns, cache behavior, and observed memory errors. |
|
Exercise PCIe and peer-to-peer paths and check bandwidth, latency, link state, and error counters. |
|
Exercise compute and framebuffer data paths with sustained matrix operations while monitoring correctness and operating conditions. |
|
Measure local framebuffer bandwidth and compare it with the applicable device threshold. |
|
Apply a controlled GEMM workload and determine whether the GPU can sustain the requested compute rate. |
|
Drive the GPU toward a requested power level, check whether it reaches the configured threshold, and monitor operating conditions. |
|
Validate GPU copy paths and report measured bandwidth over available NVLink and PCIe connections. |
|
Run the installed NCCL Tests workload to validate supported single-node collective communication paths. |
|
Run the installed Extended Utility Diagnostics for supported NVIDIA GPUs or CPUs and integrate their results into DCGM Diagnostics. |
|
Exercise GPU memory with configurable address, bit, coupling, retention, and data-pattern tests. |
|
Expose board or power-delivery instability with rapid changes in GPU current and power demand. |
Error codes, categories, severities, and shared diagnostic messages are documented separately in the diagnostics-wide Diagnostic Errors reference.