Diagnostic Plugins#

This section describes the documented tests that the DCGM diagnostic module can make available. Each page listed here covers the purpose, target entities, failure signals, parameters, prerequisites, and examples for one plugin. The test names are values accepted by dcgmi diag --run.

The Diagnostic Configuration File reference defines the dcgm-diag-v1 schema, GPU matching, merge precedence, and common fields. Each plugin page identifies the specific SKU-file map and fields it consumes.

Plugin availability is runtime-dependent. A DCGM release can know a canonical name even when the running host engine cannot load the matching plugin or its external executable on a particular system. The dcgmi diag reference is authoritative for test-selection syntax, suite membership, common options, output, and exit status.

The software result is a special case: it is a collection of built-in deployment checks rather than a standalone --run software selection. Run the level-1 suite to request those checks alone.

Interpreting results#

Plugin pages distinguish a diagnostic finding from a test that could not run:

Pass

The plugin completed the checks described on its page and did not find a failure condition. A pass applies only to those checks; it is not a general statement that the GPU is healthy.

Fail

Either the plugin found the named defect or threshold violation, or it could not complete because of a setup or execution error. Use the reported DCGM_FR_* code and message to distinguish those cases.

Skip

The plugin did not produce a health verdict. Common reasons include a disabled test, unsupported hardware, an unavailable prerequisite, an incompatible GPU mode, insufficient free resources, or an interrupted run. Do not treat a skipped result as a pass.

Many workload plugins also monitor common conditions while they run, including uncorrectable memory errors, XIDs, row-remap state, thermal violations, and—when configured—single-bit errors and PCIe replay counts. Those conditions use the corresponding codes in Diagnostic Errors. Each plugin page identifies whether it performs that common monitoring.

Diagnostic plugin reference#

Diagnostic

Purpose

software

Check deployment prerequisites and conditions such as libraries, device access, conflicting processes, environment settings, and reported GPU hardware state.

context_create

Verify that DCGM can initialize CUDA and create a basic context on the target GPU.

memory

Exercise framebuffer allocation, reads and writes, data patterns, cache behavior, and observed memory errors.

pcie

Exercise PCIe and peer-to-peer paths and check bandwidth, latency, link state, and error counters.

diagnostic

Exercise compute and framebuffer data paths with sustained matrix operations while monitoring correctness and operating conditions.

memory_bandwidth

Measure local framebuffer bandwidth and compare it with the applicable device threshold.

targeted_stress

Apply a controlled GEMM workload and determine whether the GPU can sustain the requested compute rate.

targeted_power

Drive the GPU toward a requested power level, check whether it reaches the configured threshold, and monitor operating conditions.

nvbandwidth

Validate GPU copy paths and report measured bandwidth over available NVLink and PCIe connections.

nccl_tests

Run the installed NCCL Tests workload to validate supported single-node collective communication paths.

eud and cpu_eud

Run the installed Extended Utility Diagnostics for supported NVIDIA GPUs or CPUs and integrate their results into DCGM Diagnostics.

memtest

Exercise GPU memory with configurable address, bit, coupling, retention, and data-pattern tests.

pulse_test

Expose board or power-delivery instability with rapid changes in GPU current and power demand.

Error codes, categories, severities, and shared diagnostic messages are documented separately in the diagnostics-wide Diagnostic Errors reference.