DRA Topology Provider

The DRA provider reads accelerator-domain IDs from an existing Kubernetes Node label and defaults to nvidia.com/gpu.clique, generated by the NVIDIA GPU Operator’s GPU Feature Discovery (GFD), specifically its IMEX labeler, and groups nodes by NVLink partition. Its supported use is generating Slurm topology/block data with the Slinky engine. It uses in-cluster service account auth, so no credentials are required.

Important: The DRA provider produces block topology only (topology/block — NVLink partition membership). It does not discover the backend InfiniBand or Ethernet switch fabric and therefore cannot express network proximity between partitions. When a workload exceeds one partition, Slurm’s selection of additional blocks is not guided by the physical network topology. Use the InfiniBand or NetQ provider when cross-partition locality matters.

Background: ComputeDomains and MNNVL

On GB200 NVL72 and similar Multi-Node NVLink (MNNVL) hardware, groups of nodes share a high-bandwidth NVLink fabric (1.8 TB/s chip-to-chip). Workloads that span these nodes — distributed training, disaggregated inference — benefit significantly from being placed within the same NVLink domain.

Kubernetes can expose MNNVL membership through ComputeDomains, a DRA-based abstraction representing nodes that share an NVLink domain. Separately, some NVIDIA GPU Operator deployments expose each node’s clique membership in the nvidia.com/gpu.clique label. The Topograph DRA provider reads that node label; it does not read ComputeDomain resources or discover the fabric itself.

The DRA provider supplies the label-to-Slinky block-topology bridge for this ecosystem. For more background, see:

• Enabling Multi-Node NVLink on Kubernetes for GB200 and Beyond
• Running Large-Scale GPU Workloads on Kubernetes with Slurm
• Running AI Workloads on Rack-Scale Supercomputers: From Hardware to Topology-Aware Scheduling

When to Use This Provider

Use the DRA provider only when all of the following are true:

• You are using Slinky (Slurm-on-Kubernetes) with topology/block
• Every participating node already has a valid value in the configured accelerator-domain label
• Workloads fit within one NVLink partition, or you accept that placement across multiple partitions will not account for backend-fabric locality

The label is deployment- and state-dependent; it is not guaranteed to exist in every MNNVL GPU Operator installation. If it is absent, the provider cannot derive partition membership. If you need the switch hierarchy or topology-aware selection across partitions, use the InfiniBand or NetQ provider instead.

How It Works

The DRA provider does not create its source label. Before selecting this provider, verify that the configured label exists on every participating node. When provider.params.accelerator is omitted, the source label defaults to nvidia.com/gpu.clique for backward compatibility.

Topograph reads these labels from the Kubernetes API:

1. Lists all nodes (filtered by nodeSelector if provided)
2. Uses the shared Kubernetes-label accelerator discoverer to read the configured Node label and group nodes by domain
3. Returns the NVLink domain map as block topology

If no nodes with matching labels are found, Topograph returns a 502 error with a diagnostic message indicating which label and annotations to check.

Prerequisites

• A Slinky (Slurm-on-Kubernetes) cluster configured to use topology/block
• A valid accelerator-domain label already present on every participating Kubernetes node

Parameters

Configuration

Deploy Topograph with the Helm chart and select the DRA provider and Slinky engine through the chart’s provider and engine values. Set the optional source label and nodeSelector under provider.params. The chart manages the Topograph configuration and topology request payload, so you do not need to supply either one separately.

1
provider:
2
  name: dra
3
  params:
4
    accelerator:
5
      source: kubernetes-label
6
      kubernetesLabel:
7
        key: nvidia.com/gpu.clique
8
    nodeSelector:
9
      nvidia.com/gpu.present: "true"

The accelerator object has the same shape as the Kubernetes-label source for infiniband-k8s. DRA supports only that source because it intentionally produces accelerator block domains without discovering a network fabric.

Configure Kubernetes client limits with the chart-wide settings:

1
kubeClient:
2
  qps: 50
3
  burst: 100

The DRA provider performs one filtered Node list per topology generation, so raising its limits is rarely necessary. kubeClient is the only supported Helm configuration surface and applies to both the provider and engine. Tune the Slinky client first when dynamic-node reconciliation is the source of client-side throttling.

Outside Helm, set KUBE_QPS and KUBE_BURST on the Topograph process. Kubernetes client limits are deployment settings and cannot be overridden by a topology request.

No credentials are required. The provider uses the in-cluster service account automatically. See the Slinky engine documentation for the available Helm values.

Verifying the Output

Before triggering topology generation, verify that the configured labels exist on all participating nodes. For the default label:

$
kubectl get nodes -o json | jq '.items[] | {name: .metadata.name, clique: .metadata.labels["nvidia.com/gpu.clique"]}'

If topology generation returns a 502 error, check that the expected nodes have the configured source label and the topograph.run/region / topograph.run/instance annotations (the latter two are set by Topograph itself during topology discovery). For the default label:

$
kubectl get nodes -o json | jq '.items[] | {name: .metadata.name, clique: .metadata.labels["nvidia.com/gpu.clique"], region: .metadata.annotations["topograph.run/region"], instance: .metadata.annotations["topograph.run/instance"]}'

See the Slinky engine documentation for details on topology/block output.