NVLink Partitioning

NVIDIA NVLink is a high-speed interconnect technology that allows for memory-sharing between GPUs. Sharing is allowed between all GPUs in an NVLink Partition. An NVLink Partition must consist of GPUs within the same NVLink Domain, which can be a single NVL72 rack or two NVL36 racks cabled together.

NVIDIA Infra Controller (NICo) allows you to do the following with NVLink:

• Create, update, and delete NVLink Logical Partitions using the NICo REST API.
• Provision Instances with GPUs assigned to partitions within NVLink Domains by way of NVLink Logical Partitions, without knowledge of the underlying NVLink topology.
• Update Instances to change NVLink Logical Partition assignment for its GPUs

NICo extends the concept of an NVLink Partition with the NVLink Logical Partition, which allows users to manage NVLink Partitions without having to learn the datacenter topology.

Operations: Who Does What

NVLink operations are split between operator site setup against NMX-C and tenant partition management. Notably, one operator step (fallback NMX-C endpoint registration) is not exposed via the REST API and is therefore driven through nico-admin-cli over gRPC. See Network Isolation → Who configures what, and how for the role and interface model.

The operator NMX setup (the first two rows) is detailed under Enabling NMX-C-based NVLink Partitioning. Certificate lifecycle setup and alerting are described in NMX-C Certificate Monitoring and Rotation. The tenant rows are the REST / nicocli flow described next.

Creating a NVLink Logical Partition

NICo users can create NVLink Logical Partitions and plan GPU assignments using NVLink Interfaces for Instances (as described in steps 1-2). NICo can also automatically generate NVLink Interfaces and assign them to Instances (as described in step 3).

In general, the steps are:

1. The user creates a NVLink Logical Partition using the POST /v2/org/{org}/nico/nvlink-logical-partition REST API endpoint. NICo creates an entry in the database and returns an NVLink Logical Partition ID. At this point, there is no underlying NVLink Partition associated with the NVLink Logical Partition.

2. When creating an Instance, the user specifies NVLink Interface configuration for each GPU by referencing their preferred NVLink Logical Partition ID in the POST /v2/org/{org}/nico/instance REST API endpoint request.

   a. If this is the first Instance to be added to specified NVLink Logical Partitions, NICo Core will create and assign NVLink Partitions for them and add the Instance GPUs to the NVLink Partitions.

   To ensure that machines in the same Rack are assigned to the same NVLink Partition, an Instance Type can be created for the Rack and all Machines in the Rack assigned to the same Instance Type. Alternatively users can use the Batch Instance creation REST API endpoint and set topologyOptimized to true.

3. If the user does not want to specify NVLink Interfaces for each GPU when creating an Instance, they can:

   a. Create a new VPC by specifying a value for nvLinkLogicalPartitionId or update an existing VPC with no Instances to set the nvLinkLogicalPartitionId attribute. We will refer to this as the default NVLink Logical Partition for the VPC.

   b. When creating an Instance in this VPC, user does not need to specify NVLink Interfaces, NICo will automatically generate NVLink Interfaces for the Instance and assign them to the VPC’s NVLink Logical Partition.

   c. All Instances created within this VPC will have their GPUs assigned to the same NVLink Partition as long as the Instances end up in the same Rack.

   d. If there is no space in the Rack where the NVLink Partition for an NVLink Logical Partition is located, NICo will create a new NVLink Partition in a different Rack for the same NVLink Logical Partition and continue to assign the Instance GPUs to it.

Updating an Instance to change NVLink Logical Partition assignment for its GPUs

If a NICo user wants to update an Instance to change NVLink Logical Partition assignment for its GPUs, they can do so by calling the PATCH /v2/org/{org}/nico/instance/{instance-id} REST API endpoint.

The user can specify the NVLink Logical Partition ID for each GPU in the Instance by passing the nvLinkInterfaces list.

If Instance’s VPC has a default NVLink Logical Partition, no changes to the NVLink Logical Partition assignment are allowed.

Removing Instances from a Logical Partition

If a user de-provisions an Instance, NICo will remove the Instance GPUs from the NVLink Partition.

Deleting an NVLink Logical Partition

A NICo user can call DELETE /v2/org/{org}/nico/nvlink-logical-partition/{nvLinkLogicalPartitionId} to delete an NVLink Logical Partition. This call will only succeed if there are no active Instances associated with the NVLink Logical Partition.

Retrieving NVLink Partition Information for an Instance

A NICo user can call GET /v2/org/{org}/nico/instance/{instance-id} to retrieve information about an Instance. As part of the 200 response body, NICo will return a nvLinkInterfaces list that includes both the nvLinkLogicalPartitionId and nvLinkDomainId for each GPU in the Instance.

Default NVLink Logical Partition for a VPC

It’s an optional default, not a constraint. VPCs can be created with or without a default NVLink Logical Partition.

It is optional on both POST .../vpc (Create VPC) and PATCH .../vpc/{vpcId} (Update VPC).

What setting it on a VPC actually does It’s a convenience default for instance creation. When nvLinkLogicalPartitionId is set on the VPC, you don’t have to specify nvLinkInterfaces on POST .../instance (Create Instance) or POST .../instance/batch (Batch Create Instances) — the API will auto-populate the per-GPU NVLink Interfaces to reference that VPC’s NVLink Logical Partition. That’s the entire effect. It does not reserve or lock the NVLink Logical Partition to the VPC.

No exclusivity between VPCs We intentionally don’t restrict an NVLink Logical Partition to a single VPC. The same nvLinkLogicalPartitionId may be set on multiple VPCs. This is deliberate, to preserve flexibility in how you plan networking and NVLink partitioning.

You can also manage NVLink at the Instance level If you want finer control, leave nvLinkLogicalPartitionId unset on the VPC and specify nvLinkInterfaces directly on Create Instance — each entry binds a specific deviceInstance (GPU) to an explicit nvLinkLogicalPartitionId, so different GPUs in the same instance (or across Instances in the same VPC) can operate in different NVLink Logical Partitions.

Summary

How NICo Reconciles NVLink State

NICo runs a periodic monitor against NMX-C to keep the actual NVLink partition topology aligned with the desired state implied by tenant instance configurations. The behaviour matters whenever an operator is diagnosing latency between an API call and an instance becoming Ready.

Each reconciliation pass does the following:

1. Loads every NVLink Logical Partition and every NVLink Physical Partition from the NICo database.
2. Resolves the NMX-C endpoint for each chassis and queries its current partition list, compute nodes, and GPU membership.
3. Updates the per-host NvlinkUnhealthy health report when the GPU-health query succeeds. Its PreventAllocations alerts block normal allocation while they remain in aggregate host health. See the linked probe contract for recovery, hardware replacement, manual removal, and allocation exceptions.
4. Compares observed state against desired state.
5. Issues create / update / remove operations to the fabric-management service to converge it onto desired state.
6. Updates per-machine GPU status observations in the NICo database. The per-instance configs_synced.nvlink field is derived from these observations and is what gates the instance’s Ready state.

Cadence is set by nvlink_config.monitor_run_interval (default 60s). The idle time before the next pass depends on how long a pass took and on whether it applied changes, checked in this order:

1. A pass that ran longer than the interval idles a quarter of the interval, even when it applied changes, so the monitor never runs back to back. The pass counts into carbide_nvlink_partition_monitor_iteration_overruns_total and logs a warning.
2. A pass within the interval that applied changes is rechecked after one second, or after the interval when that is shorter.
3. Any other pass idles for the remainder of the interval, but at least a quarter of it.

With the default interval, a 61 s pass that applied changes idles 15 s rather than one second. The partition owners accept a couple of minutes of staleness, so a site whose passes overrun at 60s can set 120s. On 556 simulated racks a pass took 27 to 33 s. Chassis and racks are reconciled in parallel, at most nvlink_config.partition_monitor_max_concurrent_groups (default 16, documented range 1 to 256) at a time.

Not every chassis or rack is contacted on every pass. A group is skipped when its database inputs (hosts, instances and their NVLink configs, GPU inventory, admin-network state, physical partitions in its domain, and the deleted Logical Partitions those physical partitions belong to) are unchanged since its last successful check, every instance in it has its NVLink config applied, and its full check is not due. Full checks run every nvlink_config.partition_monitor_full_check_interval (default 10m), staggered per group so they spread evenly over the interval, and catch changes made on the NMX-C side. A group whose check failed, or that applied changes, is checked again on the next pass. A group whose GPU health could not be fetched, or whose health report write failed, is also checked again on the next pass. Setting the interval to 0s disables skipping. While a group is skipped its per-domain nmxc_partition_count, nmxc_gpu_count, and nmxc_compute_node_count gauges keep the values of its last full check, and machine_status_updates_count counts only machines in the groups that were checked. GPU health is read only during a check, so on a skipped group the health labels of nmxc_gpu_count and the NvlinkUnhealthy host alert that blocks allocations refresh at the full-check interval, up to 10m by default. Lower partition_monitor_full_check_interval, or set it to 0s, where faster health detection matters more than NMX-C load. The switches and power shelves of a skipped rack are checked for a missing or different domain UUID, and the UUID observed at the rack’s last full check is republished only when one is found, so a switch or power shelf added to the rack since then receives it on the next pass.

Metrics

The monitor exposes metrics under the carbide_nvlink_partition_monitor_* namespace. Useful ones:

Instance Release and Logical Partition Deletion

When an instance is released (via ReleaseInstance):

1. The instance’s NVLink configuration is cleared from the database.
2. The monitor observes that GPUs previously assigned to the instance are no longer requested in any live partition.
3. The monitor removes those GPUs from their NMX-C partitions.
4. Once all NVLink state is removed, the machine’s GPU status observation reflects an empty domain assignment and the host becomes eligible for reuse.

When a Logical Partition is deleted, every underlying NVLink Physical Partition on each NMX-C endpoint backing it is also deleted. The deletion is rejected if any instance still references the Logical Partition.

When a host is force-deleted, the instance running on it is implicitly released and the above cleanup path runs. Operators do not need to detach NVLink configuration manually before force-deleting.

Enabling NMX-C-based NVLink Partitioning

NMX-C is the supported gRPC control path for NVLink partition management.

Configure the NMX-C client under [nvlink_config]:

[nvlink_config]
enabled = true
monitor_run_interval = "60s"
# TLS material for NMX-C. The CA path is required by the certificate
# monitor. Configure the client certificate and key together when NMX-C
# requires mTLS.
nmx_c_tls_ca_cert_path     = "/etc/nico/nmxc/ca.pem"
nmx_c_tls_client_cert_path = "/etc/nico/nmxc/client.crt"
nmx_c_tls_client_key_path  = "/etc/nico/nmxc/client.key"
nmx_c_tls_authority        = "nmxc.example.internal"
nmx_c_endpoint_port        = 9370
allow_insecure = false

Rack-associated machines use the NVOS IP of a ready Fabric Manager control-plane switch in the rack. Machines without a rack association use an explicit mapping stored in the database, keyed by chassis serial:

nico-admin-cli nvlink-nmxc-endpoints create \
    --chassis-serial <serial> \
    --endpoint https://nmxc-host:443
nico-admin-cli nvlink-nmxc-endpoints show

update and delete subcommands follow the same pattern. The partition monitor picks up new mappings for unassociated machines on the next iteration; no restart is required. Separately, the certificate monitor can use a chassis mapping as an alert-only fallback when no ready rack endpoint exists. It cannot request automatic rotation for that target because the mapping does not identify an actionable rack.

The TLS material in TOML applies uniformly to every NMX-C endpoint NICo talks to. Per-endpoint credential overrides are not currently supported; deploy a uniform trust posture across the site’s NMX-C control plane.

The machine → NMX-C GPU mapping (nvlink_info) needs no operator step. Machines discovered after NVLink partitioning is enabled are populated during discovery, and the NVLink partition manager populates or repairs the mapping for any managed machine whose nvlink_info is missing or deviates from what NMX-C reports. nico-admin-cli machine nvlink-info populate is retained only as a deprecated compatibility command and always returns an error; inspect the stored mapping with nico-admin-cli machine nvlink-info show <machine-id>.

To monitor the certificate presented by those endpoints and request rotation before expiry, follow NMX-C Certificate Monitoring and Rotation.

Verifying a Tenant’s NVLink Placement

After an instance has been created and the monitor has had at least one opportunity to run, you can confirm correct placement with the following checks. There is no single all-in-one health command; the steps below should be repeatable as a checklist.

1. Monitor is running. carbide_nvlink_partition_monitor_iteration_latency_milliseconds is being recorded. Evaluate the recent increase or rate of carbide_nvlink_partition_monitor_nmxc_connect_error_count over a time window appropriate to the reconcile interval; its historical total alone does not indicate a current connection problem.

2. Logical-partition count matches expectation. carbide_nvlink_partition_monitor_num_logical_partitions reflects the partitions a site planner expects to exist. A sudden change is worth correlating with recent tenant API activity.

3. Per-instance configuration has converged. The instance’s InstanceStatus reports configs_synced.nvlink = true and the nvLinkInterfaces list on the instance shows the expected nvLinkLogicalPartitionId and nvLinkDomainId for each GPU.

4. Per-machine GPU placement.

   nicocli machine nvlink-info --machine-id <machine-id>

   Returns the machine’s NVLink GPU status observations, including the Domain each GPU is currently assigned to. Use this to confirm that two instances expected to share an NVLink Logical Partition have actually landed in the same NVLink Domain — instances in different Domains cannot share GPU memory regardless of having the same Logical Partition ID.

5. Cleanup after release. After releasing an instance, the same machine nvlink-info output should show an empty Domain assignment on the affected GPUs within one or two reconcile intervals. Failure to clear indicates the monitor could not remove the GPU from its NMX-C partition; investigate the corresponding connect-error and op-latency metrics.