For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
GitHub
DocumentationREST API Reference
DocumentationREST API Reference
    • Home
  • Overview
    • What is NICo?
    • Key Capabilities
    • Operational Principles
    • Day 0 / Day 1 / Day 2 Lifecycle
    • Scope and Boundaries
  • Getting Started
    • Building NICo Containers
    • Quick Start Guide
  • Provisioning (Day 0 Operations)
    • Ingesting Hosts
    • Ingesting Hosts (REST API)
    • Host Validation
    • SKU Validation
  • DPU Management
    • DPU Lifecycle Management
    • DPU Configuration
    • BlueField DPU Operations
  • Architecture
    • Overview and Components
    • Redfish Workflow
    • Redfish Endpoints Reference
    • Reliable State Handling
    • Networking Integrations
    • Health Checks and Health Aggregation
    • Health Probe IDs
    • Health Alert Classifications
    • Key Group Synchronization
  • Operations
    • NVLink Partitioning
    • Release Instance API Enhancements
    • IP Resource Pools
    • BGP Peering
  • Playbooks
    • Azure OIDC for Infra Controller Web UI
    • Force Deleting and Rebuilding Hosts
    • Rebooting a Machine
    • InfiniBand Setup
  • Development
    • Codebase Overview
    • Bootable Artifacts
    • Local Development
    • Running a PXE Client in a VM
    • TLS and SPIFFE Certificates
    • SPIFFE and casbin policies with admin-cli
    • Re-creating Issuer/CA in Local Dev
    • Visual Studio Code Remote Development
    • Adding Support for New Hardware
    • Build Guide
  • Reference
    • Hardware Compatibility List
    • Release Notes
    • FAQs
    • Glossary
GitHub
NVIDIANVIDIA
Developer-friendly docs for your API
Privacy Policy | Manage My Privacy | Do Not Sell or Share My Data | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2026, NVIDIA Corporation.

LogoLogo
On this page
  • Before You Start
  • Registering Expected Machines
  • Single Machine
  • Batch (Recommended for Full-Rack Onboarding)
  • What Happens After Approval: Ingestion to Ready
  • Verifying Ingestion
  • What’s Next
  • Troubleshooting
  • Inspecting Machine State
  • Endpoint Exploration Errors
  • Common Blockers During Host + DPU Pairing
  • Managing the Expected Machines Table
  • Listing and Filtering
  • Single-Entry Operations
  • Batch Update
  • Export
Provisioning (Day 0 Operations)

Ingesting Hosts (REST API)

||View as Markdown|
Previous

Ingesting Hosts

Next

Host Validation

Provider-side hardware onboarding for NICo using the REST API and nicocli. This is the Day 0 prequel to the Typical API Call Flows in the REST API Reference — it covers how physical machines become discoverable so that subsequent calls like Retrieve All Machines return real hardware to allocate and provision.

Before You Start

Make sure the following are in place before you begin:

  1. NICo is deployed and the REST API service is reachable at a known URL.
  2. You have nicocli installed (make nico-cli from the infra-controller repo) and a working config under ~/.nico/. For setup, authentication, and config conventions, see the Quick Start Guide and the nicocli reference guide.
  3. You hold the PROVIDER_ADMIN role in the org you are operating in. Tenant Admins with targetedInstanceCreation capability can also register Expected Machines, but the canonical path is provider-side.
  4. DHCP requests from all managed host BMC networks have been forwarded to the NICo DHCP service.
  5. For every host you plan to register, you have:
    • The MAC address of the host BMC
    • The chassis serial number
    • The host BMC factory default username and password

Verify connectivity:

nicocli site list
nicocli user get

nicocli site list returns the Sites the calling org has access to. Pick the Site UUID for the data center you are onboarding hardware into — you will pass it as --site-id in every Expected Machine create call.

Registering Expected Machines

An Expected Machine pre-registers a physical machine so NICo can authenticate to it on discovery and accept it for ingestion. Each Expected Machine carries the factory default BMC credentials NICo uses for first contact, plus identifying information (chassis serial, optional rack/SKU metadata).

The Expected Machine endpoints are scoped per-org per-site. All requests require PROVIDER_ADMIN (or TENANT_ADMIN with targetedInstanceCreation).

Single Machine

Create one Expected Machine with explicit flags:

$nicocli expected-machine create \
> --site-id <site-uuid> \
> --bmc-mac-address <mac> \
> --chassis-serial-number <chassis-serial> \
> --default-bmc-username <bmc-user> \
> --default-bmc-password <bmc-password>

Required flags: --site-id, --bmc-mac-address, --chassis-serial-number. The BMC credentials are optional on the REST schema but required in practice — without them NICo cannot authenticate to the BMC for discovery.

Optional flags add metadata or pre-allocate resources:

FlagPurpose
--bmc-ip-addressPre-allocate a reserved BMC IP (IPv4 or IPv6) instead of letting DHCP assign one
--rack-idAssociate with a rack identifier
--sku-idAssociate with a SKU
--manufacturer, --model, --firmware-version, --name, --descriptionFree-text hardware metadata
--slot-id, --tray-idx, --host-idPhysical placement within a rack/tray

You can also pass the entire request body as JSON:

$nicocli expected-machine create --data-file - <<'EOF'
${
> "siteId": "<site-uuid>",
> "bmcMacAddress": "<mac>",
> "defaultBmcUsername": "<bmc-user>",
> "defaultBmcPassword": "<bmc-password>",
> "chassisSerialNumber": "<chassis-serial>",
> "fallbackDPUSerialNumbers": ["<dpu-serial-1>", "<dpu-serial-2>"],
> "labels": {
> "environment": "production",
> "rack": "A1"
> }
>}
$EOF

fallbackDPUSerialNumbers is JSON-only (no flag form) and is needed for DGX-H100 or other machines where the NetworkAdapter serial number is not available in the host Redfish.

Batch (Recommended for Full-Rack Onboarding)

For multiple machines, prefer batch-create. The endpoint accepts up to 100 Expected Machines per request and validates the whole batch atomically:

$nicocli expected-machine batch-create --data-file expected-machines.json

Where expected-machines.json is a JSON array of ExpectedMachineCreateRequest objects:

1[
2 {
3 "siteId": "<site-uuid>",
4 "bmcMacAddress": "<mac-1>",
5 "defaultBmcUsername": "<bmc-user>",
6 "defaultBmcPassword": "<bmc-password-1>",
7 "chassisSerialNumber": "<chassis-serial-1>",
8 "rackId": "rack-01"
9 },
10 {
11 "siteId": "<site-uuid>",
12 "bmcMacAddress": "<mac-2>",
13 "defaultBmcUsername": "<bmc-user>",
14 "defaultBmcPassword": "<bmc-password-2>",
15 "chassisSerialNumber": "<chassis-serial-2>",
16 "rackId": "rack-01"
17 }
18]

Two constraints apply to every batch request, both enforced at the API gateway:

  • Maximum 100 machines per request. For sites with more than 100 machines, send multiple batch-create calls.
  • All machines in one request must share the same siteId. You cannot mix sites within a single batch.

What Happens After Approval: Ingestion to Ready

Once Expected Machines are registered and the trust policy is in place, NICo’s Site Explorer automatically discovers and ingests each machine. No further operator action is required under normal circumstances.

The high-level flow:

  1. DHCP discovery: the host BMC sends a DHCP request; NICo assigns an IP and Site Explorer probes the BMC over Redfish using the factory default credentials from the Expected Machine, then rotates the BMC password to the site-wide credential. See Redfish Workflow.
  2. Preingestion: NICo runs a preingestion state machine against each discovered BMC endpoint (host and DPU). It checks BMC clock drift against site time, resetting the BMC if needed. For host endpoints, firmware components are upgraded to the minimum version required for ingestion.
  3. DPU-host pairing: Site Explorer correlates host and DPU serial numbers to form matched pairs. Once DPUs are validated and paired, the ManagedHost object is created and the state machine starts.
  4. DpuDiscoveringState / DPUInit: NICo configures Secure Boot on the DPU, installs the DPU OS (BFB image), and power-cycles the host to apply the new DPU configuration.
  5. HostInit: NICo configures BIOS, sets the host boot order, optionally collects TPM attestation measurements, waits for hardware discovery via the scout agent, and applies UEFI lockdown. When scout reports back, NICo replaces the temporary predicted host ID (prefix fm100p) with a stable host ID (prefix fm100h) derived from the host’s DMI serial data or TPM certificate.
  6. BomValidating / Validation: NICo validates discovered hardware against the expected SKU. If hardware validation is enabled, the host is rebooted and tested before proceeding.
  7. Ready: the host transitions through HostInit/Discovered and enters the available pool, ready for an instance to be assigned.

For the full DPU lifecycle, see DPU Lifecycle Management. For the complete state transitions, see Managed Host State Diagrams.

Verifying Ingestion

Once machines reach Ready, they show up in the Machine REST endpoint. List all machines on a site:

nicocli machine list --output table

Inspect a single machine:

nicocli machine get <machine-id>

Include hardware metadata (CPU, memory, interfaces, etc.):

nicocli machine get <machine-id> --include-metadata

A Ready machine has status: Ready and isUsableByTenant: true. Once at least one machine is Ready, you can continue with the rest of the Provider or Service Account flow.

What’s Next

With machines ingested and Ready, follow the relevant API flow in the REST API Getting Started reference:

  • Service Account: create Network Allocations against each Site IP Block, create a VPC, create a VPC Prefix or Subnet, create an Operating System, create an Instance.
  • Provider: create Tenant Accounts, create Instance Types, associate machines with Instance Types, create Compute and Network Allocations for tenants.

Both flows assume hardware ingestion is complete — which this page covers.

Troubleshooting

Inspecting Machine State

When a machine is not being created or is stuck in a pre-Ready state, start with the Machine REST endpoint:

nicocli machine list --output table
nicocli machine get <machine-id>

For deeper investigation, carbide-api logs filtered by the host BMC IP or DPU BMC IP are the fastest way to understand where ingestion or pairing is failing.

Endpoint Exploration Errors

Before pairing can occur, Site Explorer must successfully explore each BMC endpoint. Exploration failures are logged in carbide-api and the NICo Grafana dashboard. Common error types:

Error typeLikely cause
ConnectionTimeoutBMC unreachable on the OOB network; check cabling and DHCP routing
ConnectionRefusedNo Redfish API exposed at the target IP; the DPU admin IP is often mistakenly probed here
Unauthorized / AvoidLockoutBMC credentials do not match the Expected Machine entry or site vault; see Adding New Machines: BMC Password Requirements
MissingCredentialsCredentials not yet available in vault; check that site-wide BMC credentials are configured
UnsupportedVendorBMC vendor is not supported by this version of NICo
RedfishErrorUnexpected Redfish response; check BMC firmware version and carbide-api logs for the full response body
InvalidDpuRedfishBiosResponseDPU BIOS endpoint returned an unexpected response; the DPU may need a fresh OS install

For a complete reference of all Redfish endpoints and required response fields, see Redfish Endpoints Reference.

Common Blockers During Host + DPU Pairing

The following are the conditions in which Site Explorer cannot complete pairing and logs a host_dpu_pairing_blockers_count metric. Each requires operator investigation.

Metric labelDescriptionAction
dpu_nic_mode_unknownDPU mode cannot be determined; DPU BMC firmware is likely too oldInstall a fresh DPU OS (which also upgrades firmware)
dpu_pf0_mac_missingDPU is in DPU mode but its pf0 MAC address is not retrievableInstall a fresh DPU OS
manual_power_cycle_requiredDPU mode was changed but the host vendor does not support automated power cyclingManually power-cycle the host at the data center level
host_system_report_missingHost BMC Redfish returned no valid system report; likely a BMC firmware issue or transient errorCheck carbide-api logs for the host BMC IP
no_dpu_reported_by_hostHost BMC reports no BlueField PCIe devicesCheck DPU seating and host BMC firmware version
boot_interface_mac_mismatchHost boot MAC does not match the pf0 MAC of any discovered DPUCheck exploration reports and carbide-api logs for both host and DPU BMC IPs
viking_cpld_version_issueNVIDIA Viking (DGX): CPLDMB_0 firmware below minimum required version (0.2.1.9)Contact the data center team for a full DC power cycle

For more DPU-specific troubleshooting (Secure Boot configuration, BMC password resets, firmware version checks), see Adding New Machines to an Existing Site.

Managing the Expected Machines Table

Listing and Filtering

nicocli expected-machine list --output table
nicocli expected-machine list --site-id <site-uuid> --output table
nicocli expected-machine list --include-relation Site --output table

Pagination is on by default (--page-number, --page-size); use --all to fetch every page. Sort order is controlled by --order-by (see --help for the supported enum values).

Single-Entry Operations

nicocli expected-machine get <expected-machine-id>
nicocli expected-machine update <expected-machine-id> --rack-id rack-02
nicocli expected-machine delete <expected-machine-id>

update accepts the same flags as create (all optional on update). For complex updates, use --data-file with the full ExpectedMachineUpdateRequest JSON.

Batch Update

nicocli expected-machine batch-update --data-file updates.json

Where updates.json is a JSON array of ExpectedMachineUpdateRequest objects. Each entry must include the id field identifying which Expected Machine to update; other fields are merged onto the existing record.

Export

To dump the current table as JSON:

nicocli expected-machine list --all --output json

Suitable for backup or import-into-another-site workflows.