Operator guide for tenant creation, resource allocation, and instance provisioning on NICo, using nicocli (REST API CLI) as the primary tool.
This is a Day 1 (Configuration) activity in NICo’s lifecycle model — the phase after hardware has been discovered, validated, and ingested (Day 0). Day 1 is when operators configure tenant boundaries, define resource allocations, and provision instances so tenants can consume bare-metal infrastructure.
The primary tool throughout this guide is nicocli, the CLI client that wraps the NICo REST API. Every REST endpoint is available as a CLI command, and nicocli handles authentication, token refresh, and multi-environment configuration automatically. The carbide-admin-cli (which talks to the Core gRPC API) is referenced only where an operation has no REST API equivalent.
This guide assumes you have completed the Quick Start Guide, which covers NICo deployment, site creation, and host discovery (Day Zero). You should already have:
Registered status, with machines discovered and available for allocation.nicocli installed (make nico-cli from the infra-controller-rest repo) and reachable on $PATH.Note on CLI naming: Older docs reference
carbidecli(built viamake carbide-cli). It’s the same source under a previous name. This guide usesnicocli(built viamake nico-cli) consistently.
For nicocli mechanics and conventions (flag ordering, api.name selection, --data vs flag forms, output formats, pagination, --debug), see the nicocli reference guide. The examples in this guide assume you’ve read it.
NICo’s authorization model has three roles, all managed in the upstream identity provider (any OIDC-compatible IdP, e.g. Keycloak):
A single user can hold roles in multiple orgs simultaneously. On dev/service-account orgs, one user typically holds both Provider Admin and Tenant Admin in the same org.
The Quick Start covers nicocli login end-to-end for Keycloak-backed deployments (the default for setup.sh installs). For other identity providers and for the static-token / token-command flows useful in automation, see the nicocli reference guide.
Confirm that nicocli can reach the API and your credentials are valid:
nicocli user get returns your identity as NICo sees it.
NICo uses a lazy creation model for tenants. There is no explicit “create tenant” API call. Instead, a tenant record is automatically created the first time a Tenant Admin retrieves the current tenant for their organization:
If no tenant exists for the configured org, NICo creates one and returns it. If a tenant already exists, the same command returns the existing record. The operation is idempotent.
Each tenant maps one-to-one to an organization in the configured identity provider. The tenant’s identity (org name, display name) is derived from the IdP’s org metadata — there are no separate fields to supply.
In TUI mode:
api.org).If either condition is not met, the API returns HTTP 403. NICo trusts whatever the IdP says in the token’s claims, so getting these conditions met is an IdP administration task — it is not done through nicocli or the NICo API. The Quick Start Guide walks through the bundled Keycloak reference implementation (a dev Keycloak deployed by setup.sh with a pre-loaded realm), which is the simplest path for first-time setup. For production, point NICo at any OIDC-compatible IdP (Keycloak, Okta, Auth0, your existing enterprise IdP) by configuring the issuers block in carbide-rest-api’s config — see getting-started/installation-options/reference-install.md for the deployment-side wiring.
Check tenant health with the stats endpoint:
Example response for a tenant in active use:
A freshly created tenant shows all zeroes. In TUI mode: tenant stats. Status keys in the response are alphabetical, not lifecycle order.
When GET /v2/org/{org}/nico/tenant/current is called and no tenant exists, the API:
Subsequent calls return the existing tenant. If the org’s display name has changed in the IdP, NICo silently updates it on the next call.
A tenant alone cannot consume infrastructure — it needs a tenant account that links it to an infrastructure provider. This is a provider-side operation requiring the Provider Admin role.
The TUI prompts for the infrastructure provider ID and the tenant org name. Non-interactive (using individual flags):
The tenant account starts in Invited status. Listing tenant accounts requires a filter flag — the bare nicocli tenant-account list returns HTTP 400:
Example tenant account detail (Ready, with active allocations):
accountNumber is auto-generated. allocationCount is a live count of allocations under this account. tenantContact records the user who accepted the invitation (null until accepted, or null when the same org is both provider and tenant).
The tenant admin must accept the invitation to transition the account to Ready. The non-interactive form sends an empty PATCH body — note the flag-first ordering:
TUI:
Only accounts in Invited status can be accepted. Attempting to update a Ready account returns the verified error:
Instance types define hardware classes — they map a named category (like “GB200-NVL72” or “DGX-H100”) to a set of physical machines with specific GPU, CPU, and network configurations. Before a tenant can create compute allocations, the instance types must exist and machines must be associated with them.
Each instance type record includes:
GB200-NVL72)The site administrator defines instance types during Day Zero. NICo validates that machines associated with an instance type have compatible hardware before accepting the association.
The TUI provides richer detail:
The detail view for an instance type includes an allocationStats section showing how many machines are assigned, allocated, and available (maxAllocatable). This tells you the upper bound for a new allocation’s constraint value.
Instance type creation is a gRPC operation via carbide-admin-cli:
See the Quick Start Guide, Step 7 for carbide-admin-cli access patterns. The REST API exposes instance type CRUD endpoints, but machine association management is currently gRPC-only.
An allocation grants a tenant access to infrastructure at a specific site. Without at least one allocation, a tenant cannot create instances, VPCs, or subnets. Allocations are provider-side operations requiring the Provider Admin role.
Each allocation ties together an infrastructure provider, a tenant, and a site, with exactly one allocation constraint that specifies:
The only constraint type currently supported is Reserved, which guarantees the specified capacity. The API validator also accepts OnDemand and Preemptible, but these are not implemented end-to-end.
An allocation starts in Pending status, transitions to Registered once processed, and can move to Error or Deleting. The allocation name must be unique per tenant per site.
Using the TUI (recommended):
The TUI prompts in this order:
InstanceType or IPBlockReservedNon-interactive:
The system validates that enough machines of the specified instance type are available before accepting the allocation.
A constraintValue of 24 allocates a /24 sub-block (256 addresses). The value must be between 1 and 32 and must be greater than or equal to the parent block’s prefix length.
allocation list supports rich filter flags (verified via --help): --site-id, --tenant-id, --infrastructure-provider-id, --resource-type (InstanceType or IPBlock), --resource-type-id, --status, --constraint-type, --constraint-value. The --query flag is a free-text search over name/description/status, NOT a key-value filter — use the dedicated flags instead.
TUI equivalents: allocation list, allocation get. The list-table output prints a one-line pagination summary on stderr (e.g. Page 1/18 (5 items, 88 total). Use --all to fetch everything.); --all follows pagination for you.
Example allocation detail (IPBlock allocation, /28 reservation against a /16 pool):
derivedResourceId is the ID of the sub-resource carved from the parent (the actual sub-block for IP allocations). ResourceTypeID is mixed-case in the response — field name is what the JSON API returns.
Adjust an existing constraint value (e.g., increase a machine quota). Note the flag-first ordering with two positionals:
Or with --data:
The system validates:
Deletion is blocked if the tenant has active instances or subnets consuming resources from the allocation. Terminate dependent resources first.
A tenant can have multiple allocations at the same site with different resource types or instance types. The aggregate compute quota for a given instance type is the sum of all allocation constraints for that type across the tenant’s allocations at that site.
nicocli tenant get-current-tenantRegistered statusAfter allocations are in place, the tenant can create VPCs and subnets within their allocated resource boundaries.
A VPC is the logical network container for tenant workloads. It defines the tenant boundary for networking and provides the parent context for subnets and instances.
nicocli vpc create --help shows these flags:
Realistic non-interactive form:
TUI flow (prompts in order):
Verify the VPC reaches Ready status:
Routing profiles govern which VPCs can exchange routes with which others. The REST API accepts
external,internal, andprivileged-internal; the underlying gRPC API supports any profile defined underfnn.routing_profilesin the API server config. For details, see VPC Routing Profiles. For the full networking architecture (VRFs, VNI pools, BGP, deny prefixes), see VPC Network Virtualization.
A subnet is an IP address range within a VPC, carved from an allocated IP block.
nicocli subnet create --help shows these flags. Note the flag name --ipv4block-id (no separator between v4 and block) and the IPv6 counterpart:
Non-interactive (IPv4):
Or via --data (the JSON body uses camelCase even though the flag is single-word):
TUI flow:
Verify:
An instance in NICo is a bare-metal machine assigned to a tenant within a VPC. Creating an instance claims a machine from the tenant’s compute allocation, associates it with a VPC, and triggers the provisioning workflow (OS installation, network configuration, security lockdown).
nicocli instance create --help shows these flags:
interfaces[] and sshKeyGroupIds[] are array-typed and must go through --data / --data-file.
Non-interactive form (with one interface and one SSH key group):
If you want to target a specific machine instead, replace instanceTypeId with machineId. Machine targeting requires the tenant to have capabilities.targetedInstanceCreation: true.
TUI flow:
Ready state at the VPC’s siteAfter creation, the instance goes through these states:
Two states are easy to miss:
BootCompleted — follows Ready once the OS phones home. Instances without phone-home enabled stop at Ready.Configuring — the instance transitions through this state when its config is being updated in place (e.g. attaching a new Network Security Group, rotating SSH key groups, swapping the OS image, or pushing new user-data). Configuring does NOT trigger a reboot on its own; the instance returns to Ready once the change is applied. A reboot caused by --trigger-reboot=true goes through Rebooting, not Configuring.Monitor progress:
The instance detail response is rich — it includes interfaces[] with assigned IP addresses and VPC prefix info, ipxeScript showing the live boot script, serialConsoleUrl for console access, full machine and SKU metadata, and any active deprecations[] warnings. The API uses inline deprecations[] arrays to flag fields scheduled for removal — watch for these in your responses.
For creating multiple identical instances at once, use instance batch-create. Unlike create, batch-create takes a single shared spec plus a count — it provisions N instances with auto-generated names from the same instance type, tenant, and VPC. nicocli instance batch-create --help shows:
Non-interactive form:
interfaces[] and sshKeyGroupIds[] are still array-typed and must go through --data / --data-file if you need to attach them at create time.
For batches where each instance needs a different machine ID, OS, or interface set, call instance create in a loop instead — batch-create only handles the homogeneous case.
NICo provides instance-level power management through the REST API. These operations send commands to the underlying BMC via Redfish.
nicocli instance update --help exposes individual flags for every common operation — prefer them over --data:
Reboot:
Reboot with re-provisioning iPXE and pending updates:
TUI:
The TUI prompts for instance, custom-iPXE flag, apply-updates flag, and a confirmation.
sshKeyGroupIds[] is an array, so changes go through the body:
In TUI mode, instance delete prompts for confirmation before proceeding. Deletion triggers the full sanitization workflow: secure erase of NVMe storage, GPU and system memory wipe, TPM reset, re-attestation, and network isolation teardown. The machine returns to the available pool once sanitization completes.
For stuck or unresponsive machines that cannot be managed through the instance API, carbide-admin-cli provides direct BMC operations:
See the Machine Reboot and Force Delete playbooks in the core documentation for detailed procedures.
For provider admins needing visibility across tenants, list tenant accounts (a filter flag is required):
Non-zero error counts warrant investigation:
Provider admins can get cross-tenant compute allocation stats at a site. instance-type-stats is a sub-resource of tenant, with a stats leaf action — the full command has three tokens:
A bare nicocli tenant instance-type-stats --site-id <id> returns flag provided but not defined: -site-id — the trailing stats is required.
NICo has no first-class “disable” operation. Options:
TENANT_ADMIN from all users. Existing resources remain but cannot be managed.There is no DELETE /tenant endpoint — tenant records are permanent. To fully decommission:
Terminated status.After this sequence, the tenant record still exists but is inert.
This teardown is destructive and irreversible at the resource level. Terminated instances cannot be recovered. Always confirm with the tenant team before beginning.
This section ties together the full Day One workflow. The TUI flow is the recommended path for first-time operators — it scopes lookups (instance types to sites, VPC prefixes to VPCs, etc.) and you can’t easily get the order wrong.
Idempotent — creates the tenant lazily on first call.
Or via TUI: tenant-account create.
Use the TUI for the first one — it filters instance types by the selected site and validates capacity:
All allocations should show Registered status.
The first instance is easiest via TUI because interfaces[] is array-typed:
For automation, use --data-file — see the Launching an Instance section above.
The instance should reach Ready (or BootCompleted if phoneHomeEnabled: true). Tenant stats should now show non-zero counts for instance, vpc, and subnet.
Use --debug on any command to see the full HTTP request and response. The token is redacted in the log; the path-rewriting from nico to whatever api.name is set to is visible. Real output:
The CLI version and the API server version are independent. CLI is generated from the OpenAPI spec at the time of build; the server reports its own image version (visible in audit response apiVersion field). Different versions are expected — the wire protocol is stable enough that mismatches rarely matter.
The list view is intentionally lightweight — each entry has id, endpoint, method, statusCode, userId, clientIP, apiVersion, and timestamp. To see the full request, including the request body and the resolved user object, fetch a single entry:
audit get adds body, queryParams, extraData, durationMs, statusMessage, and the resolved user (with email and name when the caller was a human, blanks when the caller was a service account). The table form of audit list only shows id — always use --output json (or audit get) for anything more than ID discovery.
The TUI is the recommended tool for exploratory work. It handles config selection, authentication, and provides tab-complete interactive commands:
The TUI discovers all config*.yaml files in ~/.nico/ and lets you pick an environment at startup. Commands like allocation create scope lookups to the relevant site automatically, reducing the chance of selecting a resource from the wrong site.
Flag-first ordering — always put flags before positional args.