Operations

Switch Management

View as Markdown

See Operations Overview for the conventions (in-band failures, registered vs. ephemeral targets) and the async job model that the asynchronous switch RPCs below build on.

Switch firmware and system images

Switch-only RPCs. NVOS system-image installs run as async jobs; the direct firmware and listing calls are synchronous.

RPCSync/AsyncBehaviorKey inputsKey outputs
ListSwitchFirmwareSyncRead a switch’s firmware inventory for a component.rack_id, node_id, component_typeresult_json
PushSwitchFirmwareSyncUpload a local firmware file to the switch filesystem (transfer only, no install). Source must resolve within firmware_dir.rack_id, node_id, component_type, filename, local_file_pathresult_json
BatchResetSwitchFactoryDefaultAsync → GetJobStatusSubmit a destructive full NVOS factory reset per switch through NVUE REST. RMS retries reset submission with admin/admin only after configured credentials are rejected, then completes when default-credential SSH connects. RMS does not answer or change the first-login password prompt. Per-node results report job admission only.nodes, domain?NodeBatchResponse (parent job_id)
ListSwitchSystemImagesSyncList NVOS system images on a switch.rack_id, node_idimages_json
UpdateSwitchSystemImageAsync → GetSwitchSystemImageJobStatusBatch NVOS image update over an explicit node list: SFTP upload → NVOS install → reboot → steady-state verification, per node under a parent job. Early-exits if already up to date.nodes, image_filename, local_file_path (under firmware_dir)NodeBatchResponse (parent job_id), jobs[]
GetSwitchSystemImageJobStatusSyncPoll UpdateSwitchSystemImage / ApplyStoredSwitchSystemImage / ApplySwitchSystemImage jobs (leaf or parent).job_idstate, message, result_json, timestamps

Scale-up fabric manager

Switch-only RPCs targeting the on-switch NMX controller (nmx-controller) over gRPC and the NVLink switch fabric. Each builds a per-request ephemeral switch from caller-supplied node info. The domain field selects the mTLS cert set (not a DNS domain).

RPCSync/AsyncBehaviorKey inputsKey outputs
ConfigureScaleUpFabricManagerSyncFull single-switch bring-up: enable cluster, enable external gRPC, verify readiness, then issue NMX Hello + SetStaticConfig(topology). Idempotent short-circuit if already configured.node, topology_type, domain?topology_used, scale_up_fabric_state_enabled, grpc_enabled
ConfigureScaleUpFabricManagerV2Async via GetJobStatusSnapshot switch state; select one primary; disable every submitted non-primary switch; reconcile node IP addresses when required; restore and verify the selected switch’s provisioned NMX-C mTLS binding in secure mode, or unset NMX-C mTLS with insecure_switch; reconcile topology and ordered extra static config; then require exactly one enabled primary. Failures after mutation restore snapshotted switch state.nodes, primary_switch_node_id?, config, domain?job_id
BatchSetScaleUpFabricStateSyncConcurrently enable/disable the fabric cluster across switches.nodes, enabledNodeBatchResponse (node_results, stats)
GetScaleUpFabricStateSyncReturn cluster state for one switch.nodestate_json
BatchGetScaleUpFabricServiceStatusSyncPer-node nmx-controller cluster-apps status for a switch set (per-node health in the map, not the top-level status).nodesservice_statuses (map), stats
SetScaleUpFabricTelemetryInterfaceStateSyncEnable/disable the switch’s gNMI telemetry service.node, enableresult_json
BatchResetSwitchSdnFactoryDefaultAsync → GetJobStatusSubmit one destructive SDN factory-reset child job per switch under a parent job. Jobs are intentionally not persisted.nodes, domain?NodeBatchResponse (parent job_id)
GetScaleUpFabricStatusSyncRead observed topology, static configuration, cluster state, and NMX Controller status across submitted switches without mutation.nodes, domain?fabric_status

For caller compatibility, ConfigureScaleUpFabricManagerV2 recognizes temporary NodeDescriptor.attributes entries named fm_config:<key>. RMS maps entries from the selected primary switch to the corresponding fm_config keys. The job ignores these attributes on non-primary switches. If config.extra_static_configs already contains the same file and key, the job fails before switch mutation instead of choosing one value. This prefix is an unsupported compatibility bridge; callers should use config.extra_static_configs when available. If RMS reconciles an fm_config value that was not confirmed already configured, V2 restarts NMX-C before waiting for control-plane convergence. Confirmed unchanged values and changes to other static-config files do not trigger this restart.

Switch certificates and security

Switch-only. Certificate install and password rotation run as async per-switch jobs.

RPCSync/AsyncBehaviorKey inputsKey outputs
ConfigureSwitchCertificateAsync → GetConfigureSwitchCertificateJobStatusPer switch, install switch-side mTLS certificate material (resolved server-side from configured TLS roots - the operator supplies none) and bind selected services (NVUE, NMX, gNMI) to require mTLS. With insecure_switch, instead creates “unset” jobs that move switches back to non-mTLS.nodes, services[], domain?, test_helloNodeBatchResponse (parent job_id), jobs[]
BatchDisableSwitchMtlsAsync via GetJobStatusPer switch, disable mTLS for selected NVUE, NMX, or gNMI services through NVOS CLI commands. This operation does not require insecure_switch or configured TLS material.nodes, services[]NodeBatchResponse (parent job_id)
GetConfigureSwitchCertificateJobStatusSyncPoll a ConfigureSwitchCertificate job (child or parent).job_idstate, message, result_json, timestamps
UpdateSwitchSystemPasswordAsync → GetJobStatusRotate an NVOS system-user password on one or more switches. Per-node results report job admission only. Passwords are kept task-local and scrubbed from all status and logs.username, password, nodesNodeBatchResponse (parent job_id)