Skip to navigation

Scale-Up Fabric

View as MarkdownOpen in Claude

Configure and inspect the scale-up fabric manager on switches.

RPCExecutionSummary
ConfigureScaleUpFabricManagerSynchronousConfigureScaleUpFabricManager performs complete ScaleUpFabricManager configuration on a switch.
GetScaleUpFabricStatusSynchronousGetScaleUpFabricStatus synchronously inspects the fabric without changing anything and returns the observed scale-up configuration: per-switch state, topology, and static configuration reported by the fabric manager.
InspectScaleUpFabricsSynchronousInspectScaleUpFabrics discovers enabled controllers and reports their live compute and switch members, control-plane state, and sampled component status.
BatchResetSwitchSdnFactoryDefaultAsynchronous, parent and child jobsBatchResetSwitchSdnFactoryDefault submits asynchronous SDN factory-default resets.
BatchSetScaleUpFabricStateSynchronousBatchSetScaleUpFabricState synchronously sets ScaleUpFabric state and saves config across all supplied switches.
BatchGetScaleUpFabricServiceStatusSynchronousBatchGetScaleUpFabricServiceStatus retrieves cluster health and fabric-manager status for a caller-supplied list of switches using switch Rest APIs.
GetScaleUpFabricStateSynchronousGetScaleUpFabricState retrieves the cluster state for a caller-supplied switch using switch APIs.
SetScaleUpFabricTelemetryInterfaceStateSynchronousSetScaleUpFabricTelemetryInterfaceState enables or disables the FabricTelemetryInterface service.
ConfigureScaleUpFabricManager (V2)Asynchronous, one jobConfigureScaleUpFabricManager declaratively configures the scale-up fabric across all supplied switches.

For definitions of execution modes and job polling, refer to Async jobs.

ConfigureScaleUpFabricManager

rack_manager.RackManager/ConfigureScaleUpFabricManager

ConfigureScaleUpFabricManager performs complete ScaleUpFabricManager configuration on a switch. This includes enabling the scale fabric services services,configuring gRPC access,and sets multi-node scaleup topology based on the hardware platform. Credentials are resolved from the node’s stored inventory data.

This is the single-switch version. For declarative configuration of a whole fabric, use RackManagerV2/ConfigureScaleUpFabricManager.

PropertyValue
ExecutionSynchronous. The response carries the full result.
TargetCaller-supplied nodes. The request carries each node’s endpoints and credentials, and RMS does not store them.
Behavior detailsOperations guide
Protobuf referenceConfigureScaleUpFabricManager

Request: ConfigureScaleUpFabricManagerRequest

FieldTypeDescription
nodeNodeInfoTarget switch identity and direct connection information
topology_typestringTopology to be set for ScaleUpFabric
domainoptional stringOptional site selector; defaults to —default-switch-domain when unset.

Response: ConfigureScaleUpFabricManagerResponse

FieldTypeDescription
statusReturnCodeRETURN_CODE_SUCCESS if configuration completed, RETURN_CODE_FAILURE otherwise
messagestringHuman-readable status or error message
topology_usedstringmulti-node scaleup topology that was applied
scale_up_fabric_state_enabledboolWhether the ScaleUpFabricState State is now enabled
grpc_enabledboolWhether gRPC access is now enabled

Errors: Refer to Conventions for how RMS reports failures.

Replace each <placeholder> with a real value. Define RMS_GRPCURL_FLAGS and RMS_ADDR as described in Getting started.

grpcurl $RMS_GRPCURL_FLAGS \
-d '{
"node": {
"node_id": "<node_id>",
"rack_id": "<rack_id>",
"type": "COMPUTE_GB200_NVIDIA",
"bmc_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"host_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"node_descriptor": {
"attributes": {
"<key>": "<attributes>"
}
},
"additional_host_endpoints": [
{
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
}
]
},
"topology_type": "<topology_type>",
"domain": "<domain>"
}' \
"$RMS_ADDR" rack_manager.RackManager/ConfigureScaleUpFabricManager

GetScaleUpFabricStatus

rack_manager.RackManager/GetScaleUpFabricStatus

GetScaleUpFabricStatus synchronously inspects the fabric without changing anything and returns the observed scale-up configuration: per-switch state, topology, and static configuration reported by the fabric manager. This read is not associated with a reconciliation job.

PropertyValue
ExecutionSynchronous. The response carries the full result.
TargetCaller-supplied nodes. The request carries each node’s endpoints and credentials, and RMS does not store them.
Behavior detailsOperations guide
Protobuf referenceGetScaleUpFabricStatus

Request: GetScaleUpFabricStatusRequest

FieldTypeDescription
nodesNodeSetAll switches that form the scale-up fabric.
domainoptional stringOptional site selector used for switch server verification (TLS). Defaults to —default-switch-domain when unset.

Response: GetScaleUpFabricStatusResponse

FieldTypeDescription
statusReturnCodeRETURN_CODE_SUCCESS if inspected, RETURN_CODE_FAILURE otherwise
fabric_statusScaleUpFabricStatusObserved configuration; unset when status is RETURN_CODE_FAILURE
error_messagestringError details if inspection failed

Errors: Refer to Conventions for how RMS reports failures.

Replace each <placeholder> with a real value. Define RMS_GRPCURL_FLAGS and RMS_ADDR as described in Getting started.

grpcurl $RMS_GRPCURL_FLAGS \
-d '{
"nodes": {
"nodes": [
{
"node_id": "<node_id>",
"rack_id": "<rack_id>",
"type": "COMPUTE_GB200_NVIDIA",
"bmc_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"host_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"node_descriptor": {
"attributes": {
"<key>": "<attributes>"
}
},
"additional_host_endpoints": [
{
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
}
]
}
]
},
"domain": "<domain>"
}' \
"$RMS_ADDR" rack_manager.RackManager/GetScaleUpFabricStatus

InspectScaleUpFabrics

rack_manager.RackManager/InspectScaleUpFabrics

InspectScaleUpFabrics discovers enabled controllers and reports their live compute and switch members, control-plane state, and sampled component status. Compute BMCs provide physical identities for correlating observed GPUs with caller node IDs; compute hosts are not configured or tested. No devices are changed. Results do not verify a desired configuration, end-to-end communication, or bandwidth.

This server returns UNIMPLEMENTED for live multi-fabric inspection.

PropertyValue
ExecutionSynchronous. The response carries the full result.
TargetCaller-supplied nodes. The request carries each node’s endpoints and credentials, and RMS does not store them.
Behavior detailsOperations guide
Protobuf referenceInspectScaleUpFabrics

Request: InspectScaleUpFabricsRequest

FieldTypeDescription
nodesNodeSetInclude the switches to inspect and, optionally, compute trays to identify in the results. At least one switch is required. Node IDs must be nonempty and unique. RMS uses NodeInfo.type when set to a value other than NODE_TYPE_UNSPECIFIED. Otherwise, RMS uses node_descriptor to identify the node’s role. Unknown or unsupported node roles are rejected. Each switch needs host_endpoint and credentials. Each compute entry represents one tray and needs bmc_endpoint and credentials to read chassis and tray positions. RMS reads compute BMCs for discovery; it does not configure compute hosts. Without compute entries, RMS cannot match observed computes to caller node IDs. Unresolved nodes are reported as collection errors rather than silently omitted.
domainoptional stringSelects the CA and RMS client credentials for all switches in this request. If omitted, RMS uses —default-switch-domain. TLS server-name verification uses each endpoint’s identity or the configured DNS override, not this value.

Response: InspectScaleUpFabricsResponse

FieldTypeDescription
statusReturnCodeSUCCESS means collection completed without read or interpretation errors. An ABNORMAL fabric status alone does not cause FAILURE.
fabricsrepeated ScaleUpFabricInspectionObservationOne observation per enabled controller, in switch inventory order. The list is empty when no controllers are enabled; that is not an error. Successful observations are retained when another read fails.
discovery_errorsrepeated ScaleUpFabricInspectionErrorDiscovery failures that cannot be associated with a fabric. Fabric-specific errors appear only in that fabric’s inspection_errors.
error_messagestringSummary of collection or interpretation failures; empty on SUCCESS.

Errors: Refer to Conventions for how RMS reports failures. This RPC also returns these gRPC errors:

  • UNIMPLEMENTED: Live multi-fabric inspection is not implemented by this server.

Replace each <placeholder> with a real value. Define RMS_GRPCURL_FLAGS and RMS_ADDR as described in Getting started.

grpcurl $RMS_GRPCURL_FLAGS \
-d '{
"nodes": {
"nodes": [
{
"node_id": "<node_id>",
"rack_id": "<rack_id>",
"type": "COMPUTE_GB200_NVIDIA",
"bmc_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"host_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"node_descriptor": {
"attributes": {
"<key>": "<attributes>"
}
},
"additional_host_endpoints": [
{
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
}
]
}
]
},
"domain": "<domain>"
}' \
"$RMS_ADDR" rack_manager.RackManager/InspectScaleUpFabrics

BatchResetSwitchSdnFactoryDefault

rack_manager.RackManager/BatchResetSwitchSdnFactoryDefault

BatchResetSwitchSdnFactoryDefault submits asynchronous SDN factory-default resets. All switches in the request use the same domain for switch mTLS material. Returns a parent job ID. Use GetJobStatus with child job states to track per-node progress. The reset is idempotent; callers can retry after transport or service failure. Note: post hard reset, the scale-up fabric must be reconfigured.

This operation is destructive. After a reset, reconfigure the scale-up fabric.

PropertyValue
ExecutionAsynchronous, parent and child jobs. The response returns a parent job ID and the work continues after the call returns.
Job trackingPoll GetJobStatus with the job ID in response.job_id. Set include_child_job_states on GetJobStatus to read per-node child jobs.
TargetCaller-supplied nodes. The request carries each node’s endpoints and credentials, and RMS does not store them.
Behavior detailsOperations guide
Protobuf referenceBatchResetSwitchSdnFactoryDefault

Request: BatchResetSwitchSdnFactoryDefaultRequest

FieldTypeDescription
nodesNodeSetTarget switch identities and direct connection information
domainoptional stringOptional domain for the target switch group; defaults to configured switch domain when unset

Response: BatchResetSwitchSdnFactoryDefaultResponse

FieldTypeDescription
responseNodeBatchResponseresponse.job_id is the parent job ID for GetJobStatus

Errors: Refer to Conventions for how RMS reports failures.

Replace each <placeholder> with a real value. Define RMS_GRPCURL_FLAGS and RMS_ADDR as described in Getting started.

grpcurl $RMS_GRPCURL_FLAGS \
-d '{
"nodes": {
"nodes": [
{
"node_id": "<node_id>",
"rack_id": "<rack_id>",
"type": "COMPUTE_GB200_NVIDIA",
"bmc_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"host_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"node_descriptor": {
"attributes": {
"<key>": "<attributes>"
}
},
"additional_host_endpoints": [
{
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
}
]
}
]
},
"domain": "<domain>"
}' \
"$RMS_ADDR" rack_manager.RackManager/BatchResetSwitchSdnFactoryDefault

BatchSetScaleUpFabricState

rack_manager.RackManager/BatchSetScaleUpFabricState

BatchSetScaleUpFabricState synchronously sets ScaleUpFabric state and saves config across all supplied switches.

PropertyValue
ExecutionSynchronous. The response carries the full result.
TargetCaller-supplied nodes. The request carries each node’s endpoints and credentials, and RMS does not store them.
Behavior detailsOperations guide
Protobuf referenceBatchSetScaleUpFabricState

Request: BatchSetScaleUpFabricStateRequest

FieldTypeDescription
nodesNodeSetSwitch nodes to update
enabledbooltrue = enabled, false/unset = disabled

Response: BatchSetScaleUpFabricStateResponse

FieldTypeDescription
responseNodeBatchResponseresponse.job_id is unused and left empty for this synchronous RPC

Errors: Refer to Conventions for how RMS reports failures.

Replace each <placeholder> with a real value. Define RMS_GRPCURL_FLAGS and RMS_ADDR as described in Getting started.

grpcurl $RMS_GRPCURL_FLAGS \
-d '{
"nodes": {
"nodes": [
{
"node_id": "<node_id>",
"rack_id": "<rack_id>",
"type": "COMPUTE_GB200_NVIDIA",
"bmc_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"host_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"node_descriptor": {
"attributes": {
"<key>": "<attributes>"
}
},
"additional_host_endpoints": [
{
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
}
]
}
]
},
"enabled": false
}' \
"$RMS_ADDR" rack_manager.RackManager/BatchSetScaleUpFabricState

BatchGetScaleUpFabricServiceStatus

rack_manager.RackManager/BatchGetScaleUpFabricServiceStatus

BatchGetScaleUpFabricServiceStatus retrieves cluster health and fabric-manager status for a caller-supplied list of switches using switch Rest APIs.

PropertyValue
ExecutionSynchronous. The response carries the full result.
TargetCaller-supplied nodes. The request carries each node’s endpoints and credentials, and RMS does not store them.
Behavior detailsOperations guide
Protobuf referenceBatchGetScaleUpFabricServiceStatus

Request: BatchGetScaleUpFabricServiceStatusRequest

FieldTypeDescription
nodesNodeSetSwitch nodes to query

Response: BatchGetScaleUpFabricServiceStatusResponse

FieldTypeDescription
statusReturnCodeRETURN_CODE_SUCCESS if retrieved, RETURN_CODE_FAILURE otherwise
service_statusesmap of string to ScaleUpFabricServiceStatusEntrykey = node_id
statsNodeOperationStatsAggregate query counts

Errors: Refer to Conventions for how RMS reports failures.

Replace each <placeholder> with a real value. Define RMS_GRPCURL_FLAGS and RMS_ADDR as described in Getting started.

grpcurl $RMS_GRPCURL_FLAGS \
-d '{
"nodes": {
"nodes": [
{
"node_id": "<node_id>",
"rack_id": "<rack_id>",
"type": "COMPUTE_GB200_NVIDIA",
"bmc_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"host_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"node_descriptor": {
"attributes": {
"<key>": "<attributes>"
}
},
"additional_host_endpoints": [
{
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
}
]
}
]
}
}' \
"$RMS_ADDR" rack_manager.RackManager/BatchGetScaleUpFabricServiceStatus

GetScaleUpFabricState

rack_manager.RackManager/GetScaleUpFabricState

GetScaleUpFabricState retrieves the cluster state for a caller-supplied switch using switch APIs.

PropertyValue
ExecutionSynchronous. The response carries the full result.
TargetCaller-supplied nodes. The request carries each node’s endpoints and credentials, and RMS does not store them.
Behavior detailsOperations guide
Protobuf referenceGetScaleUpFabricState

Request: GetScaleUpFabricStateRequest

FieldTypeDescription
nodeNodeInfoTarget switch identity and direct connection information

Response: GetScaleUpFabricStateResponse

FieldTypeDescription
statusReturnCodeRETURN_CODE_SUCCESS if retrieved, RETURN_CODE_FAILURE otherwise
state_jsonstringJSON object containing ScaleUpFabric state information
error_messagestringError details if query failed

Errors: Refer to Conventions for how RMS reports failures.

Replace each <placeholder> with a real value. Define RMS_GRPCURL_FLAGS and RMS_ADDR as described in Getting started.

grpcurl $RMS_GRPCURL_FLAGS \
-d '{
"node": {
"node_id": "<node_id>",
"rack_id": "<rack_id>",
"type": "COMPUTE_GB200_NVIDIA",
"bmc_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"host_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"node_descriptor": {
"attributes": {
"<key>": "<attributes>"
}
},
"additional_host_endpoints": [
{
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
}
]
}
}' \
"$RMS_ADDR" rack_manager.RackManager/GetScaleUpFabricState

SetScaleUpFabricTelemetryInterfaceState

rack_manager.RackManager/SetScaleUpFabricTelemetryInterfaceState

SetScaleUpFabricTelemetryInterfaceState enables or disables the FabricTelemetryInterface service. FabricTelemetryInterface provides streaming telemetry and configuration management capabilities.

PropertyValue
ExecutionSynchronous. The response carries the full result.
TargetCaller-supplied nodes. The request carries each node’s endpoints and credentials, and RMS does not store them.
Behavior detailsOperations guide
Protobuf referenceSetScaleUpFabricTelemetryInterfaceState

Request: SetScaleUpFabricTelemetryInterfaceStateRequest

FieldTypeDescription
nodeNodeInfoTarget switch identity and direct connection information
enablebooltrue to enable ScaleUpFabricTelemetryInterface, false to disable

Response: SetScaleUpFabricTelemetryInterfaceStateResponse

FieldTypeDescription
statusReturnCodeRETURN_CODE_SUCCESS if configured, RETURN_CODE_FAILURE otherwise
result_jsonstringJSON object containing configuration result
error_messagestringError details if configuration failed

Errors: Refer to Conventions for how RMS reports failures.

Replace each <placeholder> with a real value. Define RMS_GRPCURL_FLAGS and RMS_ADDR as described in Getting started.

grpcurl $RMS_GRPCURL_FLAGS \
-d '{
"node": {
"node_id": "<node_id>",
"rack_id": "<rack_id>",
"type": "COMPUTE_GB200_NVIDIA",
"bmc_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"host_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"node_descriptor": {
"attributes": {
"<key>": "<attributes>"
}
},
"additional_host_endpoints": [
{
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
}
]
},
"enable": false
}' \
"$RMS_ADDR" rack_manager.RackManager/SetScaleUpFabricTelemetryInterfaceState

ConfigureScaleUpFabricManager (V2)

rack_manager_v2.RackManagerV2/ConfigureScaleUpFabricManager

ConfigureScaleUpFabricManager declaratively configures the scale-up fabric across all supplied switches. The caller describes the desired fabric configuration and RMS starts a single asynchronous reconciliation job that performs only the actions required to move the observed switch state toward the desired state: it skips disabling the fabric when it is already disabled, and does nothing when the observed configuration matches the request and controller activation is verified.

The call is safe to invoke periodically in a loop to keep switch configuration “as intended” without causing fabric availability issues. RMS aims to run one reconciliation job at a time; callers should poll GetJobStatus before resubmitting and avoid issuing concurrent ConfigureScaleUpFabricManager requests.

Without layout, requests with an empty topology_type or otherwise invalid config are rejected synchronously with INVALID_ARGUMENT; no job is started.

Returns a job_id; poll it via rack_manager.GetJobStatus. Without layout, successful jobs return JSON-encoded rack_manager.ScaleUpFabricStatus in rack_manager.JobStatus.result_json.

With layout, the resulting configuration contains one cluster and primary per fabric, with administrative switch connectivity matching the requested layout. Submit the complete desired fabric set in one request because the fabrics share switch connectivity settings. GetJobStatus reports execution state and errors. InspectScaleUpFabrics reports current fabric members and sampled component status, including partial configurations after a failed job. Completion requires verified administrative switch settings and configured fabric-manager control planes, not compute software or operational GPU links.

Layout requests require matching deployment rules and the complete physical switch inventory. Without layout, RMS validates every node ID and type, then configures only switches.

PropertyValue
ExecutionAsynchronous, one job. The response returns a job ID and the work continues after the call returns.
Job trackingPoll GetJobStatus with the job ID in job_id.
Job resultresult_json in the job status holds a JSON-encoded ScaleUpFabricStatus for successful no-layout and full-capacity layout jobs; partial layouts use InspectScaleUpFabrics for live status.
TargetCaller-supplied nodes. The request carries each node’s endpoints and credentials, and RMS does not store them.
Behavior detailsOperations guide
Protobuf referenceConfigureScaleUpFabricManager

Request: ConfigureScaleUpFabricManagerRequest

FieldTypeDescription
nodesNodeSetWithout layout, RMS validates all nodes and configures only switches; other recognized node types are ignored. With layout, include every physical switch, even switches outside the requested fabrics, and compute trays for selection. At least one switch is required, and node IDs must be nonempty and unique. RMS uses NodeInfo.type when set to a value other than NODE_TYPE_UNSPECIFIED. Otherwise, RMS uses node_descriptor to identify the node’s role. Every node must resolve to a supported type. With layout, only switch and compute nodes are accepted. The order of nodes does not determine which fabric they belong to. Each switch needs host_endpoint and credentials. With layout, each compute entry represents one tray and needs bmc_endpoint and credentials for reading its position. RMS reads compute BMCs for layout discovery; it does not configure compute hosts. Compute entries are required unless layout requests one fabric covering the full physical compute and switch capacity. For that layout, either omit all compute entries or include every compute tray. Omission avoids compute BMC reads. For other layouts, compute trays outside the requested fabrics may be omitted.
primary_switch_node_idoptional stringID of the switch node that should run the fabric manager (the primary host). If absent, RMS selects the switch at the lowest position in the rack.
domainoptional stringSelects the CA and RMS client credentials for all switches in this request. If omitted, RMS uses —default-switch-domain. TLS server-name verification uses each endpoint’s identity or the configured DNS override, not this value.
configScaleUpFabricConfigDesired scale-up fabric configuration to reconcile the switches toward. With layout, caller settings are shared by selected fabrics; managed static values can differ by fabric size.
layoutScaleUpFabricLayoutOptional arrangement of independent fabrics within the supplied inventory; fabric sizes may differ from the complete physical topology. Supply the complete desired fabric set in one call to coordinate shared links. Requires config and at least one fabric; primary_switch_node_id must be absent. One fabric matching the complete physical compute and switch capacity restores full connectivity and one controller cluster. Other layouts isolate unused capacity. When absent, one cluster is configured without changing managed link settings.

Response: ConfigureScaleUpFabricManagerResponse

FieldTypeDescription
job_idstringID of the asynchronous job that reconciles the scale-up fabric. Poll it via rack_manager.GetJobStatus. Read live fabric members and component status via rack_manager.InspectScaleUpFabrics. Successful no-layout jobs also return JSON-encoded rack_manager.ScaleUpFabricStatus in result_json.
selected_fabricsrepeated ScaleUpFabricMembersEmpty without layout. With layout, identifies the nodes and primary selected for each requested fabric, in request order. Selection does not indicate that configuration has completed; use job_id to track execution.

Errors: Refer to Conventions for how RMS reports failures. This RPC also returns these gRPC errors:

  • INVALID_ARGUMENT: The topology, inventory, or layout request is invalid. No job starts.
  • FAILED_PRECONDITION: Physical discovery, deployment planning, or switch ownership prevents admission. No configuration worker starts.
  • INTERNAL: Job admission or coordinator state fails internally. No configuration worker starts.
  • UNIMPLEMENTED: Layout configuration has no installed deployment catalog or the switch family is unsupported. No job starts.

Replace each <placeholder> with a real value. Define RMS_GRPCURL_FLAGS and RMS_ADDR as described in Getting started.

grpcurl $RMS_GRPCURL_FLAGS \
-d '{
"nodes": {
"nodes": [
{
"node_id": "<node_id>",
"rack_id": "<rack_id>",
"type": "COMPUTE_GB200_NVIDIA",
"bmc_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"host_endpoint": {
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
},
"node_descriptor": {
"attributes": {
"<key>": "<attributes>"
}
},
"additional_host_endpoints": [
{
"interface": {
"ip_address": "<ip_address>",
"mac_address": "<mac_address>",
"host_name": "<host_name>"
},
"port": 0,
"credentials": {
"user_pass": {
"username": "<username>",
"password": "<password>"
}
}
}
]
}
]
},
"primary_switch_node_id": "<primary_switch_node_id>",
"domain": "<domain>",
"config": {
"topology_type": "<topology_type>",
"extra_static_configs": [
{
"config_file_name": "<config_file_name>",
"key": "<key>",
"value": "<value>"
}
]
},
"layout": {
"fabrics": [
{
"compute_tray_count": 0,
"switch_tray_count": 0,
"primary_switch_node_id": "<primary_switch_node_id>"
}
]
}
}' \
"$RMS_ADDR" rack_manager_v2.RackManagerV2/ConfigureScaleUpFabricManager