Network Operator Application Notes 23.10.0 - Sphinx Test
1.0

Provisioning API - Overview

This page provides Provisioning API concepts along with an example.

Provisioning API provides a way to for the ClusterAdmin to trigger DPU provisioning in an infrastructure cluster.

Provisioning API is a set of CloudAdmin APIs.

CloudAdmin Provisioning API

The following infrastructure is used for the sake of this example.

provisioning_api_overview.png

Infrastructure Cluster

Copy
Copied!
            

✓ icp> kubectl get node NAME STATUS ROLES AGE VERSION icp-master Ready control-plane 27h v1.24.0

Universe Components

Universe components are deployed to infrastructure clusters by following the Deployment Guide.

The following lists the universe components in infrastructure cluster:

Copy
Copied!
            

✓ icp> kubectl get po -n vault NAME READY STATUS RESTARTS AGE vault-0 1/1 Running 0 28h vault-agent-injector-6fd8f84794-xqlg9 1/1 Running 0 21s ✓ icp> kubectl get po --all-namespaces --field-selector metadata.namespace!=kube-system NAMESPACE NAME READY STATUS RESTARTS AGE capi-kubeadm-bootstrap-system capi-kubeadm-bootstrap-controller-manager-677f784fb6-g64qm 1/1 Running 0 3h26m capi-kubeadm-control-plane-system capi-kubeadm-control-plane-controller-manager-7bf594b9c5-tsrcz 1/1 Running 0 3h26m capi-system capi-controller-manager-6495b5bcd-5vcp4 1/1 Running 0 3h26m capm3-system capm3-controller-manager-547dd65854-4cgzq 1/1 Running 0 3h26m capm3-system ipam-controller-manager-7c85f789d-rt4m6 1/1 Running 0 3h26m cert-manager cert-manager-6dd9658548-5vp62 1/1 Running 0 3h26m cert-manager cert-manager-cainjector-5987875fc7-rvcwg 1/1 Running 0 3h26m cert-manager cert-manager-webhook-7b4c5f579b-fvwzd 1/1 Running 0 3h26m universe icp-universe-infra-provisioning-baremetal-operator-5cc9bb6tcmv9 1/1 Running 0 3h3m universe icp-universe-infra-provisioning-controller-8d47cd467-h87nb 1/1 Running 0 3h3m universe icp-universe-infra-provisioning-bootp-66d9cc9964-x95pm 2/2 Running 0 3h3m universe icp-universe-infra-provisioning-ironic-bb7776bfb-dcgbh 2/2 Running 0 3h3m universe icp-universe-infra-provisioning-manager-c58ccc78b-jhs8x 1/1 Running 0 3h3m universe icp-universe-infra-provisioning-mariadb-6c488ddc88-h6hwb 1/1 Running 0 3h3m

This section tests universe the provisioning API in the infrastructure cluster. The proto file and generated GO client for the API can be found in the universe-api repo (refer to the Manual GRPC API usage document before starting) and use ‘grpcurl’ tool to verify the provisioning API.

Create BMC Secret

Copy
Copied!
            

✓ icp> cat << 'EOF' | tee cred.json { "cred":{ "name":"dpu-bmc-secret", "user_name": "<user name of BMC>", "password": "<password of BMC>" } } EOF ✓ icp> grpcurl -cacert=ca.crt -cert=admin.crt -key=admin.key -servername api-gateway.local -d "$(cat cred.json | tr -d '\n')" \ -proto universe/admin/provisioning/v1/provisioning.proto <infraprovisioning service ip:port> \ universe.admin.provisioning.v1.UniverseProvisioningService.CreateCredential

Create DPU Settings

Copy
Copied!
            

✓ icp> cat << EOF | tee settings.json { "settings":{ "linkType": "LINK_TYPE_ETH", "name": dpu-settings", "osImage": { "url": "<the URL of OS images>", "checksum": "<the URL of OS image's md5sum>", "checksumType": "CHECKSUM_TYPE_MD5", "diskFormat": "DISK_FORMAT_RAW" }, } } EOF ✓ icp> grpcurl -cacert=ca.crt -cert=admin.crt -key=admin.key -servername api-gateway.local -d "$(cat settings.json | tr -d '\n')" \ -proto universe/admin/provisioning/v1/provisioning.proto <infraprovisioning service ip:port> \ universe.admin.provisioning.v1.UniverseProvisioningService.CreateSettings

Create Host

Copy
Copied!
            

cat << EOF | tee host.json { "host":{ "name":"dpu-host" } } EOF ✓ icp> grpcurl -cacert=ca.crt -cert=admin.crt -key=admin.key -servername api-gateway.local -d "$(cat host.json | tr -d '\n')" \ -proto universe/admin/provisioning/v1/provisioning.proto <infraprovisioning service ip:port> \ universe.admin.provisioning.v1.UniverseProvisioningService.CreateHost

Provision DPU

Copy
Copied!
            

✓ icp> cat << EOF | tee dpu.json { "dpu":{ "name":"dpu-1", "bmc":{ "url":"dpu://<BMC ip address>", "credential_ref":{ "name":"dpu-bmc-secret" } }, "boot_mac_address":"<DPU MAC address>", "target_settings":{ "name":"dpu-settings" }, "host_ref":{ "name": "dpu-host" } } } EOF ✓ icp> grpcurl -cacert=ca.crt -cert=admin.crt -key=admin.key -servername api-gateway.local -d "$(cat dpu.json | tr -d '\n')" \ -proto universe/admin/provisioning/v1/provisioning.proto <infraprovisioning service ip:port> \ universe.admin.provisioning.v1.UniverseProvisioningService.CreateDPU

Note

DPU provisioning starts only if a “dpu.target_settings” is provided.

Check DPU Status

Copy
Copied!
            

✓ icp> grpcurl -cacert=ca.crt -cert=admin.crt -key=admin.key -servername api-gateway.local -d '{"name": "dpu-1"}' \ -proto universe/admin/provisioning/v1/provisioning.proto <infraprovisioning service ip:port> \ universe.admin.provisioning.v1.UniverseProvisioningService.GetDPU { "dpu": { "name": "dpu-1", "hostRef": { "name": "dpu-host", "uid": "" }, "bmc": { "url": "dpu://<BMC IP Address>", "credentialRef": { "name": "dpu-bmc-secret", "namespace": "" } }, "bootMacAddress": "<DPU MAC Address>", "targetSettings": { "name": "dpu-settings", "uid": "" }, "status": { "conditions": [ { "type": "Ready", "status": "STATUS_FALSE", "lastTransitionTime": "2022-11-09 12:38:23 +0000 UTC", "reason": "MachineStateProvisioning", "message": "" }, { "type": "ProvisioningStarted", "status": "STATUS_TRUE", "lastTransitionTime": "2022-11-09 12:38:23 +0000 UTC", "reason": "ProvisioningStarted", "message": "ProvisioningStarted" } ] } } }

Note

Once provisioning is done, the ‘dpu.status.conditions[type==Ready]’ is updated to ‘STATUS_TRUE’ and the ‘lastTransitionTime’ is populated with the completion time of the provisioning.

Note

Timeout is not defined for provisioning duration by default. So in case any fauilre happened during the provisioning the ‘dpu.status.conditions[type==Ready]’ remains as ‘STATUS_FALSE’.

Check HOST Status

Copy
Copied!
            

✓ icp> grpcurl -cacert=ca.crt -cert=admin.crt -key=admin.key -servername api-gateway.local -d '{"name": "hpc-cloud05"}' \ -proto universe/admin/provisioning/v1/provisioning.proto <infraprovisioning service ip:port> \ universe.admin.provisioning.v1.UniverseProvisioningService.GetHost { "host": { "name": "hpc-cloud05", "status": { "conditions": [ { "type": "AdminActionsRequired", "status": "STATUS_TRUE", "lastTransitionTime": "2022-12-30 18:18:23 +0000 UTC", "reason": "FirmwareUpdated", "message": "" } ] } } }

Note

Power cycle on the host server is needed to complete provisioning. Once provisioning is done, the ‘host.status.conditions[type==AdminActionsRequired]’ is set to ‘STATUS_TRUE’.

Previous Workload API - overview
Next Catalog API - Overview
© Copyright 2023, NVIDIA. Last updated on Feb 7, 2024.