Update NVCF Account Quota Limits

View as Markdown

This runbook updates the account quota limits for the nvcf-default account post installation. Use this when an operation fails with an error like:

1{
2 "detail": "Account 'nvcf-default': Reached or exceeded the max limit for the number of functions allowed: 10."
3}

The following limits can be updated using the same method:

FieldDescriptionMax
maxFunctionsAllowedMaximum number of functions2147483647
maxTasksAllowedMaximum number of tasks2147483647
maxTelemetriesAllowedMaximum number of telemetries50
maxRegistryCredentialsAllowedMaximum number of registry credentials50

Prerequisites

To use nvcf-cli (the simpler path), you need:

  • nvcf-cli installed and authenticated against the target cluster

To use curl directly, you need:

  • kubectl
  • curl
  • jq
  • Cluster access with permission to create service account tokens
  • Access to the nvcf and vault-system namespaces

The NVCF API and OpenBao services must be reachable through kubectl port-forward.

Update Runtime Account Limits Using nvcf-cli

If nvcf-cli is available and configured, use the admin accounts update subcommand:

$export NVCF_CLI_ENABLE_ADMIN=1
$nvcf-cli api-key generate --for function \
> --scopes invoke_function,list_functions,queue_details,list_functions_details,account_setup
$nvcf-cli admin accounts update --nca-id nvcf-default --max-functions 50

Sibling flags for other limits:

FlagLimit
--max-functionsMaximum number of functions (cap 2147483647)
--max-tasksMaximum number of tasks (cap 2147483647)
--max-telemetriesMaximum number of telemetries (cap 50)
--max-registry-credentialsMaximum number of registry credentials (cap 50)

Run nvcf-cli admin accounts update --help for the full flag list.

Retry the operation that previously failed.

Update Runtime Account Limits Using curl

Port-forward the NVCF API:

$kubectl -n nvcf port-forward svc/api 18080:8080

In another terminal, port-forward OpenBao:

$kubectl -n vault-system port-forward svc/openbao-server 18200:8200

Create an audience-bound service account token for the bootstrap service account:

$SA_TOKEN="$(kubectl -n nvcf create token nvcf-api-account-bootstrap \
> --audience=http://openbao-server.vault-system.svc.cluster.local:8200 \
> --duration=3600s)"

Log in to OpenBao using that service account token:

$VAULT_TOKEN="$(curl -s \
> -X POST \
> -H "Content-Type: application/json" \
> -d "{\"role\":\"nvcf-api-account-bootstrap\",\"jwt\":\"${SA_TOKEN}\"}" \
> "http://127.0.0.1:18200/v1/auth/jwt/login" \
> | jq -r '.auth.client_token')"

Generate an NVCF admin JWT using the OpenBao signing role:

$ADMIN_TOKEN="$(curl -s \
> -X PUT \
> -H "X-Vault-Token: ${VAULT_TOKEN}" \
> -d '{}' \
> "http://127.0.0.1:18200/v1/services/nvcf-api/jwt/sign/nvcf-api-admin" \
> | jq -r '.data.token')"

Patch one or more account limits in a single request. Include only the fields you want to change:

$curl -i -w "\nHTTP_STATUS=%{http_code}\n" \
> -X PATCH \
> "http://127.0.0.1:18080/v2/nvcf/accounts/nvcf-default" \
> -H "Content-Type: application/json" \
> -H "Authorization: Bearer ${ADMIN_TOKEN}" \
> --data '{"maxFunctionsAllowed":50}'

To update multiple limits at once:

$curl -i -w "\nHTTP_STATUS=%{http_code}\n" \
> -X PATCH \
> "http://127.0.0.1:18080/v2/nvcf/accounts/nvcf-default" \
> -H "Content-Type: application/json" \
> -H "Authorization: Bearer ${ADMIN_TOKEN}" \
> --data '{"maxFunctionsAllowed":50,"maxTasksAllowed":50,"maxTelemetriesAllowed":20}'

Expected success response:

1{
2 "account": {
3 "ncaId": "nvcf-default",
4 "adminClientIds": [
5 "ncp"
6 ],
7 "name": "nvcf-default",
8 "maxFunctionsAllowed": 50,
9 "maxTasksAllowed": 50,
10 "maxTelemetriesAllowed": 20,
11 "maxRegistryCredentialsAllowed": 10
12 }
13}

Verify the updated limits:

$curl -s \
> "http://127.0.0.1:18080/v2/nvcf/accounts/nvcf-default" \
> -H "Authorization: Bearer ${ADMIN_TOKEN}" \
> | jq .

Retry the operation that previously failed after confirming the new limits are reflected.

Persist Future Bootstrap Limits

The runtime fix is the PATCH above. To prevent future installs or account bootstrap runs from using the old defaults, also update the Helm values:

1api:
2 accountBootstrap:
3 limits:
4 maxFunctions: 50
5 maxTasks: 50
6 maxTelemetries: 20
7 maxRegistryCredentials: 10

If upgrading the existing Helm release directly, use the same chart source that was used for installation:

$helm upgrade -n nvcf api <same-helm-nvcf-api-chart> \
> --reuse-values \
> --set api.accountBootstrap.limits.maxFunctions=50

Note: the account bootstrap job creates the account and treats an existing account as success. Updating Helm values alone may not change an already-created nvcf-default account, so apply the runtime PATCH when changing an existing deployment.

Troubleshooting

If the account patch returns 401, the JWT is expired. Regenerate ADMIN_TOKEN.

If the account patch returns 403, the token is valid but does not have enough privilege. Use the OpenBao-signed nvcf-api-admin token, not a normal function management token.

If VAULT_TOKEN is null, confirm SA_TOKEN was created in the same terminal:

$echo "SA_TOKEN length: ${#SA_TOKEN}"

If the length is 0, recreate SA_TOKEN and retry the OpenBao login.