Self-hosted CLI
Self-hosted CLI
This page provides documentation for the NVCF Self-hosted CLI, a command-line interface for managing NVIDIA Cloud Functions in self-hosted deployments.
Overview
The NVCF Self-hosted CLI provides:
- Automatic Token Generation: Generate
NVCF_TOKENand API keys via direct API calls - Smart State Management: Persistent workflow context eliminates manual ID copying
- Multi-Environment Support: Separate configurations for dev/staging/production
- gRPC Invocation: Native support for gRPC function invocation
- Shell Completion: Autocompletion for bash, zsh, fish, and PowerShell
Prerequisites
- Network access to NVCF API endpoints
- A source checkout with Bazel/Bazelisk when you build from the repository
- NGC CLI installed when you download the CLI release from NGC
Installation
You can build nvcf-cli from this repository or download a packaged CLI release
from NGC. Use the source build when you are validating local changes or running
the local k3d quickstart from a repository checkout.
Build from the repository
Run the build from the repository root:
The binary is written to:
Install it on your PATH:
If your environment cannot reach the configured Bazel remote cache, disable the remote cache for this build:
Download from NGC
The CLI is available as a resource from NGC. See download-nvcf-cli for detailed download and extraction instructions.
The downloaded package includes:
nvcf-cli- The CLI binary.nvcf-cli.yaml.template- Configuration templateexamples/- Sample configuration filesUSAGE-GUIDE.md- Detailed usage documentation
Configuration
The CLI uses YAML configuration files. If you downloaded the packaged CLI, copy the included template:
If you built the CLI from source, create .nvcf-cli.yaml from the examples
below or from src/clis/nvcf-cli/examples/config-dev.yaml.
Configuration files are searched in this order:
- Explicit path via
--configflag (highest priority) - Current directory:
./.nvcf-cli.yaml - Home directory:
~/.nvcf-cli.yaml
Place your .nvcf-cli.yaml in the directory where you run the CLI for project-specific configuration, or in your home directory for global configuration.
Self-Hosted Configuration
For self-hosted deployments, the CLI must be configured to communicate with your gateway. The gateway uses hostname-based routing for HTTP services.
For Gateway routing details, including architecture diagrams, verification commands, and production DNS/HTTPS setup, see gateway-routing.
Prepare Gateway API ingress
For remote Helmfile deployments, set up Gateway API ingress before you configure the CLI. The CLI calls the configured API, API Keys, invocation, and gRPC endpoints during token minting, cluster registration, health checks, and function operations.
Complete Gateway quickstart before you configure the CLI. That procedure installs the Gateway API CRDs, creates and labels the required namespaces, installs Envoy Gateway, creates the GatewayClass and Gateway, waits for the Gateway to be programmed, and exports:
These are the same Gateway setup steps used by the Helmfile and standalone install paths. Keep the exported values in your shell, then configure the CLI. The local k3d quickstart uses local route hostnames instead.
For test environments without production DNS, use the Gateway load balancer address as the stack domain:
For production environments, set STACK_DOMAIN to the DNS name that your
HTTPRoute hostnames use.
Configuring the CLI
Create your configuration file:
Complete self-hosted configuration:
For test environments without production DNS, the URL fields and host fields can use the Gateway load balancer address:
Verifying Your Configuration
After configuring the CLI, verify connectivity:
If you see a 404 error, verify:
- The
api_keys_hostvalue matches your HTTPRoute hostname - The gateway load balancer is accessible
- The API Keys service is running:
kubectl get pods -n api-keys
Why Host headers are needed: the Envoy Gateway uses hostname-based routing to
direct traffic to different backend services through a single load balancer.
Without the correct Host header, the gateway cannot match the request to a
route and returns 404.
gRPC does not need Host headers because it uses a dedicated TCP listener on port 10081. The gateway routes all traffic on that port directly to the gRPC service without hostname matching.
Production Setup: DNS and HTTPS
The Host header configuration above is designed for testing and development. For production deployments, configure proper DNS and TLS to eliminate the need for Host header overrides.
With proper DNS and HTTPS configured:
- DNS records resolve service hostnames directly to your Gateway’s load balancer
- TLS certificates secure all traffic
- The CLI uses simple URLs without Host header overrides
- Browsers and other clients can access services directly
For complete instructions on setting up DNS records and TLS certificates, see production-dns-https in the Gateway Routing guide.
Multi-Environment Setup
Use the --config flag to manage multiple environments with separate configuration files:
Each configuration maintains separate state files (e.g., ~/.nvcf-cli.dev.state for dev.yaml).
Debug Mode
Enable debug mode for detailed logging by adding to your configuration file:
Or use the --debug flag or NVCF_DEBUG=true environment variable per-command.
Quick Start
For immediate testing, you can use load_tester_supreme from nvcf-onprem (see self-hosted-artifact-manifest), which supports the {"message": "hello world"} request body above. For more function samples, see the nv-cloud-function-helpers repository and function-creation for function creation documentation.
Authentication
The CLI stores three bearer credential types:
NVCF_TOKEN: Generated bynvcf-cli init. The default CLI credential for management operations and self-hosted cluster management.NVCF_API_KEY: Generated bynvcf-cli api-key generate. The default CLI credential for function invocation, function discovery, and queue status.NVCF_NVCT_API_KEY: Generated bynvcf-cli api-key generate. Used automatically for alltasksubcommands.
For NVCF API endpoints, either bearer type can be used when it includes the
required scope. The CLI prefers NVCF_API_KEY for read, invoke, and queue
commands. It prefers NVCF_TOKEN for management commands when both credentials
are configured. Self-hosted SIS cluster management uses NVCF_TOKEN. Task
commands always use NVCF_NVCT_API_KEY.
See Scope reference for the self-hosted scope matrix used by CLI commands and API endpoints.
Generate NVCF_TOKEN
Refresh NVCF_TOKEN
Refresh your token while preserving function context:
Generate API Keys
api-key generate mints both a function key (NVCF_API_KEY) and a task key
(NVCF_NVCT_API_KEY) in a single command. Use --for to generate only one.
Default function key scopes:
Default task key scopes:
Command Reference
Self-hosted Deployment Commands
Use these commands to install and inspect self-hosted NVCF deployments. For the local k3d installation flow, see Quickstart.
Bundle source overrides:
--control-plane-stackselects the control-plane stack bundle.--compute-plane-stackselects the compute-plane stack bundle.- Both flags accept local paths, git URLs, and
oci://references.
self-hosted up supports only a single local k3d cluster. It requires
--env local, a current k3d-* kube context, and no split-context flags. For
separate control-plane and GPU clusters, use the explicit control-plane and
compute-plane install primitives with Self-Managed Clusters.
Cluster Registration
Self-managed GPU clusters must be registered with the control plane before the NVCA
operator can start an agent. Registration records the GPU cluster’s OIDC issuer and public
JWKS with the control plane (ICMS) so the agent’s projected service account tokens (PSAT)
validate at runtime. The cluster register command performs this registration and prints
the Helm values the operator install needs.
init does double duty: it mints the admin token and discovers the control-plane issuer.
Run init before cluster register. The one-click self-hosted up flow runs both
internally.
cluster register flags:
Issuer and JWKS discovery: cluster register detects the GPU cluster’s OIDC issuer and
fetches its public JWKS, then sends them to ICMS. Detection precedence:
--oidc-issuer-urlif provided (manual override).- A SPIRE OIDC discovery service in the cluster, if present.
- The Kubernetes API server OIDC endpoint (default).
The detected source is recorded as identitySource in the output: psat for the
Kubernetes API server, spire for SPIRE, or custom for a manual issuer.
Output: the command prints a summary (cluster group ID, cluster ID, OIDC issuer, region)
followed by a --- Helm values for nvca-operator --- block. Copy that YAML block into a
<cluster-name>-register-values.yaml file and pass it to the operator install. The values
schema:
For load-balancer-fronted gateways that route by hostname, add the matching host-header
overrides (selfManaged.icmsServiceHostHeaderOverride,
selfManaged.revalServiceHostHeaderOverride, selfManaged.natsHostOverride) to these
values. See self-managed-clusters for how the
register values feed the operator install and when host-header overrides are required.
General Commands
API Key Commands
Task Commands
Task commands manage NVCT (NVIDIA Cloud Tasks) workloads. They require a task
API key, which api-key generate mints automatically alongside the function key.
Function Management Commands
Create Function
All function create flags:
Example function JSON:
LLM functions use functionType: "LLM" and define model routing metadata under models[].llmConfig:
For LLM models, llmConfig.routingMethod accepts round_robin, power_of_two, groq_multiregion, pulsar, or random.
Supported LLM paths are /v1/chat/completions, /v1/responses, and /v1/embeddings.
llmConfig.tokenRateLimit accepts one or more comma-separated positive integer token limits in <value>-<unit> format. Supported units are S (seconds), M (minutes), H (hours), D (days), and W (weeks). Use 1000-S for a single limit, or 1000-S,5000-M,100000-H,500000-D,1000000-W for a combined limit with distinct units. Use JSON input for combined limits because inline CLI model specs use commas as field separators.
Deploy Function
The function deploy command group manages deployments with the following subcommands:
Key function deploy create flags:
Example deployment JSON:
List and Get Functions
Update Function
LLM model updates can also be provided in the input file:
Invoke Function
Note: The CLI function invoke command detects LLM functions automatically.
For LLM functions, --model-name and --inference-url are required. The CLI uses the LLM invocation route and sets the OpenAI model value to <function-id>/<model-name>.
For LLM Gateway endpoint behavior, routing, and session stickiness details, see LLM Gateway.
For raw HTTP invocation, HTTP streaming, gRPC metadata, and invocation error behavior, see Generic HTTP Function Invocation and gRPC Function Invocation.
Additional function invoke flags:
Queue Management
Delete Function
Registry Credentials Commands
Manage container registry credentials for function images and Helm charts. For comprehensive setup instructions including IAM configuration for AWS ECR, see third-party-registries-self-hosted.
Troubleshooting
Authentication Errors
401 Unauthorized on function creation:
403 Forbidden on invocation:
Token Expiration
Connection Issues
Token Usage Summary
For additional troubleshooting, see self-hosted-troubleshooting.