Deployment
Self-hosted NVCF installation includes the core components required for NVCF inference. Optional components such as caching and low latency streaming support are also available. Vanity Gateway routing is available only in stack packages that include the Vanity Gateway addon. NVCF UI is available only in stack packages that include the NVCF UI addon.
For an umbrella release, start with Consume an NVCF release. Use the selected bundle and matching docs edition for installation.
For source-based local development on k3d, start with the Quickstart. The quickstart uses nvcf-cli self-hosted up to install the control plane, register the local k3d cluster, install NVCA, and run basic health checks.
For a full list of required artifacts, see self-hosted-artifact-manifest.
Want to try NVCF locally first? See Local Development to create a k3d cluster, then use the Quickstart local k3d flow.
Choose an installation path
The control plane and GPU cluster can be the same Kubernetes cluster or separate clusters when you use Helmfile or the explicit CLI install primitives. The quickstart supports only a single local k3d cluster. For a complete Amazon EKS example of both topologies, see the CSP End-to-End Example.
For remote installs, prepare the Gateway API ingress path and CLI endpoint configuration before registering GPU clusters or running post-install CLI checks. See Helmfile Installation, Self-Managed Clusters, and Gateway Routing.
Installation order
For a fresh installation, install the control plane and any enabled Observability profile, then register each GPU cluster and install the compute plane. Existing installations use the upgrade procedure.
After the core deployment works, configure the optional features your workload needs: caches, low latency streaming, Vanity Gateway or NVCF UI.
Kubernetes Cluster Requirements
Cluster Version
- Supported versions are the latest Kubernetes minor release and the two prior minor releases (N-2). See official Kubernetes docs for current supported versions.
- Support for dynamic persistent volume provisioning
Required Operators and Components
NVIDIA GPU Operator
Required for GPU workload scheduling. The GPU Operator automates the management of all NVIDIA software components needed to provision GPUs in Kubernetes, including:
- NVIDIA device drivers
- Kubernetes device plugin for GPU discovery
- GPU feature discovery for node labeling
- Container runtime integration (containerd, CRI-O, or Docker)
- Monitoring and telemetry tools
See NVIDIA GPU Operator documentation for installation instructions.
Fake GPU Operator for development and testing:
For environments without actual GPU hardware, install the fake GPU operator to simulate GPU resources. See fake-gpu-operator for full instructions.
SMB CSI Driver
The SMB CSI driver
(smb.csi.k8s.io) must be installed on every GPU cluster. NVCA uses the
driver for shared model cache storage that function worker pods mount. Install
and verify the driver before registering the GPU cluster. See the
Self-Managed Clusters prerequisites
for the installation command.
Network Policies
Your cluster must support Kubernetes Network Policies if network isolation is required.
Persistent Storage
A StorageClass must be configured for persistent volumes. Common options:
- Amazon EKS:
gp3(default) - Local development:
local-path - Other platforms: Any CSI-compatible storage class
Some cloud providers have minimum PVC size requirements. For example, AWS EBS gp3 volumes have a 1Gi minimum.
Cluster Sizing and Storage
See infrastructure-sizing for node pool specifications, storage recommendations, and three recommended sizing tiers (Development, Minimal HA, and Production).
