Before You Deploy

Artifact Manifest

View as Markdown

This section provides a comprehensive list of all components required for NVIDIA Cloud Functions (NVCF) Self-Hosted deployment for basic inference. Additional components are needed for Low Latency Streaming (Simulation).

Artifacts Overview

The following inventories list the artifacts for an inference-only self-hosted NVCF deployment. Artifacts are grouped by deployment plane and type.

Artifact version compatibility

Newer artifact versions might be available. NVCF self-managed stack and compute-plane stack releases are QA-qualified as umbrella releases with the specific versions shown on this page. Use these versions together. NVIDIA cannot guarantee compatibility when you substitute other artifact versions.

Prepare Helm charts for Helmfile

The self-managed Helmfile bundles currently expect NVCF charts in an OCI registry. Chart distributions that start with https:// are Helm repository charts. Before deploying with Helmfile, copy each required repository chart version into the OCI registry configured by global.helm.sources in the Helmfile bundles.

The following example copies one public NVCF chart into an OCI registry:

$export CHART_NAME="helm-nvcf-api"
$export CHART_VERSION="1.22.5"
$export TARGET_REGISTRY="<registry-host>"
$export TARGET_REPOSITORY="<repository>"
$
$helm repo add nvcf https://helm.ngc.nvidia.com/nvidia/nvcf
$helm repo update
$helm pull "nvcf/${CHART_NAME}" --version "${CHART_VERSION}"
$
$helm registry login "${TARGET_REGISTRY}"
$helm push "${CHART_NAME}-${CHART_VERSION}.tgz" \
>  "oci://${TARGET_REGISTRY}/${TARGET_REPOSITORY}"

Repeat this process for every required chart with an https:// distribution. Copy required charts with an nvcr.io distribution into the same target repository so Helmfile can resolve all NVCF charts from one source. Configure the stack environment with that OCI location:

1global:
2  helm:
3    sources:
4      registry: "<registry-host>"
5      repository: "<repository>"

See Image Mirroring for additional registry examples.

Some supporting components such as the GPU Operator, OpenBao, NATS, Cassandra, etc. can alternatively be pulled directly from public NGC Catalog or other public opensource repositories if desired.

The following tables list the complete artifact inventory.

Control plane Helm charts

ArtifactVersionRequiredDescriptionDistributionSource code
helm-admin-token-issuer-proxy1.4.3OptionalDeploys the admin token issuer proxy used by the reference architecture.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-admin-token-issuer-proxy:1.4.3GitHub
helm-nvcf-api1.23.6RequiredDeploys the NVCF API service.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-api:1.23.6
helm-nvcf-api-keys1.6.0RequiredDeploys the API key management service.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-api-keys:1.6.0GitHub
helm-nvcf-cassandra0.15.5RequiredDeploys Cassandra and its initialization jobs.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-cassandra:0.15.5GitHub / Upstream
helm-nvcf-cert-manager0.1.0RequiredDeploys the NVCF cert-manager configuration.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-cert-manager:0.1.0GitHub / Upstream
helm-nvcf-ess-api1.6.1RequiredDeploys the Encrypted Secrets Service API.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-ess-api:1.6.1
helm-nvcf-grpc-proxy1.6.7RequiredDeploys the gRPC proxy service.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-grpc-proxy:1.6.7GitHub
helm-nvcf-invocation-service1.5.5RequiredDeploys the HTTP invocation service.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-invocation-service:1.5.5GitHub
helm-nvcf-llm-api-gateway1.2.0OptionalDeploys the OpenAI-compatible LLM API gateway.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-llm-api-gateway:1.2.0GitHub
helm-nvcf-llm-request-router1.6.3OptionalDeploys the LLM request router.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-llm-request-router:1.6.3GitHub
helm-nvcf-nats0.7.1RequiredDeploys NATS messaging for the control plane.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-nats:0.7.1GitHub / Upstream
helm-nvcf-nats-auth-callout-service1.1.3RequiredDeploys the NATS authorization callout service.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-nats-auth-callout-service:1.1.3GitHub
helm-nvcf-notary-service1.4.2RequiredDeploys the notary service for signing and validation.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-notary-service:1.4.2
helm-nvcf-nvct-api1.4.3RequiredDeploys the NVCF tenant API service.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-nvct-api:1.4.3
helm-nvcf-openbao-server0.30.23RequiredDeploys OpenBao secret management.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-openbao-server:0.30.23GitHub / Upstream
helm-nvcf-rate-limiter1.0.3OptionalDeploys request rate limiting for supported invocation paths.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-rate-limiter:1.0.3GitHub
helm-nvcf-sis1.18.3RequiredDeploys the Spot Instance Service.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-sis:1.18.3
helm-nvcf-state-metrics1.0.1OptionalDeploys NVCF state metrics for observability.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-state-metrics:1.0.1
helm-nvcf-vanity-gateway0.1.0-nvcf-10204.1OptionalDeploys the optional vanity hostname gateway.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvcf-vanity-gateway:0.1.0-nvcf-10204.1
helm-reval1.3.8RequiredDeploys the function revalidation service.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-reval:1.3.8GitHub
nvcf-example-dashboards1.6.0OptionalDeploys example Grafana dashboards for NVCF telemetry.https://helm.ngc.nvidia.com/nvidia/nvcf/nvcf-example-dashboards:1.6.0
nvcf-gateway-routes1.14.0OptionalDeploys Gateway API routes for the reference architecture.https://helm.ngc.nvidia.com/nvidia/nvcf/nvcf-gateway-routes:1.14.0GitHub
nvcf-observability-reference-stack1.10.0OptionalDeploys a reference observability backend for evaluation.https://helm.ngc.nvidia.com/nvidia/nvcf/nvcf-observability-reference-stack:1.10.0

Control plane services and images

ArtifactVersionRequiredDescriptionDistributionSource code
admin-token-issuer-proxy1.0.2OptionalProxies admin token requests for the reference architecture.nvcr.io/nvidia/nvcf/admin-token-issuer-proxy:1.0.2
alpine-k8s1.36.1RequiredProvides Kubernetes command-line utilities for deployment jobs.nvcr.io/0833294136851237/selfhosted-ga/alpine-k8s:1.36.1-ea
cert-manager-cainjectorv1.20.2RequiredInjects certificate authority data into Kubernetes resources.nvcr.io/nvidia/nvcf/cert-manager-cainjector:v1.20.2Upstream
cert-manager-controllerv1.20.2RequiredReconciles certificates and issuers for the control plane.nvcr.io/nvidia/nvcf/cert-manager-controller:v1.20.2Upstream
cert-manager-startupapicheckv1.20.2RequiredVerifies that the cert-manager API is ready.nvcr.io/nvidia/nvcf/cert-manager-startupapicheck:v1.20.2Upstream
cert-manager-webhookv1.20.2RequiredValidates and converts cert-manager API resources.nvcr.io/nvidia/nvcf/cert-manager-webhook:v1.20.2Upstream
ess-apiv0.57.26RequiredProvides encrypted application secrets to NVCF workloads.nvcr.io/nvidia/nvcf/ess-api:v0.57.26
llm-api-gateway0.8.3OptionalExposes OpenAI-compatible APIs for LLM functions.nvcr.io/0833294136851237/selfhosted-ga/llm-api-gateway:0.8.3-eaGitHub
llm-request-router0.3.0OptionalRoutes LLM requests to eligible worker instances.nvcr.io/0833294136851237/selfhosted-ga/stargate:0.3.0-eaGitHub
nats-box0.19.7-nonrootRequiredProvides NATS administration and diagnostic utilities.nvcr.io/nvidia/nvcf/nats-box:0.19.7-nonrootUpstream
nats-server2.11.17-alpine3.22RequiredProvides messaging for function deployment and invocation.nvcr.io/nvidia/nvcf/nats-server:2.11.17-alpine3.22Upstream
nats-server-config-reloader0.23.0RequiredReloads NATS server configuration when mounted settings change.nvcr.io/0833294136851237/selfhosted-ga/nats-server-config-reloader:0.23.0-eaUpstream
notary-service1.8.1RequiredSigns and validates functions and cluster nodes.nvcr.io/nvidia/nvcf/notary-service:1.8.1
nvcf-api-keys-service1.5.0RequiredCreates and manages NVCF API keys.nvcr.io/nvidia/nvcf/nvcf-api-keys-service:1.5.0
nvcf-grpc-proxy1.29.1RequiredProxies bidirectional gRPC traffic between the control and compute planes.nvcr.io/nvidia/nvcf/nvcf-grpc-proxy:1.29.1GitHub
nvcf-invocation-service0.8.5RequiredRoutes stateless HTTP function invocation requests.nvcr.io/nvidia/nvcf/nvcf-invocation-service:0.8.5GitHub
nvcf-nats-auth-callout-service0.5.10RequiredAuthorizes NATS clients for NVCF services and workloads.nvcr.io/nvidia/nvcf/nvcf-nats-auth-callout-service:0.5.10GitHub
nvcf-openbao2.5.4-nv-1.3.0RequiredStores and manages control-plane secrets.nvcr.io/0833294136851237/selfhosted-ga/nvcf-openbao:2.5.4-nv-1.3.0-eaUpstream
nvcf-openbao-migrations0.16.1RequiredApplies the OpenBao configuration required by NVCF.nvcr.io/nvidia/nvcf/nvcf-openbao-migrations:0.16.1GitHub
nvcf-service-oss1.9.0-hotfix.1RequiredProvides the primary NVCF control-plane API.nvcr.io/nvidia/nvcf/nvcf-service-oss:1.9.0-hotfix.1
nvct-service-oss1.5.9-hotfix.1RequiredProvides tenant-scoped NVCF control-plane operations.nvcr.io/nvidia/nvcf/nvct-service-oss:1.5.9-hotfix.1
oss-vault-k8s1.7.4RequiredIntegrates Kubernetes workloads with OpenBao secrets.nvcr.io/nvidia/nvcf/oss-vault-k8s:1.7.4
reval-server0.2.2RequiredRevalidates function state in the background.nvcr.io/nvidia/nvcf/reval-server:0.2.2GitHub
spot1.563.1-hotfix.1RequiredManages deployments, clusters, and function instances.nvcr.io/nvidia/nvcf/spot:1.563.1-hotfix.1

Compute plane Helm charts

ArtifactVersionRequiredDescriptionDistributionSource code
csi-driver-smbsupportedOptionalProvides SMB persistent volumes for supported deployments.https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/chartsUpstream
ebs-csi-driversupportedOptionalProvides Amazon EBS persistent volumes for EKS clusters.https://kubernetes-sigs.github.io/aws-ebs-csi-driverUpstream
gpu-operatorsupportedRequiredManages NVIDIA GPU software on Kubernetes nodes.https://helm.ngc.nvidia.com/nvidiaUpstream
helm-nvca-operator1.12.7RequiredDeploys the NVCA operator and compute-plane integration.https://helm.ngc.nvidia.com/nvidia/nvcf/helm-nvca-operator:1.12.7GitHub
nvcf-container-cache0.25.22OptionalDeploys container image caching on GPU cluster nodes.https://helm.ngc.nvidia.com/nvidia/nvcf/nvcf-container-cache:0.25.22GitHub

Compute plane services and images

ArtifactVersionRequiredDescriptionDistributionSource code
ess-agent1.3.1RequiredInjects encrypted application secrets into function workloads.nvcr.io/nvidia/nvcf/ess-agent:1.3.1GitHub
gpu-operator-validatorsupportedRequiredValidates GPU Operator components on GPU nodes.https://catalog.ngc.nvidia.com/orgs/nvidia/teams/cloud-native/containers/gpu-operator-validatorUpstream
k8s-device-pluginsupportedRequiredAdvertises NVIDIA GPU resources to Kubernetes.https://catalog.ngc.nvidia.com/orgs/nvidia/teams/k8s/containers/device-pluginUpstream
nvca3.0.3RequiredRegisters GPU clusters and orchestrates deployments in-cluster.nvcr.io/nvidia/nvcf/nvca:3.0.3GitHub
nvca-operator3.0.3RequiredReconciles NVCA resources and compute-plane configuration.nvcr.io/nvidia/nvcf/nvca-operator:3.0.3
nvcf-container-cachev1.1.36OptionalCaches container image layers on GPU cluster nodes.nvcr.io/nvidia/nvcf/nvcf-container-cache:v1.1.36GitHub
nvcf-image-credential-helper0.10.2RequiredResolves container image credentials for function workloads.nvcr.io/nvidia/nvcf/nvcf-image-credential-helper:0.10.2GitHub
nvcf-proxy-tls-certsv1.2.10OptionalConfigures TLS trust for the optional container cache proxy.nvcr.io/nvidia/nvcf/nvcf-proxy-tls-certs:v1.2.10
nvcf_worker_init1.0.1RequiredPrepares function resources before the user container starts.nvcr.io/nvidia/nvcf/nvcf_worker_init:1.0.1GitHub
nvcf_worker_llm_credentials1.0.1OptionalMaintains a current NVCF worker token for LLM function workloads.nvcr.io/nvidia/nvcf/nvcf_worker_llm_credentials:1.0.1GitHub
nvcf_worker_utils1.0.1RequiredProxies NATS traffic between function containers and the control plane.nvcr.io/nvidia/nvcf/nvcf_worker_utils:1.0.1GitHub
pylon0.3.1OptionalConnects LLM worker pods to the LLM request router.nvcr.io/nvidia/nvcf/pylon:0.3.1GitHub

EA-only CVE-impacted artifacts

These Early Access artifacts have known CVE impact. Use only the QA-qualified versions listed for this EA stack.

ArtifactVersionRequiredDescriptionDistributionSource code
bitnami-cassandra5.0.6-nv-1RequiredStores NVCF account, function, cluster, and service state during Early Access.nvcr.io/0833294136851237/selfhosted-ga/bitnami-cassandra:5.0.6-nv-1-eaUpstream
nvcf-cassandra-migrations0.8.1RequiredApplies the Cassandra schemas required by Early Access NVCF services.nvcr.io/0833294136851237/selfhosted-ga/nvcf-cassandra-migrations:0.8.1-eaGitHub

Tools and deployment resources

ArtifactVersionDescriptionDistributionSource code
nvcf-cli1.10.3Manages functions, deployments, and clusters from the command line.nvcr.io/nvidia/nvcf/nvcf-cli:1.10.3GitHub
nvcf-compute-plane-stack1.0.6Provides the Helmfile bundle for compute-plane deployment.nvcr.io/nvidia/nvcf/nvcf-compute-plane-stack:1.0.6
nvcf-self-managed-stack0.6.0Provides the Helmfile bundle for control-plane deployment.nvcr.io/nvidia/nvcf/nvcf-self-managed-stack:0.6.0GitHub