Cloud FunctionsAll required self-hosted NVCF artifacts (see self-hosted-artifact-manifest) must be available to be pulled by pods in your Kubernetes cluster for a successful installation using the helmfile (nvcf-self-managed-stack) automation. This page provides examples on how to pull artifacts from NGC and push them to your desired registry.
Mirroring images is not the same as configuring image pull secrets. This page covers how to copy NVCF artifacts into your registry. If your registry is private, Kubernetes also needs credentials to pull those images at runtime. For instructions on configuring image pull secrets for the NVCF control plane pods, see control-plane-image-pull-secrets in the installation guide.
Use this page to mirror NVCF artifacts into the registry your cluster can access. The examples below show the commands for pulling from NGC and pushing to a private registry such as ECR, Harbor, or another OCI-compatible registry.
For ECR mirroring, make sure your AWS credentials have permission to create repositories and push images. Verify access with aws sts get-caller-identity.
You must have access to the NGC nvcf-onprem organization to begin.
nvcf-onprem organization in the upper right.
If you plan to deploy Low Latency Streaming (LLS), you must mirror the following additional artifacts beyond the core NVCF control plane:
Container Images:
streaming-proxy - Streaming Proxy container for streamingHelm Charts:
gdn-streaming - GDN Streaming Proxy Helm chartOptional (for streaming workloads):
usd-composer)See self-hosted-lls-installation for LLS deployment instructions.
Important: The examples below show how to pull individual artifacts. You must pull each image, chart, and resource listed in the self-hosted-artifact-manifest individually. These examples demonstrate the process for one artifact of each type - you will need to repeat these steps for every artifact required for your deployment.
Complete the following for each artifact:
See the self-hosted-artifact-manifest for the complete list of all required artifacts.
Platform Architecture Mismatch
When pulling images, Docker pulls the architecture matching your local machine by default. If you’re running on an Apple Silicon Mac (arm64) but deploying to an amd64 cluster (most EKS/GKE clusters), you must specify the target platform:
Failing to specify the correct platform will result in exec format error when pods attempt to start. See image-mirroring-troubleshooting for more details.
Login using the Personal API key you have generated in the previous step:
Pull the image (specify platform matching your target cluster):
OCI-compliant Helm Charts
Repository-based Helm Charts (Non-OCI)
Some charts like the GPU Operator and the Omniverse DDCS, UCC, storage-service, and discovery-service charts are available from traditional Helm repositories rather than OCI registries. These can be pulled directly from the public NVIDIA NGC Catalog.
The GPU Operator and related components (gpu-operator-validator, k8s-device-plugin), plus the Omniverse DDCS/UCC/storage charts, are available from public NVIDIA Helm repositories. You can either:
Converting Non-OCI Charts for ECR
To push repository-based Helm charts to Amazon ECR (which requires OCI format), you must convert them:
ECR will properly track both container images and Helm charts under the same repository name and version, so you can use consistent naming for both. The repository prefix (e.g., nvcf-self-hosted) must match your global.image.repository environment configuration.
Using NGC CLI
First, ensure you have the NGC CLI installed and configured using the Personal API key you created.
nvcf-self-managed-stackThe nvcf-self-managed-stack repository contains Helmfile configurations for deploying the NVCF control plane components.
Check for the latest version before downloading. The version shown below is an example only.
Download and extract:
If you don’t have access to this repository, contact your NVIDIA representative.
nvcf-cliThe nvcf-cli is a command-line interface for managing NVIDIA Cloud Functions in self-hosted deployments.
Check for the latest version before downloading. The version shown below is an example only.
Download and extract:
The extracted directory contains:
nvcf-cli - The CLI binary.nvcf-cli.yaml.template - Configuration templateexamples/ - Sample configuration files for different environmentsUSAGE-GUIDE.md - Detailed usage documentationSee self-hosted-cli for detailed configuration instructions
If you don’t have access to this repository, contact your NVIDIA representative.
Ensure all artifacts listed in the self-hosted-artifact-manifest are mirrored to your registry before beginning the installation process.
This example assumes you’re configured and authenticated using the AWS CLI.
Identify Your AWS Account ID
The examples below use <aws-account-id> as a placeholder. To get your AWS account ID, run:
ECR Repository Naming Convention
The Helm templates expect images at: {{ registry }}/{{ repository }}/image-name:tag
For example, with environment configuration:
The resulting image path would be: <aws-account-id>.dkr.ecr.us-east-1.amazonaws.com/nvcf-self-hosted/nvcf-openbao:2.5.1-nv-1.1.0
In ECR, you must create repositories with the full path including the prefix, e.g., nvcf-self-hosted/bitnami-cassandra, nvcf-self-hosted/nvcf-openbao, etc.
Initial Setup
Push an Image to ECR
Push a Helm Chart to ECR
Replace <aws-account-id> with your AWS account ID (run aws sts get-caller-identity --query Account --output text). The REPO_PREFIX value must match your global.image.repository setting in your environment config. Adjust the region as needed.
This example shows how to push images and Helm charts to Volcano Engine Container Registry (CR) using the web console, Docker commands and Helm commands.
Volcano Engine CR Repository Naming Convention
The Helm templates expect images at: {{ registry }}/{{ repository }}/image-name:tag
For example, with environment configuration:
The resulting image path would be: cr-example-cn-beijing.cr.volces.com/nvcf-self-hosted/nvcf-openbao:2.5.1-nv-1.1.0
Docker Authentication
Replace cr-example-cn-beijing.cr.volces.com with your Volcano Engine CR endpoint, your-username with your username, and your-password with your password.
Navigate to your Volcano Engine Container Registry instance web console to get the username and password. If you haven’t set the password, you can set it by clicking “Set Repository Instance Password”.
Push an Image to Volcano Engine CR
Push a Helm Chart to Volcano Engine CR
Symptom: Pods fail to start with Init:CrashLoopBackOff or CrashLoopBackOff status. Checking the logs shows:
or
Cause: This error occurs when container images were pulled/pushed with an architecture that doesn’t match your cluster’s node architecture. This commonly happens when:
Solution:
Delete the incorrectly mirrored images from your registry (e.g., ECR):
Clean local Docker cache to ensure fresh pulls:
Re-mirror images with the correct platform:
When pulling images, explicitly specify the target platform:
Then re-tag and push to your registry.
Force Kubernetes to re-pull images by either:
imagePullPolicy: Always temporarily in your Helm values