GuidesInference & Deployment

NGC Developer Guide

View as Markdown

NGC is NVIDIA’s hub for GPU-optimized AI software, serving as a GPU-optimized container registry for GPU-accelerated containers, pre-trained models, and AI toolkits specifically optimized for NVIDIA hardware.

Every asset in NGC is tested for security and validated across different GPUs for performance and scalability. This isn’t just a repository - it’s a curated, enterprise-grade catalog.

What’s in the NGC Catalog?

Asset TypeWhat You GetExample Use Case
ContainersGPU-optimized Docker imagesPyTorch, TensorFlow, RAPIDS
Pre-trained ModelsReady-to-deploy AI modelsNeMo LLMs, computer vision models
Helm ChartsKubernetes deployment configsDeploy inference services on K8s
ResourcesDatasets, Jupyter notebooksTraining data, example workflows
SDKsDomain-specific toolkitsHealthcare, autonomous vehicles
CollectionsCurated bundlesEnd-to-end AI pipelines

Browse the catalog at catalog.ngc.nvidia.com.

Quick Start

1

Create an NGC Account

Go to ngc.nvidia.com and sign up with your NVIDIA account (or corporate SSO).

2

Generate an API Key

In the NGC web UI:

  1. Click your profile → Setup
  2. Generate API Key
  3. Save it securely - you’ll only see it once!
3

Install NGC CLI

# Using pip (recommended)
pip install ngc-cli
# Or download directly
wget -O ngc https://api.ngc.nvidia.com/v2/resources/nvidia/ngc-apps/ngc_cli/versions/3.31.0/files/ngccli_linux.zip
unzip ngccli_linux.zip
chmod +x ngc-cli/ngc
export PATH=$PATH:$(pwd)/ngc-cli
4

Configure Authentication

# Interactive setup
ngc config set
# Or set environment variables
export NGC_API_KEY=<your-api-key>
export NGC_ORG=<your-org>  # Optional: for enterprise users

For a Brev Launchable, save the API key as an organization secret and bind it to a Text launch parameter instead of storing it in source code or a parameter default.

5

Pull Your First Container

# Pull PyTorch container optimized for GPUs
docker pull nvcr.io/nvidia/pytorch:24.01-py3
# Run it
docker run --gpus all -it nvcr.io/nvidia/pytorch:24.01-py3

Core Developer Workflows

Pulling Containers

# Format: nvcr.io/<organization>/<repository>:<tag>
# NVIDIA's public containers
docker pull nvcr.io/nvidia/pytorch:24.01-py3
docker pull nvcr.io/nvidia/tensorflow:24.01-tf2-py3
docker pull nvcr.io/nvidia/tritonserver:24.01-py3
# Partner containers (ISV)
docker pull nvcr.io/isv-ngc-partner/company/container:version

Downloading Models

# List available models
ngc registry model list
# Download a specific model
ngc registry model download-version nvidia/nemo/megatron_gpt_345m:1.0
# Download to specific directory
ngc registry model download-version nvidia/nemo/megatron_gpt_345m:1.0 --dest ./models/

Using Resources (Datasets, Notebooks)

# List resources
ngc registry resource list
# Download a resource
ngc registry resource download-version nvidia/tao/tao_getting_started:1.0

Working with Helm Charts

# Add NGC Helm repo
helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
# Search for charts
helm search repo nvidia
# Install a chart (e.g., GPU Operator)
helm install gpu-operator nvidia/gpu-operator \
  --set driver.enabled=true \
  --namespace gpu-operator \
  --create-namespace

NGC CLI Power Commands

# Search the catalog
ngc registry image list --format_type csv | grep pytorch
# Get detailed info about an image
ngc registry image info nvidia/pytorch:24.01-py3
# List all versions of a model
ngc registry model list --org nvidia --name nemo
# Check your orgs and teams
ngc org list
ngc team list
# Upload to private registry (enterprise)
ngc registry image push my-org/my-team/my-container:v1.0
# Scripting: JSON output for automation
ngc registry image list --format_type json | jq '.[] | .name'

Registry Architecture

Key URLs:

  • nvcr.io - Main registry for pulling containers
  • catalog.ngc.nvidia.com - Web UI for browsing
  • api.ngc.nvidia.com - REST API endpoint

Deployment Options

EnvironmentHow to Use NGC
Local/DGXDirect docker pull nvcr.io/...
AWSNGC containers on EC2 GPU instances, SageMaker integration.
GCPNGC containers on Compute Engine with GPUs.
AzureNGC containers on Azure VMs with NVIDIA GPUs.
KubernetesNGC Helm charts, GPU Operator.
EdgeNGC containers optimized for Jetson.

Private Registry (Enterprise)

For enterprise users, NGC offers private registries to host your own containers and models:

# Push to private registry
docker tag my-container:latest nvcr.io/my-org/my-team/my-container:v1.0
docker push nvcr.io/my-org/my-team/my-container:v1.0
# Manage access
ngc org user add --org my-org --email colleague@company.com --role REGISTRY_USER

Roles:

  • REGISTRY_USER - Can pull from private registry
  • REGISTRY_READ - Read-only access
  • ADMIN - Full management access

Integration with NVIDIA Tools

ToolNGC Integration
NVIDIA AI WorkbenchPull models/containers directly from NGC.
NeMoPre-trained LLMs hosted on NGC.
Triton Inference ServerContainer and model hosting.
TAO ToolkitPre-trained vision models.
RAPIDSGPU-accelerated data science containers.
DeepStreamVideo analytics containers.

Best Practices

NVIDIA updates containers monthly. Use specific tags (:24.01-py3) in production, not :latest.

All NGC containers pass security scanning. Check the “Security” tab in the catalog for CVE reports.

Download containers and models to a local registry for air-gapped deployments.

Use NGC CLI in pipelines:

ngc registry image pull nvidia/pytorch:24.01-py3 --format_type json

NGC containers are optimized for multi-GPU setups out of the box.

ResourceURL
NGC Catalogcatalog.ngc.nvidia.com
NGC Documentationdocs.nvidia.com/ngc
NGC CLI Docsdocs.ngc.nvidia.com/cli
NGC User GuideNGC User Guide
Public Cloud DeploymentNGC Deploy Guide
Private Registry GuidePrivate Registry Guide

Summary

NGC = GPU-optimized container registry + model hub + Helm charts

# Get started in 3 commands:
pip install ngc-cli
ngc config set  # Enter your API key
docker pull nvcr.io/nvidia/pytorch:24.01-py3

You’re ready to start building.