Is this page helpful?

Quick Start Guide#

Deploy and configure NVIDIA AI Enterprise in 30-60 minutes. This guide covers bare-metal, public cloud, and virtualized deployments.

What You’ll Learn

  • Activate your NVIDIA Enterprise Account and access NGC Catalog

  • Install NVIDIA AI Enterprise software components

  • Deploy and verify GPU-accelerated containers

  • Run your first AI workload

For detailed deployment instructions, refer to the NVIDIA AI Enterprise Deployment Guide.

Note

These instructions don’t apply to NVIDIA DGX systems. Refer to NVIDIA DGX Systems for DGX-specific documentation.

Attention

Already have an account? If you previously purchased NVIDIA AI Enterprise and have an NVIDIA Enterprise Account, skip to Installing NVIDIA AI Enterprise Software Components.

Activating the Accounts for NVIDIA AI Enterprise#

After your order for NVIDIA AI Enterprise is processed, you’ll receive an order confirmation message. This message contains the information you need to access NVIDIA AI Enterprise and technical support from NVIDIA. To access NVIDIA AI Enterprise and technical support from NVIDIA, you must have an NVIDIA Enterprise Account, which provides login access to the following NVIDIA software components:

  • NVIDIA NGC - provides access to all enterprise software, services, and management tools included in NVIDIA AI Enterprise

  • NVIDIA Enterprise Support Portal - provides access to support services for NVIDIA AI Enterprise

  • NVIDIA Licensing Portal - provides access to your entitlements and options for managing your NVIDIA AI Enterprise license servers. Installing and managing NVIDIA AI Enterprise license servers is a prerequisite for deploying NVIDIA vGPU for Compute.

These components can be accessed from the NVIDIA Application Hub. To activate your account and access NVIDIA AI Enterprise, follow these steps:

Before You Begin#

Before you start, ensure the following prerequisites are met:

Hardware Requirements

Software & Licensing

  • Valid NVIDIA AI Enterprise software subscription

  • For bundled GPUs (such as NVIDIA H100 PCIe), activate your license before installation

Additional Information

Refer to the NVIDIA AI Enterprise Release Notes for supported hardware, software versions, and known issues.

Your Order Confirmation Message#

After your order for NVIDIA AI Enterprise is processed, you’ll receive an order confirmation message with your NVIDIA Entitlement Certificate attached. The certificate contains your product activation keys and provides instructions for using it.

If you’re a data center administrator, follow the instructions in the NVIDIA Entitlement Certificate to use the certificate. Otherwise, forward your order confirmation message, including the attached NVIDIA Entitlement Certificate, to a data center administrator in your organization.

NVIDIA Enterprise Account Requirements#

You need an NVIDIA Enterprise Account to access NVIDIA AI Enterprise software and technical support.

Choose Your Scenario

  1. First-time purchase - Follow the Register link in your NVIDIA Entitlement Certificate to create your NVIDIA Enterprise Account.

  2. Previous purchaser - You already have an NVIDIA Enterprise Account. Download software from the NVIDIA AI Enterprise Infra 7 collection on NVIDIA NGC. Refer to Accessing the NVIDIA AI Enterprise AI and Application Software.

  3. Evaluation to purchased license - You have two options:

Creating your NVIDIA Enterprise Account#

Create an NVIDIA Enterprise Account to access NVIDIA AI Enterprise software and technical support. For component details, refer to Activating the Accounts for Getting NVIDIA AI Enterprise.

Before You Begin

  • Have your order confirmation message with the NVIDIA Entitlement Certificate ready

  • Choose a unique email address (if creating a new account separate from an evaluation account)

Steps

  1. Follow the Register link in the instructions for using your NVIDIA Entitlement Certificate.

  2. Fill out the NVIDIA Enterprise Account Registration page form and click REGISTER. A message confirming that an account has been created will appear. The email address you provided will then receive an email instructing you to log in to your account on the NVIDIA Application Hub.

  3. Open the email instructing you to log in to your account and click Log In.

  4. On the open NVIDIA Application Hub Login page, type the email address you provided in the text-entry field and click Sign In.

  5. On the Create Your Account page that opens, provide and confirm a password for the account and click Create Account. A message prompting you to verify your email address appears. An email instructing you to verify your email address is sent to your provided email address.

  6. Open the email instructing you to verify your email address and click Verify Email Address. A message confirming that your email address is confirmed appears.

From the NVIDIA Application Hub page, you can now log in to the components listed in Activating the Accounts for Getting NVIDIA AI Enterprise.

Linking an Evaluation Account to an NVIDIA Enterprise Account for Purchased Licenses#

Link your evaluation account to purchased licenses by registering with the same email address you used for your evaluation account. To create a separate account instead, see Creating your NVIDIA Enterprise Account and use a different email address.

  1. Follow the Register link in the instructions for using the NVIDIA Entitlement Certificate for your purchased licenses.

  2. Fill out the NVIDIA Enterprise Account Registration page form, specifying the email address with which you created your existing account, and click Register.

    NVIDIA Enterprise Account Registration page

  3. When a message stating that your email address is already linked to an evaluation account is displayed, click LINK TO NEW ACCOUNT.

    Link To New Account page

  4. Log in to the NVIDIA Licensing Portal with the credentials for your existing account.

Installing NVIDIA AI Enterprise Software Components#

The NVIDIA NGC Catalog#

NVIDIA AI Enterprise components are distributed through the NVIDIA NGC Catalog. Infrastructure and workload management components are available in the NVIDIA AI Enterprise Infra 7 collection. Tools for AI development and use cases are available from the NVIDIA AI Enterprise Software Suite.

Accessing the NVIDIA AI Enterprise Infrastructure Software#

Infrastructure and workload management components of NVIDIA AI Enterprise are distributed as resources in the NVIDIA AI Enterprise Infra 7 collection.

The NVIDIA AI Enterprise Infra 7 collection includes infrastructure management and orchestration software to manage and scale AI workloads.

Table 66 NVIDIA AI Enterprise Infrastructure Software#

Component

Description

Category

Core Infrastructure Drivers

NVIDIA GPU Data Center Driver

Containerized GPU driver for bare metal and Kubernetes deployments

Infrastructure Driver

NVIDIA DOCA-OFED Driver for Networking

High-performance networking for InfiniBand and Ethernet

Infrastructure Driver

Virtualization

NVIDIA Virtual GPU Manager

Hypervisor-based GPU virtualization and partitioning

Virtualization

NVIDIA vGPU for Compute Guest Driver

Guest VM driver for virtualized GPU access

Virtualization

Kubernetes Operators

NVIDIA GPU Operator

Automates GPU driver and toolkit lifecycle in Kubernetes

Kubernetes Orchestration

NVIDIA Network Operator

Manages networking components for GPU workloads

Kubernetes Orchestration

NVIDIA DPU Operator (DPF)

DOCA Platform Framework for data processing units

Kubernetes Orchestration

NVIDIA NIM Operator

Deploys and manages NVIDIA Inference Microservices in Kubernetes

Kubernetes Orchestration

Cluster Management & Orchestration

NVIDIA Base Command Manager

Enterprise cluster provisioning and workload orchestration

Cluster Management

Before downloading any NVIDIA AI Enterprise software assets, ensure you have signed in to NVIDIA NGC from the NVIDIA NGC Sign In page.

  1. Go to the NVIDIA AI Enterprise Infra 7 collection on NVIDIA NGC.

  2. Click the Artifacts tab and select the resource.

  3. Click Download and choose to download the resource using a direct download in the browser, the displayed wget command, or the NVIDIA NGC CLI.

Accessing the NVIDIA AI Enterprise AI and Application Software#

Tools for AI development and use cases are available from the NVIDIA AI Enterprise Software Suite and are distributed through the NVIDIA NGC Catalog.

Before downloading any NVIDIA AI Enterprise software assets, ensure you have signed in to NVIDIA NGC from the NVIDIA NGC Sign In page.

  1. View the NVIDIA AI Enterprise Software Suite on NVIDIA NGC.

  2. Browse the NVIDIA AI Enterprise Software Suite to find software assets.

  3. Click an asset to learn more about it or download it.

NVIDIA AI Enterprise Deployment Options#

Choose a deployment approach based on your infrastructure and performance requirements. Each option includes getting started instructions, comprehensive deployment guides, and reference tutorials.

Table 67 NVIDIA AI Enterprise Deployment Options#

Category

Getting Started

Deployment Guide

Reference Tutorials

Deploy on bare metal

NVIDIA AI Enterprise Bare Metal Deployment Guide

Deploy on the public cloud

Installing NVIDIA AI Enterprise on Microsoft Azure

NVIDIA AI Enterprise Cloud Deployment Guide

Running an LLM NIM on Microsoft Azure using NVIDIA AI Enterprise

Deploy in virtualized environments using NVIDIA vGPU for Compute

Installing NVIDIA AI Enterprise on VMware vSphere

NVIDIA AI Enterprise VMware Deployment Guide

Installing NVIDIA AI Enterprise on Bare Metal Ubuntu 22.04#

This section provides instructions for a bare-metal, single-node deployment of NVIDIA AI Enterprise using Docker on an NVIDIA-Certified Systems.

NVIDIA AI Enterprise Software Prerequisites#

To enable NVIDIA GPU acceleration for compute and AI workloads running in data centers:

  1. Download the NVIDIA GPU data center drivers from NVIDIA Drivers.

  2. Within the Manual Driver Search section, select Data Center/Tesla, your architecture type, and Linux 64-bit to download the .run file.

Installing NVIDIA AI Enterprise using the TRD Driver on Ubuntu 22.04 from a .run File#

Installation of the NVIDIA AI Enterprise software driver for Linux requires:

  • Compiler toolchain

  • Kernel headers

Prerequisites

Ensure you follow the CUDA pre-installation steps.

Note

If you prefer to use a Debian package to install CUDA, refer to the Debian instructions.

Steps

  1. Log into the system and check for updates.

    sudo apt-get update

  2. Install the GCC compiler and the make tool in the terminal.

    sudo apt-get install build-essential

  3. Copy the NVIDIA AI Enterprise Linux driver package, for example, NVIDIA-Linux-x86_64-xxx.xx.xx.run, to the host machine where you are installing the driver.

    Where xxx.xx.xx is the current NVIDIA AI Enterprise version and driver version.

  4. Navigate to the directory containing the NVIDIA Driver .run file. Then, add the Executable permission to the NVIDIA Driver file using the chmod command.

    sudo chmod +x NVIDIA-Linux-x86_64-xxx.xx.xx.run

  5. From a console shell, run the driver installer as the root user and accept the defaults.

    sudo sh ./NVIDIA-Linux-x86_64-xxx.xx.xx.run

  6. Reboot the system.

    sudo reboot

  7. After the system has rebooted, confirm that you can see your NVIDIA GPU device in the output from nvidia-smi.

    # Verify GPU driver installation and GPU detection
$ nvidia-smi

Expected output: GPU information table showing driver version 580.126.09, CUDA version, GPU name (e.g., H100, A100), and memory usage. If no output appears or you see command not found, verify the driver installation completed successfully.

Installing the NVIDIA Container Toolkit#

The NVIDIA Container Toolkit enables building and running GPU-accelerated Docker containers. It includes a libnvidia-container library and utilities that configure containers to use NVIDIA GPUs automatically. For more information, refer to the NVIDIA Container Toolkit documentation.

NVIDIA Container Toolkit

  1. Install Docker - Refer to the Install Docker Engine on Ubuntu documentation for the installation procedure for Ubuntu.

  2. Install the NVIDIA Container Toolkit - To enable the Docker repository and install the NVIDIA Container Toolkit, refer to the Installing the NVIDIA Container Toolkit documentation.

  3. After installing the NVIDIA Container Toolkit, refer to the Configuration documentation to configure the Docker container runtime.

Verifying the Installation of NVIDIA Container Toolkit#

  1. Run the nvidia-smi command contained in the latest official NVIDIA CUDA Toolkit image that is compatible with the release of the NVIDIA CUDA Toolkit driver running on your machine.

    Note

    Do not use a release of the NVIDIA CUDA Toolkit image later than the release of the NVIDIA CUDA Toolkit driver that is running on your machine. For a list of all NVIDIA CUDA Toolkit images, refer to nvidia/cuda on Docker Hub.

    # Test GPU access from container (uses all available GPUs)
# --rm: automatically remove container after exit
# --runtime=nvidia: use NVIDIA Container Runtime
# --gpus all: expose all GPUs to container
$ sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

    Tip

    Troubleshooting: If you encounter nvidia-container-cli: initialization error, verify:

    1. NVIDIA driver is loaded: nvidia-smi returns successfully on host

    2. Container toolkit is installed: which nvidia-container-cli

    3. Docker daemon restarted: sudo systemctl restart docker

  2. Start a GPU-enabled container on any two available GPUs.

    $ docker run --runtime=nvidia --gpus 2 nvidia/cuda:xx.x.x-base-ubuntu22.04 nvidia-smi

    Where nvidia/cuda:xx.x.x is the current supported CUDA Toolkit version.

  3. Start a GPU-enabled container on two specific GPUs identified by their index numbers.

    $ docker run --runtime=nvidia --gpus '"device=1,2"' nvidia/cuda:xx.x.x-base-ubuntu22.04 nvidia-smi

    Where nvidia/cuda:xx.x.x is the current supported CUDA Toolkit version.

  4. Start a GPU-enabled container on two specific GPUs, with one GPU identified by its UUID and the other GPU identified by its index number.

    $ docker run --runtime=nvidia --gpus '"device=UUID-ABCDEF,1"' nvidia/cuda:xx.x.x-base-ubuntu22.04 nvidia-smi

    Where nvidia/cuda:xx.x.x is the current supported CUDA Toolkit version.

  5. Specify a GPU capability for the container.

    $ docker run --runtime=nvidia --gpus all,capabilities=utility nvidia/cuda:xx.x.x-base-ubuntu22.04 nvidia-smi

    Where nvidia/cuda:xx.x.x is the current supported CUDA Toolkit version.

Installing Software Distributed as Container Images#

The NGC container images accessed through the NVIDIA NGC Catalog include AI and data science applications and frameworks. Each container image contains the entire user-space software stack required to run the application or framework, including the CUDA libraries, cuDNN, required Magnum IO components, TensorRT, and the framework.

Ensure that you have completed the following tasks in the NGC Private Registry User Guide:

Perform this task from the host machine. Obtain the Docker pull command to download each of the following applications and deep learning framework components from the listing for the application or component in the NGC Catalog.

Refer to the NGC supported software page for the complete list.

Running an NVIDIA Production Branch PyTorch Container#

  1. Pull the PyTorch container from NGC. Use Docker to pull the official NVIDIA PyTorch container:

    sudo docker pull nvcr.io/nvidia/pytorch-pb25h1:25.03.05-py3

  2. Run the PyTorch container interactively and access its environment.

    sudo docker run --gpus all -it --rm nvcr.io/nvidia/pytorch-pb25h1:25.03.05-py3

    Additional options for mounting host directories:

    sudo docker run --gpus all -it --rm -v $(pwd)/data:/data nvcr.io/nvidia/pytorch-pb25h1:25.03.05-py3

    Add recommended performance flags:

    sudo docker run --gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 -it --rm nvcr.io/nvidia/pytorch-pb25h1:25.03.05-py3

  3. Verify PyTorch and GPU access.

    1. Inside the container, start Python.

      # python

    2. Check PyTorch and CUDA availability.

      import torch
print(torch.cuda.is_available())

      This should print True if GPU access is working.

  4. Run your PyTorch code. Place your code and datasets in a mounted host directory for easy access (for example, /data). Run your scripts as usual inside the interactive shell.

Running ResNet-50 with TensorRT#

  1. Launch the NVIDIA TensorRT container image on all GPUs in interactive mode, specifying that the container will be deleted when stopped.

    $ sudo docker run --gpus all -it --rm nvcr.io/nvidia/tensorrt:xx.xx-py3

    Where xx.xx is the container version. For example, 25.03.

  2. From within the container runtime, change to the directory that contains test data for the ResNet-50 convolutional neural network.

    # cd /workspace/tensorrt/data/resnet50

  3. Run the ResNet-50 convolutional neural network with FP32, FP16, and INT8 precision and confirm that each test is completed with the result PASSED.

    1. To run ResNet-50 with the default FP32 precision, run this command:

      # trtexec --onnx=ResNet50.onnx \
--duration=90 --avgRuns=100 --percentile=99 \
--memPoolSize=workspace:1024

    2. To run ResNet-50 with FP16 precision, add the --fp16 option:

      # trtexec --onnx=ResNet50.onnx \
--duration=90 --avgRuns=100 --percentile=99 \
--memPoolSize=workspace:1024 --fp16

    3. To run ResNet-50 with INT8 precision, add the --int8 option:

      # trtexec --onnx=ResNet50.onnx \
--duration=90 --avgRuns=100 --percentile=99 \
--memPoolSize=workspace:1024 --int8

  4. Press Ctrl+P > Ctrl+Q to exit the container runtime and return to the Linux command shell.

Running NVIDIA NIM on Bare Metal Ubuntu 22.04#

NVIDIA Inference Microservices (NIM) provides a path for developing AI-powered enterprise applications and deploying AI models in production. You can download and run NIM from the NVIDIA NGC Catalog. The following instructions deploy a Llama3 8B Instruct NIM on your bare metal host machine and run inference.

  1. Pull and run meta/llama3-8b using Docker (this will download the full model and run it in your local environment).

    $ docker login nvcr.io
Username: $oauthtoken
Password: <PASTE_API_KEY_HERE>

  2. Pull and run NVIDIA NIM. This will download the optimized model for your infrastructure.

    export NGC_API_KEY=<PASTE_API_KEY_HERE>
export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p "$LOCAL_NIM_CACHE"
docker run -it --rm \
    --gpus all \
    --shm-size=16GB \
    -e NGC_API_KEY=$NGC_API_KEY \
    -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
    -u $(id -u) \
    -p 8000:8000 \
    nvcr.io/nim/meta/llama3-8b-instruct:1.0.0

  3. Make a local API call.

    curl -X 'POST' \
'http://0.0.0.0:8000/v1/chat/completions' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
    "model": "meta/llama3-8b-instruct",
    "messages": [{"role":"user", "content":"Write a limerick about the wonders of GPU computing."}],
    "max_tokens": 64
}'

For more information about running inference on this locally deployed LLM NIM, refer to Launch NVIDIA NIM for LLMs.

Installing NVIDIA AI Enterprise on Public Cloud#

NVIDIA AI Enterprise can be run on Amazon Web Services (AWS), Google Cloud, Microsoft Azure, Oracle Cloud Infrastructure (OCI), Alibaba Cloud, and Tencent Cloud.

This NVIDIA AI Enterprise Quick Start Guide section provides minimal instructions for deploying NVIDIA AI Enterprise on Microsoft Azure using the NVIDIA AI Enterprise VMI.

Installing NVIDIA AI Enterprise on Microsoft Azure using the NVIDIA AI Enterprise VMI#

The NVIDIA AI Enterprise On-Demand VMI generates a GPU-accelerated virtual machine instance in minutes with pre-installed software to accelerate Machine Learning, Deep Learning, Data Science, and HPC workloads.

The On-Demand VMI is preconfigured with the following software:

  • Ubuntu Operating System

  • NVIDIA GPU Data Center Driver

  • Docker-ce

  • NVIDIA Container Toolkit

  • CSP CLI, NGC CLI

  • Miniconda, JupyterLab, Git

  • Token Activation Script

Getting started with NVIDIA AI Enterprise in your Enterprise (On-Demand) VMI cloud instance requires two steps:

  1. Authorize the VMI cloud instance with NVIDIA NGC by copying over the provided instance ID token into the Activate Subscription page on NGC. There are four key steps to complete this part of the process:

    1. Get an identity token from the VMI.

    2. Activate your NVIDIA AI Enterprise subscription with the token.

    3. Generate an API key to access the catalog.

    4. Put the API key on the VMI.

    Follow the NGC Catalog access instructions to complete this step.

  2. Pulling and running NVIDIA AI Enterprise Containers. Refer to this section of the Cloud Deployment Guide for pulling and running NGC container images through the NVIDIA NGC Catalog.

Detailed instructions on installing NVIDIA AI Enterprise on the public cloud can be found in the Microsoft Azure Overview.

Running an LLM NIM on Microsoft Azure using NVIDIA AI Enterprise#

Refer to the Appendix section of the Cloud Deployment Guide for pulling and running NGC container images through the NVIDIA NGC Catalog.

Installing NVIDIA AI Enterprise in Virtualized Environments using NVIDIA vGPU for Compute#

The NVIDIA AI Enterprise VMware Deployment Guide offers detailed information for deploying NVIDIA AI Enterprise on a third-party NVIDIA-Certified Systems running VMware vSphere using NVIDIA vGPU for Compute.

NVIDIA AI Enterprise Software Prerequisites#

  1. NVIDIA Virtual GPU Manager

  2. NVIDIA vGPU for Compute Guest Driver

  3. NVIDIA License System

To download the NVIDIA vGPU for Compute software drivers, follow the instructions for Accessing the NVIDIA AI Enterprise Infrastructure Software.

The NVIDIA AI Enterprise license entitles customers to download NVIDIA vGPU for Compute software through NVIDIA NGC. Deploying NVIDIA AI Enterprise using vGPU requires a valid license enforced by installing the NVIDIA License System. The NVIDIA License System is a pool of floating licenses for licensed NVIDIA software products configured with licenses obtained from the NVIDIA Licensing Portal.

NVIDIA License System supports the following types of service instances:

  • Cloud License Service (CLS) instance. A CLS instance is hosted on the NVIDIA Licensing Portal.

  • Delegated License Service (DLS) instance. A DLS instance is hosted on-premises at a location accessible from your private network, such as inside your data center.

An NVIDIA vGPU for Compute client VM with a network connection obtains a license by leasing it from an NVIDIA License System service instance. The service instance serves the license to the client over the network from a pool of floating licenses obtained from the NVIDIA Licensing Portal. The license is returned to the service instance when the licensed client no longer requires the license.

To activate an NVIDIA vGPU for Compute, software licensing must be configured for the vGPU VM client when booted. NVIDIA vGPU for Compute VMs run at a reduced capability until a license is acquired.

Refer to the NVIDIA Licensing Quick Start Guide for instructions on configuring an express Cloud License Service (CLS) instance and verifying the license status of a licensed vGPU VM client.

If you use Delegated License Service (DLS) instances to serve licenses, refer to the NVIDIA License System User Guide. It provides detailed instructions on installing, configuring, and managing the NVIDIA License System.

Obtaining NVIDIA Base Command Manager#

NVIDIA Base Command Manager streamlines cluster provisioning, workload management, and infrastructure monitoring. In bare-metal deployments, it simplifies the installation of supported operating systems.

Before obtaining NVIDIA Base Command Manager, ensure you have activated the accounts for NVIDIA AI Enterprise, as explained in Activating the Accounts for Getting NVIDIA AI Enterprise.

  1. Request your NVIDIA Base Command Manager product keys by emailing your entitlement certificate to sw-bright-sales-ops@NVIDIA.onmicrosoft.com. After your entitlement certificate has been reviewed, you will receive a product key from which you can generate a license key for the number of licenses you purchased.

  2. Download NVIDIA Base Command Manager for your operating system.

The Base Command Manager product manuals provide detailed instructions on deploying and using Base Command Manager.

After obtaining NVIDIA Base Command Manager, follow the steps in the NVIDIA Base Command Manager Installation Manual to create and license your head node.