Is this page helpful?

AI Enterprise Workload Setup#

Workload deployment task: install the NVIDIA GPU Operator (optional Kubernetes path) and pull NVIDIA AI Enterprise application containers (Docker, Podman, Cloud Native Stack) onto licensed Guest VMs. This step assumes Host Setup and Installing NVIDIA vGPU Guest Driver have completed and the VM is licensed.

Installing the NVIDIA GPU Operator Using a Bash Shell Script#

A bash script that installs the NVIDIA GPU Operator with the NVIDIA vGPU for Compute Driver is available in the NVIDIA AI Enterprise Infra 8 collection.

Note

Use this path only when the Guest VM does not already have the vGPU for Compute Driver; the GPU Operator installs that driver.

Refer to the GPU Operator for deploying the vGPU for Compute Driver with the script.

Installing NVIDIA AI Enterprise Applications Software#

Prerequisites

Before you install any NVIDIA AI Enterprise container:

  • Guest OS is supported.

  • The VM has a valid vGPU for Compute license (refer to Licensing vGPU VMs).

  • At least one NVIDIA GPU is visible to the system.

  • The vGPU for Compute Guest Driver is installed; nvidia-smi lists the GPU.

Installing Docker Engine

Install Docker for your Guest VM Linux distribution using the official Docker Installation Guide.

Installing the NVIDIA Container Toolkit

The NVIDIA Container Toolkit adds a runtime and helpers so Docker containers use NVIDIA GPUs automatically. Enable the Docker repo and install the toolkit on the Guest VM per Installing the NVIDIA Container Toolkit.

Then configure the Docker runtime using Configuration.

Verifying the Installation: Run a Sample CUDA Container

Run a sample CUDA container test on the GPU per Running a Sample Workload.

Accessing NVIDIA AI Enterprise Containers on NGC

NVIDIA AI Enterprise application images live in the NVIDIA NGC Catalog under the NVIDIA AI Enterprise Supported label.

Each image ships the user-space stack for that workload: CUDA libraries, cuDNN, Magnum IO where needed, TensorRT, and the framework.

  1. Create an NGC API key using the catalog URL NVIDIA provides.

  2. Authenticate with Docker to NGC Registry. In your shell, run:

    docker login nvcr.io
Username: $oauthtoken
Password: <paste-your-NGC_API_key-here>

A successful login (``Login Succeeded``) lets you pull containers from NGC.

  3. From the NVIDIA vGPU for Compute VM, browse the NGC Catalog for containers labeled NVIDIA AI Enterprise Supported.

  4. Copy the relevant docker pull command.

    sudo docker pull nvcr.io/nvaie/rapids-pb25h1:x.y.z-runtime

    Where x.y.z is the version of your container.

  5. Run the container with GPU access.

    sudo docker run --gpus all -it --rm nvcr.io/nvaie/rapids-pb25h1:x.y.z-runtime

    Where x.y.z is the version of your container.

    This starts an interactive session with all vGPUs on the Guest VM exposed to the container.

Podman (a Docker alternative) follows a similar install flow for NVIDIA AI Enterprise containers. See NVIDIA AI Enterprise: RHEL with KVM Deployment Guide.

Cloud Native Stack (CNS) bundles Ubuntu or RHEL, Kubernetes, Helm, and the NVIDIA GPU and Network Operator for cloud-native GPU workloads.

Use the repository installation guides for OS-specific steps and for deploying an NGC Catalog app to validate GPU access.