AI Enterprise Workload Setup#

Workload deployment task: install the NVIDIA® GPU Operator (optional Kubernetes path) and pull NVIDIA AI Enterprise application containers (Docker, Podman, Cloud Native Stack) onto Guest VMs. This step assumes the Host Setup has completed.

The vGPU for Compute Guest Driver can be installed inside the Guest VM by one of two paths, and the prerequisites differ. Each section below states the prerequisites that apply to it.

Operator-managed

The NVIDIA GPU Operator installs the driver and applies the license. Begin at Installing the NVIDIA GPU Operator Using a Bash Shell Script, on a Guest VM that does not yet have the driver.

Manually installed

Complete Installing NVIDIA vGPU Guest Driver and license the VM first, then deploy containers.

Installing the NVIDIA GPU Operator Using a Bash Shell Script#

A bash script that installs the NVIDIA GPU Operator with the NVIDIA vGPU™ for Compute Driver is available in the NVIDIA AI Enterprise Infra 9 collection.

Prerequisites

  • The Guest VM does not already have the vGPU for Compute Driver installed. The GPU Operator installs that driver, so this path replaces Installing NVIDIA vGPU Guest Driver rather than following it.

  • An NVIDIA License System (NLS) client configuration token has been generated for the Guest VM. The script applies it, so the VM does not need to be licensed beforehand.

  • An NGC API key, used to create the image pull secret for the prebuilt vGPU for Compute Driver image.

Refer to the GPU Operator for deploying the vGPU for Compute Driver with the script.

Installing NVIDIA AI Enterprise Applications Software#

Prerequisites

Before you install any NVIDIA AI Enterprise container:

  • Guest OS is supported.

  • The VM has a valid vGPU for Compute license (refer to Licensing vGPU VMs).

  • At least one NVIDIA GPU is visible to the system.

  • The vGPU for Compute Guest Driver is installed; nvidia-smi lists the GPU.

Installing Docker Engine

Install Docker for your Guest VM Linux distribution using the official Docker Installation Guide.

Installing the NVIDIA Container Toolkit

The NVIDIA Container Toolkit adds a runtime and helpers so Docker containers use NVIDIA GPUs automatically. Enable the Docker repo and install the toolkit on the Guest VM per Installing the NVIDIA Container Toolkit.

Then configure the Docker runtime using Configuration.

Verifying the Installation: Run a Sample CUDA® Container

Run a sample CUDA container test on the GPU per Running a Sample Workload.

Accessing NVIDIA AI Enterprise Containers on NGC™

NVIDIA AI Enterprise application images live in the NVIDIA NGC Catalog under the NVIDIA AI Enterprise Supported label.

Each image ships the user-space stack for that workload: CUDA libraries, cuDNN™, Magnum IO™ where needed, TensorRT™, and the framework.

  1. Create an NGC API key using the catalog URL NVIDIA provides.

  2. Authenticate with Docker to NGC Registry. In your shell, run:

    docker login nvcr.io
    Username: $oauthtoken
    Password: <paste-your-NGC_API_key-here>
    
    A successful login (``Login Succeeded``) lets you pull containers from NGC.
    
  3. From the NVIDIA vGPU for Compute VM, browse the NGC Catalog for containers labeled NVIDIA AI Enterprise Supported.

  4. Copy the relevant docker pull command.

    sudo docker pull nvcr.io/nvaie/rapids-pb25h1:x.y.z-runtime
    

    Where x.y.z is the version of your container.

  5. Run the container with GPU access.

    sudo docker run --gpus all -it --rm nvcr.io/nvaie/rapids-pb25h1:x.y.z-runtime
    

    Where x.y.z is the version of your container.

    This starts an interactive session with all vGPUs on the Guest VM exposed to the container.

Podman (a Docker alternative) follows a similar install flow for NVIDIA AI Enterprise containers. Refer to NVIDIA AI Enterprise: RHEL with KVM Deployment Guide.

Cloud Native Stack (CNS) bundles Ubuntu or RHEL, Kubernetes, Helm, and the NVIDIA GPU and Network Operator for cloud-native GPU workloads.

Use the repository installation guides for OS-specific steps and for deploying an NGC Catalog app to validate GPU access.