Is this page helpful?

Configure GPU Settings for Project Container#

Overview#

AI Workbench lets you request GPUs for a project container.

Project GPU settings are environment variables that get their own UI/UX to simplify things. You can assign none, some or all of the GPUs on a given machine to a project.

Requesting GPUs for a project is a declaration of intent, not a guarantee.

You can request GPUs for any project, even if the container doesn’t have CUDA installed. However, in that case trying to mount the GPU at runtime will fail.

If a project container has requested GPUs, starting it triggers Workbench to check for available GPUs on that host. If there are insufficient GPUs or the container does not have CUDA, AI Workbench will notify you that it can’t satisfy the request.

Otherwise, it will run the container and mount the requested GPUs.

GPU requests are written to the *spec.yaml* file and versioned.

Versioned GPU assignments travel with a project to different machines and users.

Key Concepts#

GPU Request

The number of GPUs (0-8) you want allocated to your project container.

Shared Memory

Memory allocated for inter-GPU communication, required when using multiple GPUs.

Request GPUs for Project Container#

Prerequisites: The *spec.yaml* file must have a declared CUDA version.

AI Workbench activates the GPU reservation field based on the CUDA metadata for the project. If the spec.yaml file doesn’t indicate CUDA in the container, the feature will not activate.

Step One: Go to the Hardware configuration section in the project.

  1. Go to Location Manager > Project Card

  2. Go to Project Tab > Environment > Project Container > Hardware

Step Two: Set the desired number of GPUs for the project.

  1. Enter between 0 and 8 in Hardware > Number of Desired GPUs

  2. (optional) If more than one GPU, set the shared memory.

Step Three: (optional) Start or restart the container to mount the GPUs.

  • Launch an application in the Application Launcher

  • Restart the container with Project Tab > Environment > Project Container > Restart

Step Four: (optional) Commit the changes.

  1. Select Commit

  2. Edit the commit message

  3. Select Create a Commit > Commit