GPU Configuration#

Overview of GPU Configuration#

  • You install AI Workbench on a system that has NVIDIA GPUs

  • You create or clone a project with a GPU enabled base image, i.e. with CUDA and a framework like PyTorch or TensorFlow installed

  • You start working in the project and computing with a GPU

Workbench provides a managed layer between your development environment and the underlying GPU hardware. It automatically detects the number and type of available GPUs and lets you request and configure them on a project by project basis.

Note

AI Workbench will install or update all of the necesary software required to use NVIDIA GPUs. You do not need to install anything.

The only exception to this is on Windows where you must install the NVIDIA drivers yourself.

Project Containers vs Multi-Container (Compose)#

Workbench handles GPUs for both single and multi-container development environments, but the user experience is different.

Major Differences Are:

  • You can request GPUs for a project container in the Desktop App or CLI

  • But you need to configure the GPU request in the Compose file for multi-container environments

Requesting GPUs for the Project Container#

To use GPUs in a project container, you set the “Number of Desired GPUs” to a value between 1-8, and when the container starts, Workbench checks an internal reference of available GPUs on the host.

If enough GPUs are available, they are “reserved” and explicitly passed into the container. If not, you are notified and container will not start.

Project containers support the following options:

Number of Desired GPUs: How many GPUs to request when starting this project.

Shared Memory: How much shared memory (in MB) should be set when starting the container (i.e the size of /dev/shm)

You request GPUs in Project Tab > Project Container > Hardware.

  • Click in the Number of Desired GPUs box and set the number between 0 and 8

  • If requesting more than 1, set Shared Memory to the amount of memory (in MiBs) you want shared between the GPUs

Limitations of GPU configuration for project containers:

  • The project container must have CUDA installed.

  • The CUDA version in the container must be compatible with the host’s GPU drivers.

Important

Windows has a limitation here. If you select 1 GPU, due to a driver limitation, all GPUs are passed into a GPU enabled projects.

For example, if you have a Windows system with 4 GPUs and you only reserve 1 for the project, all 4 GPUs will be passed into the container.

Requesting GPUs for Multi-Container Environments (Compose)#

See Multi-Container Environments.

FAQs#

Can I use the CLI to configure GPUs for the project container?#

Yes. The CLI has commands for this.

What happens if I request more GPUs than are available?#

If there aren’t enough GPUs available, Workbench will not run the container and will notify you of the discrepancy. Then you have a few options:

  • Reduce the number of requested GPUs

  • Stop another project to free up GPUs

  • Start the container without GPUs

Next Steps#