Using Multi-Instance GPUs

Important

This section applies to Linux only.

Applications that are developed with the Video Effects SDK can be deployed on Multi-Instance GPU (MIG) technology on supported devices, such as NVIDIA DGX™ A100. MIG lets you partition a device into multiple GPU instances, up to seven, each with separate streaming multiprocessors, separate slices of the GPU memory, and separate pathways to the memory. This process ensures that heavy resource usage by an application on one partition does not impact the performance of the applications running on other partitions.

To run an application on a MIG partition, you do not need to call any additional SDK API in your application. You can specify which MIG instance to use for execution during the invocation of your application.

You can select the MIG instance using one of the following options:

• The bare-metal method of using the CUDA_VISIBLE_DEVICES environment variable.

• The container method by using the NVIDIA Container Toolkit. MIG is supported only on Linux.

For more information about MIG and its usage, refer to the NVIDIA Multi-Instance GPU User Guide.