Is this page helpful?

Multi-Instance GPU (MIG)#

NVIDIA® Jetson™ Linux supports Multi-Instance GPU (MIG), which partitions GPU resources so that multiple workloads can run concurrently with hardware-level isolation. Use MIG when you need dedicated GPU slices for separate applications or tenants on a single Jetson device.

This page describes MIG support on Jetson platforms and how to configure and use it with this release of Jetson Linux.

Getting Started with MIG on Jetson Thor#

The following procedure walks through enabling MIG on a Jetson Thor system, creating two GPU instances with profiles 83 and 78, and running a graphics workload on the graphics-capable partition.

Prerequisite#

SSH into the Jetson Thor machine. SSH is required because in the next step you turn off the display.

Enable MIG Mode#

  1. Switch to multi-user target (stops the display manager):

    $ sudo systemctl isolate multi-user.target

  2. Enable persistence mode:

    $ sudo nvidia-smi -pm 1

    Output:

    Enabled Legacy persistence mode for GPU 00000000:01:00.0.
All done.

  3. Enable MIG mode:

    $ sudo nvidia-smi -mig 1

    Output:

    Enabled MIG Mode for GPU 00000000:01:00.0
All done.

Query Available Profiles#

List the GPU instance profiles available on the device:

$ sudo nvidia-smi mig -lgip

Output:

+-------------------------------------------------------------------------------+
| GPU instance profiles:                                                        |
| GPU   Name               ID    Instances   Memory     P2P    SM    DEC   ENC  |
|                                Free/Total   GiB              CE    JPEG  OFA  |
|===============================================================================|
|   0  MIG 1g.0gb+me       78     1/1        0.00       No      6     1     1   |
|                                                               1     1     1   |
+-------------------------------------------------------------------------------+
|   0  MIG 1g.0gb          80     1/1        0.00       No      6     1     1   |
|                                                               1     1     0   |
+-------------------------------------------------------------------------------+
|   0  MIG 1g.0gb+gfx      81     1/1        0.00       No      6     1     1   |
|                                                               1     1     0   |
+-------------------------------------------------------------------------------+
|   0  MIG 2g.0gb          82     1/1        0.00       No     12     1     1   |
|                                                               1     1     0   |
+-------------------------------------------------------------------------------+
|   0  MIG 2g.0gb+gfx      83     1/1        0.00       No     12     1     1   |
|                                                               1     1     0   |
+-------------------------------------------------------------------------------+
|   0  MIG 3g.0gb           0     1/1        0.00       No     20     2     2   |
|                                                               1     2     1   |
+-------------------------------------------------------------------------------+
|   0  MIG 3g.0gb+gfx      32     1/1        0.00       No     20     2     2   |
|                                                               1     2     1   |
+-------------------------------------------------------------------------------+

For this example, use profiles 83 (MIG 2g.0gb+gfx) and 78 (MIG 1g.0gb+me).

Create MIG Instances#

Create the two GPU instances:

$ sudo nvidia-smi mig -cgi 83 -C
Successfully created GPU instance ID  1 on GPU  0 using profile MIG 2g.0gb+gfx (ID 83)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  1 using profile MIG 2g.0gb (ID  1)

$ sudo nvidia-smi mig -cgi 78 -C
Successfully created GPU instance ID  2 on GPU  0 using profile MIG 1g.0gb+me (ID 78)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  2 using profile MIG 1g.0gb (ID  0)

Verify MIG Instances#

Confirm that both MIG instances are present:

$ sudo nvidia-smi mig -lgi
+---------------------------------------------------------+
| GPU instances:                                          |
| GPU   Name               Profile  Instance   Placement  |
|                            ID       ID       Start:Size |
|=========================================================|
|   0  MIG 1g.0gb+me         78        2          2:1     |
+---------------------------------------------------------+
|   0  MIG 2g.0gb+gfx        83        1          0:2     |
+---------------------------------------------------------+

$ nvidia-smi -L
GPU 0: NVIDIA Thor (UUID: GPU-a7c66ad2-6dbb-0ab8-c1a2-37ba6dba3600)
  MIG 2g.0gb      Device  0: (UUID: MIG-db9a29c9-a9e7-5117-a862-f3be7e6cfbfe)
  MIG 1g.0gb      Device  1: (UUID: MIG-08c7bcb6-d7dc-5613-9449-2e1995352301)

Run Graphics on the MIG Graphics Partition#

Profile 83 is the partition that supports graphics. Assign it using the UUID from nvidia-smi -L:

export MIG_DEVICE_UUID=MIG-db9a29c9-a9e7-5117-a862-f3be7e6cfbfe

Start the display manager again:

sudo systemctl start gdm3

You should see the Ubuntu desktop on your monitor. Set DISPLAY to :0 or :1 and launch vkcube:

$ DISPLAY=:0 vkcube

Output:

Selected GPU 0: NVIDIA Thor, type: IntegratedGpu

Use nvidia-smi to confirm that processes are running on GPU instance ID 1 (profile 83). vkcube uses both compute and graphics on that instance:

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0    1    0            83929      G   /usr/lib/xorg/Xorg                        0MiB |
|    0    1    0            84073      G   /usr/bin/gnome-shell                      0MiB |
|    0    1    0            84561      G   /usr/libexec/gnome-initial-setup          0MiB |
|    0    1    0            85307    C+G   vkcube                                    0MiB |
+-----------------------------------------------------------------------------------------+

Run Applications on Each Partition#

Launch CUDA or graphics applications on the desired partition:

  • Profile 83 (MIG 2g.0gb+gfx): CUDA_VISIBLE_DEVICES=0

  • Profile 78 (MIG 1g.0gb+me): CUDA_VISIBLE_DEVICES=1

For example, run glmark2 or other CUDA programs on the corresponding device index.