Is this page helpful?

Multi Instance GPU Management#

This chapter describes NVML operations that are associated with Multi Instance GPU management.

Macros#

NVML_COMPUTE_INSTANCE_ENGINE_PROFILE_COUNT: Number of engine profiles.
NVML_COMPUTE_INSTANCE_ENGINE_PROFILE_SHARED: All the engines except multiprocessors would be shared.
NVML_COMPUTE_INSTANCE_PROFILE_1_SLICE: Compute instance profiles.
NVML_COMPUTE_INSTANCE_PROFILE_1_SLICE_REV1: 1_SLICE compute instance profile (rev1).
NVML_COMPUTE_INSTANCE_PROFILE_2_SLICE: 2_SLICE compute instance profile.
NVML_COMPUTE_INSTANCE_PROFILE_3_SLICE: 3_SLICE compute instance profile.
NVML_COMPUTE_INSTANCE_PROFILE_4_SLICE: 4_SLICE compute instance profile.
NVML_COMPUTE_INSTANCE_PROFILE_6_SLICE: 6_SLICE compute instance profile.
NVML_COMPUTE_INSTANCE_PROFILE_7_SLICE: 7_SLICE compute instance profile (perf optimized for host work scheduling).
NVML_COMPUTE_INSTANCE_PROFILE_7_SLICE_NVL: 7_SLICE compute instance profile (perf optimized for multi-GPU use).
NVML_COMPUTE_INSTANCE_PROFILE_8_SLICE: 8_SLICE compute instance profile.
NVML_COMPUTE_INSTANCE_PROFILE_CAPS_GFX: MIG compute instance profile capability.
NVML_COMPUTE_INSTANCE_PROFILE_COUNT: Number of compute instance profiles.
NVML_DEVICE_MIG_DISABLE: Disable Multi Instance GPU mode.
NVML_DEVICE_MIG_ENABLE: Enable Multi Instance GPU mode.
NVML_GPU_INSTANCE_PROFILE_1_SLICE: GPU instance profiles.
NVML_GPU_INSTANCE_PROFILE_1_SLICE_ALL_ME: 1_SLICE profile with all of GPU Decode, Encoder, JPEG, OFA engines.
NVML_GPU_INSTANCE_PROFILE_1_SLICE_GFX: 1_SLICE gfx capable profile.
NVML_GPU_INSTANCE_PROFILE_1_SLICE_NO_ME: 1_SLICE profile with none of Decode, Encoder, JPEG, OFA engines.
NVML_GPU_INSTANCE_PROFILE_1_SLICE_REV1: 1_SLICE GPU instance profile (rev1).
NVML_GPU_INSTANCE_PROFILE_1_SLICE_REV2: 1_SLICE GPU instance profile (rev2).
NVML_GPU_INSTANCE_PROFILE_2_SLICE: 2_SLICE GPU instance profile.
NVML_GPU_INSTANCE_PROFILE_2_SLICE_ALL_ME
NVML_GPU_INSTANCE_PROFILE_2_SLICE_GFX: 2_SLICE gfx capable profile.
NVML_GPU_INSTANCE_PROFILE_2_SLICE_NO_ME: 2_SLICE profile with none of Decode, Encoder, JPEG, OFA engines.
NVML_GPU_INSTANCE_PROFILE_2_SLICE_REV1: 2_SLICE GPU instance profile (rev1).
NVML_GPU_INSTANCE_PROFILE_3_SLICE: 3_SLICE GPU instance profile.
NVML_GPU_INSTANCE_PROFILE_3_SLICE_GFX: 3_SLICE gfx + media capable profile.
NVML_GPU_INSTANCE_PROFILE_4_SLICE: 4_SLICE GPU instance profile.
NVML_GPU_INSTANCE_PROFILE_4_SLICE_GFX: 4_SLICE gfx capable profile.
NVML_GPU_INSTANCE_PROFILE_6_SLICE: 6_SLICE GPU instance profile.
NVML_GPU_INSTANCE_PROFILE_7_SLICE: 7_SLICE GPU instance profile.
NVML_GPU_INSTANCE_PROFILE_8_SLICE: 8_SLICE GPU instance profile.
NVML_GPU_INSTANCE_PROFILE_CAPS_GFX: GFX support.
NVML_GPU_INSTANCE_PROFILE_CAPS_P2P: MIG GPU instance profile capability.
NVML_GPU_INSTANCE_PROFILE_COUNT: Total number of GPU instance profiles.
NVML_GPU_INTSTANCE_PROFILE_CAPS_P2P: Deprecated, do not use.
nvmlComputeInstanceProfileInfo_v2: Version identifier value for nvmlComputeInstanceProfileInfo_v2_t::version .
nvmlComputeInstanceProfileInfo_v3: Version identifier value for nvmlComputeInstanceProfileInfo_v3_t::version .
nvmlGpuInstanceProfileInfo_v2: Version identifier value for nvmlGpuInstanceProfileInfo_v2_t::version .
nvmlGpuInstanceProfileInfo_v3: Version identifier value for nvmlGpuInstanceProfileInfo_v3_t::version .

Functions#

nvmlReturn_t nvmlComputeInstanceDestroy(nvmlComputeInstance_t computeInstance): Destroy compute instance.
nvmlReturn_t nvmlComputeInstanceGetInfo_v2(nvmlComputeInstance_t computeInstance, nvmlComputeInstanceInfo_t *info): Get compute instance information.
nvmlReturn_t nvmlDeviceCreateGpuInstance(nvmlDevice_t device, unsigned int profileId, nvmlGpuInstance_t *gpuInstance): Create GPU instance.
nvmlReturn_t nvmlDeviceCreateGpuInstanceWithPlacement(nvmlDevice_t device, unsigned int profileId, const nvmlGpuInstancePlacement_t *placement, nvmlGpuInstance_t *gpuInstance): Create GPU instance with the specified placement.
nvmlReturn_t nvmlDeviceGetComputeInstanceId(nvmlDevice_t device, unsigned int *id): Get compute instance ID for the given MIG device handle.
nvmlReturn_t nvmlDeviceGetDeviceHandleFromMigDeviceHandle(nvmlDevice_t migDevice, nvmlDevice_t *device): Get parent device handle from a MIG device handle.
nvmlReturn_t nvmlDeviceGetGpuInstanceById(nvmlDevice_t device, unsigned int id, nvmlGpuInstance_t *gpuInstance): Get GPU instances for given instance ID.
nvmlReturn_t nvmlDeviceGetGpuInstanceId(nvmlDevice_t device, unsigned int *id): Get GPU instance ID for the given MIG device handle.
nvmlReturn_t nvmlDeviceGetGpuInstancePossiblePlacements_v2(nvmlDevice_t device, unsigned int profileId, nvmlGpuInstancePlacement_t *placements, unsigned int *count): Get GPU instance placements.
nvmlReturn_t nvmlDeviceGetGpuInstanceProfileInfo(nvmlDevice_t device, unsigned int profile, nvmlGpuInstanceProfileInfo_t *info): Get GPU instance profile information.
nvmlReturn_t nvmlDeviceGetGpuInstanceProfileInfoByIdV(nvmlDevice_t device, unsigned int profileId, nvmlGpuInstanceProfileInfo_v2_t *info): GPU instance profile query function that accepts profile ID, instead of profile name.
nvmlReturn_t nvmlDeviceGetGpuInstanceProfileInfoV(nvmlDevice_t device, unsigned int profile, nvmlGpuInstanceProfileInfo_v2_t *info): Versioned wrapper around nvmlDeviceGetGpuInstanceProfileInfo that accepts a versioned nvmlGpuInstanceProfileInfo_v2_t or later output structure.
nvmlReturn_t nvmlDeviceGetGpuInstanceRemainingCapacity(nvmlDevice_t device, unsigned int profileId, unsigned int *count): Get GPU instance profile capacity.
nvmlReturn_t nvmlDeviceGetGpuInstances(nvmlDevice_t device, unsigned int profileId, nvmlGpuInstance_t *gpuInstances, unsigned int *count): Get GPU instances for given profile ID.
nvmlReturn_t nvmlDeviceGetMaxMigDeviceCount(nvmlDevice_t device, unsigned int *count): Get the maximum number of MIG devices that can exist under a given parent NVML device.
nvmlReturn_t nvmlDeviceGetMigDeviceHandleByIndex(nvmlDevice_t device, unsigned int index, nvmlDevice_t *migDevice): Get MIG device handle for the given index under its parent NVML device.
nvmlReturn_t nvmlDeviceGetMigMode(nvmlDevice_t device, unsigned int *currentMode, unsigned int *pendingMode): Get MIG mode for the device.
nvmlReturn_t nvmlDeviceIsMigDeviceHandle(nvmlDevice_t device, unsigned int *isMigDevice): Test if the given handle refers to a MIG device.
nvmlReturn_t nvmlDeviceSetMigMode(nvmlDevice_t device, unsigned int mode, nvmlReturn_t *activationStatus): Set MIG mode for the device.
nvmlReturn_t nvmlGpuInstanceCreateComputeInstance(nvmlGpuInstance_t gpuInstance, unsigned int profileId, nvmlComputeInstance_t *computeInstance): Create compute instance.
nvmlReturn_t nvmlGpuInstanceCreateComputeInstanceWithPlacement(nvmlGpuInstance_t gpuInstance, unsigned int profileId, const nvmlComputeInstancePlacement_t *placement, nvmlComputeInstance_t *computeInstance): Create compute instance with the specified placement.
nvmlReturn_t nvmlGpuInstanceDestroy(nvmlGpuInstance_t gpuInstance): Destroy GPU instance.
nvmlReturn_t nvmlGpuInstanceGetComputeInstanceById(nvmlGpuInstance_t gpuInstance, unsigned int id, nvmlComputeInstance_t *computeInstance): Get compute instance for given instance ID.
nvmlReturn_t nvmlGpuInstanceGetComputeInstancePossiblePlacements(nvmlGpuInstance_t gpuInstance, unsigned int profileId, nvmlComputeInstancePlacement_t *placements, unsigned int *count): Get compute instance placements.
nvmlReturn_t nvmlGpuInstanceGetComputeInstanceProfileInfo(nvmlGpuInstance_t gpuInstance, unsigned int profile, unsigned int engProfile, nvmlComputeInstanceProfileInfo_t *info): Get compute instance profile information.
nvmlReturn_t nvmlGpuInstanceGetComputeInstanceProfileInfoV(nvmlGpuInstance_t gpuInstance, unsigned int profile, unsigned int engProfile, nvmlComputeInstanceProfileInfo_v2_t *info): Versioned wrapper around nvmlGpuInstanceGetComputeInstanceProfileInfo that accepts a versioned nvmlComputeInstanceProfileInfo_v2_t or later output structure.
nvmlReturn_t nvmlGpuInstanceGetComputeInstanceRemainingCapacity(nvmlGpuInstance_t gpuInstance, unsigned int profileId, unsigned int *count): Get compute instance profile capacity.
nvmlReturn_t nvmlGpuInstanceGetComputeInstances(nvmlGpuInstance_t gpuInstance, unsigned int profileId, nvmlComputeInstance_t *computeInstances, unsigned int *count): Get compute instances for given profile ID.
nvmlReturn_t nvmlGpuInstanceGetInfo(nvmlGpuInstance_t gpuInstance, nvmlGpuInstanceInfo_t *info): Get GPU instance information.

Structs#

nvmlComputeInstanceInfo_t
nvmlComputeInstancePlacement_t
nvmlComputeInstanceProfileInfo_t: Compute instance profile information.
nvmlComputeInstanceProfileInfo_v2_t: Compute instance profile information (v2).
nvmlComputeInstanceProfileInfo_v3_t: Compute instance profile information (v3).
nvmlGpuInstanceInfo_t
nvmlGpuInstancePlacement_t
nvmlGpuInstanceProfileInfo_t: GPU instance profile information.
nvmlGpuInstanceProfileInfo_v2_t: GPU instance profile information (v2).
nvmlGpuInstanceProfileInfo_v3_t: GPU instance profile information (v3).

Typedefs#

nvmlComputeInstance_t

Macros#

NVML_COMPUTE_INSTANCE_ENGINE_PROFILE_COUNT#: Number of engine profiles.

NVML_COMPUTE_INSTANCE_ENGINE_PROFILE_SHARED#: All the engines except multiprocessors would be shared.

NVML_COMPUTE_INSTANCE_PROFILE_1_SLICE#

Compute instance profiles.

These macros should be passed to nvmlGpuInstanceGetComputeInstanceProfileInfo to retrieve the detailed information about a compute instance such as profile ID, engine counts 1_SLICE compute instance profile.

NVML_COMPUTE_INSTANCE_PROFILE_1_SLICE_REV1#: 1_SLICE compute instance profile (rev1).

NVML_COMPUTE_INSTANCE_PROFILE_2_SLICE#: 2_SLICE compute instance profile.

NVML_COMPUTE_INSTANCE_PROFILE_3_SLICE#: 3_SLICE compute instance profile.

NVML_COMPUTE_INSTANCE_PROFILE_4_SLICE#: 4_SLICE compute instance profile.

NVML_COMPUTE_INSTANCE_PROFILE_6_SLICE#: 6_SLICE compute instance profile.

NVML_COMPUTE_INSTANCE_PROFILE_7_SLICE#: 7_SLICE compute instance profile (perf optimized for host work scheduling).

NVML_COMPUTE_INSTANCE_PROFILE_7_SLICE_NVL#: 7_SLICE compute instance profile (perf optimized for multi-GPU use).

NVML_COMPUTE_INSTANCE_PROFILE_8_SLICE#: 8_SLICE compute instance profile.

NVML_COMPUTE_INSTANCE_PROFILE_CAPS_GFX#

MIG compute instance profile capability.

Bit field values representing MIG profile capabilities nvmlComputeInstanceProfileInfo_v3_t::capabilities

NVML_COMPUTE_INSTANCE_PROFILE_COUNT#: Number of compute instance profiles.

NVML_DEVICE_MIG_DISABLE#

Disable Multi Instance GPU mode.

NVML_DEVICE_MIG_ENABLE#

Enable Multi Instance GPU mode.

NVML_GPU_INSTANCE_PROFILE_1_SLICE#

GPU instance profiles.

These macros should be passed to nvmlDeviceGetGpuInstanceProfileInfo to retrieve the detailed information about a GPU instance such as profile ID, engine counts. 1_SLICE GPU instance profile.

NVML_GPU_INSTANCE_PROFILE_1_SLICE_ALL_ME#: 1_SLICE profile with all of GPU Decode, Encoder, JPEG, OFA engines.

NVML_GPU_INSTANCE_PROFILE_1_SLICE_GFX#: 1_SLICE gfx capable profile.

NVML_GPU_INSTANCE_PROFILE_1_SLICE_NO_ME#: 1_SLICE profile with none of Decode, Encoder, JPEG, OFA engines.

NVML_GPU_INSTANCE_PROFILE_1_SLICE_REV1#: 1_SLICE GPU instance profile (rev1).

NVML_GPU_INSTANCE_PROFILE_1_SLICE_REV2#: 1_SLICE GPU instance profile (rev2).

NVML_GPU_INSTANCE_PROFILE_2_SLICE#: 2_SLICE GPU instance profile.

NVML_GPU_INSTANCE_PROFILE_2_SLICE_ALL_ME#

NVML_GPU_INSTANCE_PROFILE_2_SLICE_GFX#: 2_SLICE gfx capable profile.

NVML_GPU_INSTANCE_PROFILE_2_SLICE_NO_ME#: 2_SLICE profile with none of Decode, Encoder, JPEG, OFA engines.

NVML_GPU_INSTANCE_PROFILE_2_SLICE_REV1#: 2_SLICE GPU instance profile (rev1).

NVML_GPU_INSTANCE_PROFILE_3_SLICE#: 3_SLICE GPU instance profile.

NVML_GPU_INSTANCE_PROFILE_3_SLICE_GFX#: 3_SLICE gfx + media capable profile.

NVML_GPU_INSTANCE_PROFILE_4_SLICE#: 4_SLICE GPU instance profile.

NVML_GPU_INSTANCE_PROFILE_4_SLICE_GFX#: 4_SLICE gfx capable profile.

NVML_GPU_INSTANCE_PROFILE_6_SLICE#: 6_SLICE GPU instance profile.

NVML_GPU_INSTANCE_PROFILE_7_SLICE#: 7_SLICE GPU instance profile.

NVML_GPU_INSTANCE_PROFILE_8_SLICE#: 8_SLICE GPU instance profile.

NVML_GPU_INSTANCE_PROFILE_CAPS_GFX#: GFX support.

NVML_GPU_INSTANCE_PROFILE_CAPS_P2P#

MIG GPU instance profile capability.

Bit field values representing MIG profile capabilities nvmlGpuInstanceProfileInfo_v3_t::capabilities Peer-to-Peer support.

NVML_GPU_INSTANCE_PROFILE_COUNT#: Total number of GPU instance profiles.

NVML_GPU_INTSTANCE_PROFILE_CAPS_P2P#: Deprecated, do not use.

nvmlComputeInstanceProfileInfo_v2#: Version identifier value for nvmlComputeInstanceProfileInfo_v2_t::version.

nvmlComputeInstanceProfileInfo_v3#: Version identifier value for nvmlComputeInstanceProfileInfo_v3_t::version.

nvmlGpuInstanceProfileInfo_v2#: Version identifier value for nvmlGpuInstanceProfileInfo_v2_t::version.

nvmlGpuInstanceProfileInfo_v3#: Version identifier value for nvmlGpuInstanceProfileInfo_v3_t::version.

Functions#

nvmlReturn_t nvmlComputeInstanceDestroy( nvmlComputeInstance_t computeInstance, )#

Destroy compute instance.

For Ampere or newer fully supported devices. Supported on Linux only. Requires privileged user.

Parameters:

computeInstance – The compute instance handle

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If computeInstance is invalid
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation
NVML_ERROR_IN_USE If the compute instance is in use. This error would be returned if processes (e.g. CUDA application) are active on the compute instance.

nvmlReturn_t nvmlComputeInstanceGetInfo_v2( nvmlComputeInstance_t computeInstance, nvmlComputeInstanceInfo_t *info, )#

Get compute instance information.

For Ampere or newer fully supported devices. Supported on Linux only.

Parameters:

computeInstance – The compute instance handle
info – Return compute instance information

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If computeInstance or info are invalid
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation

nvmlReturn_t nvmlDeviceCreateGpuInstance( nvmlDevice_t device, unsigned int profileId, nvmlGpuInstance_t *gpuInstance, )#

Create GPU instance.

For Ampere or newer fully supported devices. Supported on Linux only. Requires privileged user.

If the parent device is unbound, reset or the GPU instance is destroyed explicitly, the GPU instance handle would become invalid. The GPU instance must be recreated to acquire a valid handle.

Parameters:

device – The identifier of the target device
profileId – The GPU instance profile ID. See nvmlDeviceGetGpuInstanceProfileInfo
gpuInstance – Returns the GPU instance handle

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If device, profile, profileId or gpuInstance are invalid
NVML_ERROR_NOT_SUPPORTED If device doesn’t have MIG mode enabled or in vGPU guest
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation
NVML_ERROR_INSUFFICIENT_RESOURCES If the requested GPU instance could not be created

nvmlReturn_t nvmlDeviceCreateGpuInstanceWithPlacement( nvmlDevice_t device, unsigned int profileId, const nvmlGpuInstancePlacement_t *placement, nvmlGpuInstance_t *gpuInstance, )#

Create GPU instance with the specified placement.

For Ampere or newer fully supported devices. Supported on Linux only. Requires privileged user.

If the parent device is unbound, reset or the GPU instance is destroyed explicitly, the GPU instance handle would become invalid. The GPU instance must be recreated to acquire a valid handle.

Parameters:

device – The identifier of the target device
profileId – The GPU instance profile ID. See nvmlDeviceGetGpuInstanceProfileInfo
placement – The requested placement. See nvmlDeviceGetGpuInstancePossiblePlacements_v2
gpuInstance – Returns the GPU instance handle

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If device, profile, profileId, placement or gpuInstance are invalid
NVML_ERROR_NOT_SUPPORTED If device doesn’t have MIG mode enabled or in vGPU guest
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation
NVML_ERROR_INSUFFICIENT_RESOURCES If the requested GPU instance could not be created

nvmlReturn_t nvmlDeviceGetComputeInstanceId( nvmlDevice_t device, unsigned int *id, )#

Get compute instance ID for the given MIG device handle.

Compute instance IDs are unique per GPU instance and remain valid until the compute instance is destroyed.

For Ampere or newer fully supported devices. Supported on Linux only.

Parameters:

device – Target MIG device handle
id – Compute instance ID

Returns:

NVML_SUCCESS if instance ID was successfully retrieved
NVML_ERROR_UNINITIALIZED if the library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT if device or id reference is invalid
NVML_ERROR_NOT_SUPPORTED if this query is not supported by the device
NVML_ERROR_UNKNOWN on any unexpected error

nvmlReturn_t nvmlDeviceGetDeviceHandleFromMigDeviceHandle( nvmlDevice_t migDevice, nvmlDevice_t *device, )#

Get parent device handle from a MIG device handle.

For Ampere or newer fully supported devices. Supported on Linux only.

Parameters:

migDevice – MIG device handle
device – Device handle

Returns:

NVML_SUCCESS if device handle was successfully created
NVML_ERROR_UNINITIALIZED if the library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT if migDevice or device is invalid
NVML_ERROR_NOT_SUPPORTED if this query is not supported by the device
NVML_ERROR_UNKNOWN on any unexpected error

nvmlReturn_t nvmlDeviceGetGpuInstanceById( nvmlDevice_t device, unsigned int id, nvmlGpuInstance_t *gpuInstance, )#

Get GPU instances for given instance ID.

For Ampere or newer fully supported devices. Supported on Linux only. Requires privileged user.

Parameters:

device – The identifier of the target device
id – The GPU instance ID
gpuInstance – Returns GPU instance

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If device, id or gpuInstance are invalid
NVML_ERROR_NOT_SUPPORTED If device doesn’t have MIG mode enabled
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation
NVML_ERROR_NOT_FOUND If the GPU instance is not found.

nvmlReturn_t nvmlDeviceGetGpuInstanceId( nvmlDevice_t device, unsigned int *id, )#

Get GPU instance ID for the given MIG device handle.

GPU instance IDs are unique per device and remain valid until the GPU instance is destroyed.

For Ampere or newer fully supported devices. Supported on Linux only.

Parameters:

device – Target MIG device handle
id – GPU instance ID

Returns:

NVML_SUCCESS if instance ID was successfully retrieved
NVML_ERROR_UNINITIALIZED if the library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT if device or id reference is invalid
NVML_ERROR_NOT_SUPPORTED if this query is not supported by the device
NVML_ERROR_UNKNOWN on any unexpected error

nvmlReturn_t nvmlDeviceGetGpuInstancePossiblePlacements_v2( nvmlDevice_t device, unsigned int profileId, nvmlGpuInstancePlacement_t *placements, unsigned int *count, )#

Get GPU instance placements.

A placement represents the location of a GPU instance within a device. This API only returns all the possible placements for the given profile regardless of whether MIG is enabled or not. A created GPU instance occupies memory slices described by its placement. Creation of new GPU instance will fail if there is overlap with the already occupied memory slices.

For Ampere or newer fully supported devices. Supported on Linux only. Requires privileged user.

Parameters:

device – The identifier of the target device
profileId – The GPU instance profile ID. See nvmlDeviceGetGpuInstanceProfileInfo
placements – Returns placements allowed for the profile. Can be NULL to discover number of allowed placements for this profile. If non-NULL must be large enough to accommodate the placements supported by the profile.
count – Returns number of allowed placemenets for the profile.

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If device, profileId or count are invalid
NVML_ERROR_NOT_SUPPORTED If device doesn’t support MIG or profileId isn’t supported
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation

nvmlReturn_t nvmlDeviceGetGpuInstanceProfileInfo( nvmlDevice_t device, unsigned int profile, nvmlGpuInstanceProfileInfo_t *info, )#

Get GPU instance profile information.

Information provided by this API is immutable throughout the lifetime of a MIG mode.

For Ampere or newer fully supported devices. Supported on Linux only.

Note

This API can be used to enumerate all MIG profiles supported by NVML in a forward compatible way by invoking it on profile values starting from 0, until the API returns NVML_ERROR_INVALID_ARGUMENT.

Parameters:

device – The identifier of the target device
profile – One of the NVML_GPU_INSTANCE_PROFILE_*
info – Returns detailed profile information

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If device, profile or info are invalid
NVML_ERROR_NOT_SUPPORTED If device doesn’t support MIG or profile isn’t supported
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation

nvmlReturn_t nvmlDeviceGetGpuInstanceProfileInfoByIdV( nvmlDevice_t device, unsigned int profileId, nvmlGpuInstanceProfileInfo_v2_t *info, )#

GPU instance profile query function that accepts profile ID, instead of profile name.

It accepts a versioned nvmlGpuInstanceProfileInfo_v2_t or later output structure.

For Ampere or newer fully supported devices. Supported on Linux only.

Note

The caller must set the nvmlGpuInstanceProfileInfo_v2_t::version field to the appropriate version prior to calling this function. For example:

nvmlGpuInstanceProfileInfo_v2_t profileInfo =
    { .version = nvmlGpuInstanceProfileInfo_v2 };
nvmlReturn_t result = nvmlDeviceGetGpuInstanceProfileInfoV(device,
                                                           profile,
                                                           &profileInfo);

Parameters:

device – The identifier of the target device
profileId – One of the profile IDs.
info – Returns detailed profile information

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If device, profileId, info, or info->version are invalid
NVML_ERROR_NOT_SUPPORTED If device doesn’t have MIG mode enabled or profile isn’t supported
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation

nvmlReturn_t nvmlDeviceGetGpuInstanceProfileInfoV( nvmlDevice_t device, unsigned int profile, nvmlGpuInstanceProfileInfo_v2_t *info, )#

Versioned wrapper around nvmlDeviceGetGpuInstanceProfileInfo that accepts a versioned nvmlGpuInstanceProfileInfo_v2_t or later output structure.

For Ampere or newer fully supported devices. Supported on Linux only.

Note

The caller must set the nvmlGpuInstanceProfileInfo_v2_t::version field to the appropriate version prior to calling this function. For example:

nvmlGpuInstanceProfileInfo_v2_t profileInfo =
    { .version = nvmlGpuInstanceProfileInfo_v2 };
nvmlReturn_t result = nvmlDeviceGetGpuInstanceProfileInfoV(device,
                                                           profile,
                                                           &profileInfo);

Parameters:

device – The identifier of the target device
profile – One of the NVML_GPU_INSTANCE_PROFILE_*
info – Returns detailed profile information

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If device, profile, info, or info->version are invalid
NVML_ERROR_NOT_SUPPORTED If device doesn’t have MIG mode enabled or profile isn’t supported
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation

nvmlReturn_t nvmlDeviceGetGpuInstanceRemainingCapacity( nvmlDevice_t device, unsigned int profileId, unsigned int *count, )#

Get GPU instance profile capacity.

For Ampere or newer fully supported devices. Supported on Linux only. Requires privileged user.

Parameters:

device – The identifier of the target device
profileId – The GPU instance profile ID. See nvmlDeviceGetGpuInstanceProfileInfo
count – Returns remaining instance count for the profile ID

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If device, profileId or count are invalid
NVML_ERROR_NOT_SUPPORTED If device doesn’t have MIG mode enabled or profileId isn’t supported
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation

nvmlReturn_t nvmlDeviceGetGpuInstances( nvmlDevice_t device, unsigned int profileId, nvmlGpuInstance_t *gpuInstances, unsigned int *count, )#

Get GPU instances for given profile ID.

For Ampere or newer fully supported devices. Supported on Linux only. Requires privileged user.

Parameters:

device – The identifier of the target device
profileId – The GPU instance profile ID. See nvmlDeviceGetGpuInstanceProfileInfo
gpuInstances – Returns pre-exiting GPU instances, the buffer must be large enough to accommodate the instances supported by the profile. See nvmlDeviceGetGpuInstanceProfileInfo
count – The count of returned GPU instances

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If device, profileId, gpuInstances or count are invalid
NVML_ERROR_NOT_SUPPORTED If device doesn’t have MIG mode enabled
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation

nvmlReturn_t nvmlDeviceGetMaxMigDeviceCount( nvmlDevice_t device, unsigned int *count, )#

Get the maximum number of MIG devices that can exist under a given parent NVML device.

Returns zero if MIG is not supported or enabled.

For Ampere or newer fully supported devices. Supported on Linux only.

Parameters:

device – Target device handle
count – Count of MIG devices

Returns:

NVML_SUCCESS if count was successfully retrieved
NVML_ERROR_UNINITIALIZED if the library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT if device or count reference is invalid
NVML_ERROR_UNKNOWN on any unexpected error

nvmlReturn_t nvmlDeviceGetMigDeviceHandleByIndex( nvmlDevice_t device, unsigned int index, nvmlDevice_t *migDevice, )#

Get MIG device handle for the given index under its parent NVML device.

If the compute instance is destroyed either explicitly or by destroying, resetting or unbinding the parent GPU instance or the GPU device itself the MIG device handle would remain invalid and must be requested again using this API. Handles may be reused and their properties can change in the process.

For Ampere or newer fully supported devices. Supported on Linux only.

Parameters:

device – Reference to the parent GPU device handle
index – Index of the MIG device
migDevice – Reference to the MIG device handle

Returns:

NVML_SUCCESS if migDevice handle was successfully created
NVML_ERROR_UNINITIALIZED if the library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT if device, index or migDevice reference is invalid
NVML_ERROR_NOT_SUPPORTED if this query is not supported by the device
NVML_ERROR_NOT_FOUND if no valid MIG device was found at index
NVML_ERROR_UNKNOWN on any unexpected error

nvmlReturn_t nvmlDeviceGetMigMode( nvmlDevice_t device, unsigned int *currentMode, unsigned int *pendingMode, )#

Get MIG mode for the device.

For Ampere or newer fully supported devices.

Changing MIG modes may require device unbind or reset. The “pending” MIG mode refers to the target mode following the next activation trigger.

Parameters:

device – The identifier of the target device
currentMode – Returns the current mode, NVML_DEVICE_MIG_DISABLE or NVML_DEVICE_MIG_ENABLE
pendingMode – Returns the pending mode, NVML_DEVICE_MIG_DISABLE or NVML_DEVICE_MIG_ENABLE

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If device, currentMode or pendingMode are invalid
NVML_ERROR_NOT_SUPPORTED If device doesn’t support MIG mode
NVML_ERROR_UNKNOWN on any unexpected error

nvmlReturn_t nvmlDeviceIsMigDeviceHandle( nvmlDevice_t device, unsigned int *isMigDevice, )#

Test if the given handle refers to a MIG device.

A MIG device handle is an NVML abstraction which maps to a MIG compute instance. These overloaded references can be used (with some restrictions) interchangeably with a GPU device handle to execute queries at a per-compute instance granularity.

For Ampere or newer fully supported devices. Supported on Linux only.

Parameters:

device – NVML handle to test
isMigDevice – True when handle refers to a MIG device

Returns:

NVML_SUCCESS if device status was successfully retrieved
NVML_ERROR_UNINITIALIZED if the library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT if device handle or isMigDevice reference is invalid
NVML_ERROR_NOT_SUPPORTED if this check is not supported by the device
NVML_ERROR_UNKNOWN on any unexpected error

nvmlReturn_t nvmlDeviceSetMigMode( nvmlDevice_t device, unsigned int mode, nvmlReturn_t *activationStatus, )#

Set MIG mode for the device.

For Ampere or newer fully supported devices. Requires root user.

This mode determines whether a GPU instance can be created.

This API may unbind or reset the device to activate the requested mode. Thus, the attributes associated with the device, such as minor number, might change. The caller of this API is expected to query such attributes again.

On certain platforms like pass-through virtualization, where reset functionality may not be exposed directly, VM reboot is required. activationStatus would return NVML_ERROR_RESET_REQUIRED for such cases.

activationStatus would return the appropriate error code upon unsuccessful activation. For example, if device unbind fails because the device isn’t idle, NVML_ERROR_IN_USE would be returned. The caller of this API is expected to idle the device and retry setting the mode.

Note

On Windows, only disabling MIG mode is supported. activationStatus would return NVML_ERROR_NOT_SUPPORTED as GPU reset is not supported on Windows through this API.

Parameters:

device – The identifier of the target device
mode – The mode to be set, NVML_DEVICE_MIG_DISABLE or NVML_DEVICE_MIG_ENABLE
activationStatus – The activationStatus status

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If device,mode or activationStatus are invalid
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation
NVML_ERROR_NOT_SUPPORTED If device doesn’t support MIG mode

nvmlReturn_t nvmlGpuInstanceCreateComputeInstance( nvmlGpuInstance_t gpuInstance, unsigned int profileId, nvmlComputeInstance_t *computeInstance, )#

Create compute instance.

For Ampere or newer fully supported devices. Supported on Linux only. Requires privileged user.

If the parent device is unbound, reset or the parent GPU instance is destroyed or the compute instance is destroyed explicitly, the compute instance handle would become invalid. The compute instance must be recreated to acquire a valid handle.

Parameters:

gpuInstance – The identifier of the target GPU instance
profileId – The compute instance profile ID. See nvmlGpuInstanceGetComputeInstanceProfileInfo
computeInstance – Returns the compute instance handle

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If gpuInstance, profile, profileId or computeInstance are invalid
NVML_ERROR_NOT_SUPPORTED If profileId isn’t supported
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation
NVML_ERROR_INSUFFICIENT_RESOURCES If the requested compute instance could not be created

nvmlReturn_t nvmlGpuInstanceCreateComputeInstanceWithPlacement( nvmlGpuInstance_t gpuInstance, unsigned int profileId, const nvmlComputeInstancePlacement_t *placement, nvmlComputeInstance_t *computeInstance, )#

Create compute instance with the specified placement.

For Ampere or newer fully supported devices. Supported on Linux only. Requires privileged user.

If the parent device is unbound, reset or the parent GPU instance is destroyed or the compute instance is destroyed explicitly, the compute instance handle would become invalid. The compute instance must be recreated to acquire a valid handle.

Parameters:

gpuInstance – The identifier of the target GPU instance
profileId – The compute instance profile ID. See nvmlGpuInstanceGetComputeInstanceProfileInfo
placement – The requested placement. See nvmlGpuInstanceGetComputeInstancePossiblePlacements
computeInstance – Returns the compute instance handle

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If gpuInstance, profile, profileId or computeInstance are invalid
NVML_ERROR_NOT_SUPPORTED If profileId isn’t supported
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation
NVML_ERROR_INSUFFICIENT_RESOURCES If the requested compute instance could not be created

nvmlReturn_t nvmlGpuInstanceDestroy(nvmlGpuInstance_t gpuInstance)#

Destroy GPU instance.

For Ampere or newer fully supported devices. Supported on Linux only. Requires privileged user.

Parameters:

gpuInstance – The GPU instance handle

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If gpuInstance is invalid
NVML_ERROR_NOT_SUPPORTED If device doesn’t have MIG mode enabled or in vGPU guest
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation
NVML_ERROR_IN_USE If the GPU instance is in use. This error would be returned if processes (e.g. CUDA application) or compute instances are active on the GPU instance.

nvmlReturn_t nvmlGpuInstanceGetComputeInstanceById( nvmlGpuInstance_t gpuInstance, unsigned int id, nvmlComputeInstance_t *computeInstance, )#

Get compute instance for given instance ID.

For Ampere or newer fully supported devices. Supported on Linux only. Requires privileged user.

Parameters:

gpuInstance – The identifier of the target GPU instance
id – The compute instance ID
computeInstance – Returns compute instance

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If device, ID or computeInstance are invalid
NVML_ERROR_NOT_SUPPORTED If device doesn’t have MIG mode enabled
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation
NVML_ERROR_NOT_FOUND If the compute instance is not found.

nvmlReturn_t nvmlGpuInstanceGetComputeInstancePossiblePlacements( nvmlGpuInstance_t gpuInstance, unsigned int profileId, nvmlComputeInstancePlacement_t *placements, unsigned int *count, )#

Get compute instance placements.

For Ampere or newer fully supported devices. Supported on Linux only. Requires privileged user.

A placement represents the location of a compute instance within a GPU instance. This API only returns all the possible placements for the given profile. A created compute instance occupies compute slices described by its placement. Creation of new compute instance will fail if there is overlap with the already occupied compute slices.

Parameters:

gpuInstance – The identifier of the target GPU instance
profileId – The compute instance profile ID. See nvmlGpuInstanceGetComputeInstanceProfileInfo
placements – Returns placements allowed for the profile. Can be NULL to discover number of allowed placements for this profile. If non-NULL must be large enough to accommodate the placements supported by the profile.
count – Returns number of allowed placemenets for the profile.

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If gpuInstance, profileId or count are invalid
NVML_ERROR_NOT_SUPPORTED If device doesn’t have MIG mode enabled or profileId isn’t supported
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation

nvmlReturn_t nvmlGpuInstanceGetComputeInstanceProfileInfo( nvmlGpuInstance_t gpuInstance, unsigned int profile, unsigned int engProfile, nvmlComputeInstanceProfileInfo_t *info, )#

Get compute instance profile information.

Information provided by this API is immutable throughout the lifetime of a MIG mode.

For Ampere or newer fully supported devices. Supported on Linux only.

Note

This API can be used to enumerate all MIG profiles supported by NVML in a forward compatible way by invoking it on profile values starting from 0, until the API returns NVML_ERROR_INVALID_ARGUMENT.

Parameters:

gpuInstance – The identifier of the target GPU instance
profile – One of the NVML_COMPUTE_INSTANCE_PROFILE_*
engProfile – One of the NVML_COMPUTE_INSTANCE_ENGINE_PROFILE_*
info – Returns detailed profile information

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If gpuInstance, profile, engProfile or info are invalid
NVML_ERROR_NOT_SUPPORTED If profile isn’t supported
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation

nvmlReturn_t nvmlGpuInstanceGetComputeInstanceProfileInfoV( nvmlGpuInstance_t gpuInstance, unsigned int profile, unsigned int engProfile, nvmlComputeInstanceProfileInfo_v2_t *info, )#

Versioned wrapper around nvmlGpuInstanceGetComputeInstanceProfileInfo that accepts a versioned nvmlComputeInstanceProfileInfo_v2_t or later output structure.

For Ampere or newer fully supported devices. Supported on Linux only.

Note

The caller must set the nvmlGpuInstanceProfileInfo_v2_t::version field to the appropriate version prior to calling this function. For example:

nvmlComputeInstanceProfileInfo_v2_t profileInfo =
    { .version = nvmlComputeInstanceProfileInfo_v2 };
nvmlReturn_t result = nvmlGpuInstanceGetComputeInstanceProfileInfoV(gpuInstance,
                                                                    profile,
                                                                    engProfile,
                                                                    &profileInfo);

Parameters:

gpuInstance – The identifier of the target GPU instance
profile – One of the NVML_COMPUTE_INSTANCE_PROFILE_*
engProfile – One of the NVML_COMPUTE_INSTANCE_ENGINE_PROFILE_*
info – Returns detailed profile information

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If gpuInstance, profile, engProfile, info, or info->version are invalid
NVML_ERROR_NOT_SUPPORTED If profile isn’t supported
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation

nvmlReturn_t nvmlGpuInstanceGetComputeInstanceRemainingCapacity( nvmlGpuInstance_t gpuInstance, unsigned int profileId, unsigned int *count, )#

Get compute instance profile capacity.

For Ampere or newer fully supported devices. Supported on Linux only. Requires privileged user.

Parameters:

gpuInstance – The identifier of the target GPU instance
profileId – The compute instance profile ID. See nvmlGpuInstanceGetComputeInstanceProfileInfo
count – Returns remaining instance count for the profile ID

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If gpuInstance, profileId or availableCount are invalid
NVML_ERROR_NOT_SUPPORTED If profileId isn’t supported
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation

nvmlReturn_t nvmlGpuInstanceGetComputeInstances( nvmlGpuInstance_t gpuInstance, unsigned int profileId, nvmlComputeInstance_t *computeInstances, unsigned int *count, )#

Get compute instances for given profile ID.

For Ampere or newer fully supported devices. Supported on Linux only. Requires privileged user.

Parameters:

gpuInstance – The identifier of the target GPU instance
profileId – The compute instance profile ID. See nvmlGpuInstanceGetComputeInstanceProfileInfo
computeInstances – Returns pre-exiting compute instances, the buffer must be large enough to accommodate the instances supported by the profile. See nvmlGpuInstanceGetComputeInstanceProfileInfo
count – The count of returned compute instances

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If gpuInstance, profileId, computeInstances or count are invalid
NVML_ERROR_NOT_SUPPORTED If profileId isn’t supported
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation

nvmlReturn_t nvmlGpuInstanceGetInfo( nvmlGpuInstance_t gpuInstance, nvmlGpuInstanceInfo_t *info, )#

Get GPU instance information.

For Ampere or newer fully supported devices. Supported on Linux only.

Parameters:

gpuInstance – The GPU instance handle
info – Return GPU instance information

Returns:

NVML_SUCCESS Upon success
NVML_ERROR_UNINITIALIZED If library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT If gpuInstance or info are invalid
NVML_ERROR_NO_PERMISSION If user doesn’t have permission to perform the operation

Typedefs#

typedef struct nvmlComputeInstance_st *nvmlComputeInstance_t#