Is this page helpful?

CPU and Memory Affinity#

This chapter describes NVML operations that are associated with CPU and memory affinity.

Macros#

NVML_AFFINITY_SCOPE_NODE: Scope of NUMA node for affinity queries.
NVML_AFFINITY_SCOPE_SOCKET: Scope of processor socket for affinity queries.

Functions#

nvmlReturn_t nvmlDeviceClearCpuAffinity(nvmlDevice_t device): Clear all affinity bindings for the calling thread.
nvmlReturn_t nvmlDeviceGetAddressingMode(nvmlDevice_t device, nvmlDeviceAddressingMode_t *mode): Get the addressing mode for a given GPU.
nvmlReturn_t nvmlDeviceGetCpuAffinity(nvmlDevice_t device, unsigned int cpuSetSize, unsigned long *cpuSet): Retrieves an array of unsigned ints (sized to cpuSetSize) of bitmasks with the ideal CPU affinity for the device For example, if processors 0, 1, 32, and 33 are ideal for the device and cpuSetSize == 2, result[0] = 0x3, result[1] = 0x3 This is equivalent to calling nvmlDeviceGetCpuAffinityWithinScope with NVML_AFFINITY_SCOPE_NODE .
nvmlReturn_t nvmlDeviceGetCpuAffinityWithinScope(nvmlDevice_t device, unsigned int cpuSetSize, unsigned long *cpuSet, nvmlAffinityScope_t scope): Retrieves an array of unsigned ints (sized to cpuSetSize) of bitmasks with the ideal CPU affinity within node or socket for the device.
nvmlReturn_t nvmlDeviceGetMemoryAffinity(nvmlDevice_t device, unsigned int nodeSetSize, unsigned long *nodeSet, nvmlAffinityScope_t scope): Retrieves an array of unsigned ints (sized to nodeSetSize) of bitmasks with the ideal memory affinity within node or socket for the device.
nvmlReturn_t nvmlDeviceGetNumaNodeId(nvmlDevice_t device, unsigned int *node): Get the NUMA node of the given GPU device.
nvmlReturn_t nvmlDeviceGetRepairStatus(nvmlDevice_t device, nvmlRepairStatus_t *repairStatus): Get the repair status for TPC/Channel repair.
nvmlReturn_t nvmlDeviceGetUnrepairableMemoryFlag_v1(nvmlDevice_t device, nvmlUnrepairableMemoryStatus_v1_t *unrepairableMemoryStatus): Get the unrepairable memory flag for a given GPU.
nvmlReturn_t nvmlDeviceSetCpuAffinity(nvmlDevice_t device): Sets the ideal affinity for the calling thread and device using the guidelines given in nvmlDeviceGetCpuAffinity() .

Typedefs#

nvmlAffinityScope_t

Macros#

NVML_AFFINITY_SCOPE_NODE#: Scope of NUMA node for affinity queries.

NVML_AFFINITY_SCOPE_SOCKET#: Scope of processor socket for affinity queries.

Functions#

nvmlReturn_t nvmlDeviceClearCpuAffinity(nvmlDevice_t device)#

Clear all affinity bindings for the calling thread.

Note, this is a change as of version 8.0 as older versions cleared the affinity for a calling process and all children.

For Kepler or newer fully supported devices. Supported on Linux only.

Parameters:

device – The identifier of the target device

Returns:

NVML_SUCCESS if the calling process has been successfully unbound
NVML_ERROR_INVALID_ARGUMENT if device is invalid
NVML_ERROR_UNINITIALIZED if the library has not been successfully initialized
NVML_ERROR_UNKNOWN on any unexpected error

nvmlReturn_t nvmlDeviceGetAddressingMode( nvmlDevice_t device, nvmlDeviceAddressingMode_t *mode, )#

Get the addressing mode for a given GPU.

Addressing modes can be one of:

HMM: System allocated memory (malloc, mmap) is addressable from the device (GPU), via software-based mirroring of the CPU’s page tables, on the GPU.
ATS: System allocated memory (malloc, mmap) is addressable from the device (GPU), via Address Translation Services. This means that there is (effectively) a single set of page tables, and the CPU and GPU both use them.
None: Neither HMM nor ATS is active.

For Turing or newer fully supported devices. Supported on Linux only.

Parameters:

device – [in] The device handle
mode – [out] Pointer to addressing mode of the device

Returns:

NVML_SUCCESS if mode is retrieved successfully
NVML_ERROR_ARGUMENT_VERSION_MISMATCH if the provided version is invalid/unsupported
NVML_ERROR_NOT_SUPPORTED if request is not supported on the current platform
NVML_ERROR_INVALID_ARGUMENT if device node is invalid

nvmlReturn_t nvmlDeviceGetCpuAffinity( nvmlDevice_t device, unsigned int cpuSetSize, unsigned long *cpuSet, )#

Retrieves an array of unsigned ints (sized to cpuSetSize) of bitmasks with the ideal CPU affinity for the device For example, if processors 0, 1, 32, and 33 are ideal for the device and cpuSetSize == 2, result[0] = 0x3, result[1] = 0x3 This is equivalent to calling nvmlDeviceGetCpuAffinityWithinScope with NVML_AFFINITY_SCOPE_NODE.

For Kepler or newer fully supported devices. Supported on Linux only.

Parameters:

device – The identifier of the target device
cpuSetSize – The size of the cpuSet array that is safe to access
cpuSet – Array reference in which to return a bitmask of CPUs, 64 CPUs per unsigned long on 64-bit machines, 32 on 32-bit machines

Returns:

NVML_SUCCESS if cpuAffinity has been filled
NVML_ERROR_UNINITIALIZED if the library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT if device is invalid, cpuSetSize == 0, or cpuSet is NULL
NVML_ERROR_NOT_SUPPORTED if the device does not support this feature
NVML_ERROR_GPU_IS_LOST if the target GPU has fallen off the bus or is otherwise inaccessible
NVML_ERROR_UNKNOWN on any unexpected error

nvmlReturn_t nvmlDeviceGetCpuAffinityWithinScope( nvmlDevice_t device, unsigned int cpuSetSize, unsigned long *cpuSet, nvmlAffinityScope_t scope, )#

Retrieves an array of unsigned ints (sized to cpuSetSize) of bitmasks with the ideal CPU affinity within node or socket for the device.

For example, if processors 0, 1, 32, and 33 are ideal for the device and cpuSetSize == 2, result[0] = 0x3, result[1] = 0x3

For Kepler or newer fully supported devices. Supported on Linux only.

Note

If requested scope is not applicable to the target topology, the API will fall back to reporting the CPU affinity for the immediate non-I/O ancestor of the device.

Parameters:

device – The identifier of the target device
cpuSetSize – The size of the cpuSet array that is safe to access
cpuSet – Array reference in which to return a bitmask of CPUs, 64 CPUs per unsigned long on 64-bit machines, 32 on 32-bit machines
scope – Scope that change the default behavior

Returns:

NVML_SUCCESS if cpuAffinity has been filled
NVML_ERROR_UNINITIALIZED if the library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT if device is invalid, cpuSetSize == 0, cpuSet is NULL or sope is invalid
NVML_ERROR_NOT_SUPPORTED if the device does not support this feature
NVML_ERROR_GPU_IS_LOST if the target GPU has fallen off the bus or is otherwise inaccessible
NVML_ERROR_UNKNOWN on any unexpected error

nvmlReturn_t nvmlDeviceGetMemoryAffinity( nvmlDevice_t device, unsigned int nodeSetSize, unsigned long *nodeSet, nvmlAffinityScope_t scope, )#

Retrieves an array of unsigned ints (sized to nodeSetSize) of bitmasks with the ideal memory affinity within node or socket for the device.

For example, if NUMA node 0, 1 are ideal within the socket for the device and nodeSetSize == 1, result[0] = 0x3

For Kepler or newer fully supported devices. Supported on Linux only.

Note

If requested scope is not applicable to the target topology, the API will fall back to reporting the memory affinity for the immediate non-I/O ancestor of the device.

Parameters:

device – The identifier of the target device
nodeSetSize – The size of the nodeSet array that is safe to access
nodeSet – Array reference in which to return a bitmask of NODEs, 64 NODEs per unsigned long on 64-bit machines, 32 on 32-bit machines
scope – Scope that change the default behavior

Returns:

NVML_SUCCESS if NUMA node Affinity has been filled
NVML_ERROR_UNINITIALIZED if the library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT if device is invalid, nodeSetSize == 0, nodeSet is NULL or scope is invalid
NVML_ERROR_NOT_SUPPORTED if the device does not support this feature
NVML_ERROR_GPU_IS_LOST if the target GPU has fallen off the bus or is otherwise inaccessible
NVML_ERROR_UNKNOWN on any unexpected error

nvmlReturn_t nvmlDeviceGetNumaNodeId( nvmlDevice_t device, unsigned int *node, )#

Get the NUMA node of the given GPU device.

This only applies to platforms where the GPUs are NUMA nodes.

Parameters:

device – [in] The device handle
node – [out] NUMA node ID of the device

Returns:

NVML_SUCCESS if the NUMA node is retrieved successfully
NVML_ERROR_NOT_SUPPORTED if request is not supported on the current platform
NVML_ERROR_INVALID_ARGUMENT if device node is invalid

nvmlReturn_t nvmlDeviceGetRepairStatus( nvmlDevice_t device, nvmlRepairStatus_t *repairStatus, )#

Get the repair status for TPC/Channel repair.

For Ampere or newer fully supported devices.

Parameters:

device – [in] The identifier of the target device
repairStatus – [out] Reference to nvmlRepairStatus_t

Returns:

NVML_SUCCESS if the query was successful
NVML_ERROR_ARGUMENT_VERSION_MISMATCH if the provided version is invalid/unsupported
NVML_ERROR_UNINITIALIZED if the library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT if device is invalid
NVML_ERROR_NOT_SUPPORTED if the device does not support this feature
NVML_ERROR_UNKNOWN on any unexpected error

nvmlReturn_t nvmlDeviceGetUnrepairableMemoryFlag_v1( nvmlDevice_t device, nvmlUnrepairableMemoryStatus_v1_t *unrepairableMemoryStatus, )#

Get the unrepairable memory flag for a given GPU.

For Hopper or newer fully supported devices.

Parameters:

device – [in] The identifier of the target device
unrepairableMemoryStatus – [out] Reference to nvmlUnrepairableMemoryStatus_v1_t

Returns:

NVML_SUCCESS if the query was successful
NVML_ERROR_ARGUMENT_VERSION_MISMATCH if the provided version is invalid/unsupported
NVML_ERROR_UNINITIALIZED if the library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT if device is invalid
NVML_ERROR_NOT_SUPPORTED if the device does not support this feature
NVML_ERROR_UNKNOWN on any unexpected error

nvmlReturn_t nvmlDeviceSetCpuAffinity(nvmlDevice_t device)#

Sets the ideal affinity for the calling thread and device using the guidelines given in nvmlDeviceGetCpuAffinity().

Note, this is a change as of version 8.0. Older versions set the affinity for a calling process and all children. Currently supports up to 1024 processors.

For Kepler or newer fully supported devices. Supported on Linux only.

Parameters:

device – The identifier of the target device

Returns:

NVML_SUCCESS if the calling process has been successfully bound
NVML_ERROR_UNINITIALIZED if the library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT if device is invalid
NVML_ERROR_NOT_SUPPORTED if the device does not support this feature
NVML_ERROR_GPU_IS_LOST if the target GPU has fallen off the bus or is otherwise inaccessible
NVML_ERROR_UNKNOWN on any unexpected error

Typedefs#

typedef unsigned int nvmlAffinityScope_t#