Changing Scheduling Behavior for Time-Sliced vGPUs and MIG-Backed, Time-Sliced vGPUs#
All GPUs that support NVIDIA vGPU software implement several vGPU schedulers, which control how GPU processing cycles are shared among vGPUs that reside on the same physical GPU.
On a single-instance GPU, you can change the scheduling behavior for all time-sliced vGPUs on the GPU. On a GPU that supports the Multi-Instance GPU (MIG) feature and is partitioned into multiple GPU instances, you can change the scheduling behavior for MIG-backed, time-sliced vGPUs within each GPU instance. The scheduling behavior for each GPU instance is set independently of the behavior for other GPU instances on the same GPU. The scheduling behavior that is set for a GPU instance applies to all the vGPUs that reside on the GPU instance
Scheduling Policies for Time-Sliced vGPUs#
All GPUs that support NVIDIA vGPU software implement the best effort, equal share, and fixed share vGPU schedulers. The best effort scheduler is the default scheduler for all supported GPU architectures.
- Best effort scheduler
The physical GPU’s processing cycles are shared in a way that aims to balance performance across vGPUs. The best effort scheduler allows a vGPU to use GPU processing cycles that are not being used by other vGPUs. Under some circumstances, a VM running a graphics-intensive application might adversely affect the performance of graphics-light applications running in other VMs. To address this issue, the equal share and fixed share schedulers impose a limit on GPU processing cycles used by a vGPU.
- Equal share scheduler
The physical GPU is shared equally amongst the running vGPUs that reside on it. As vGPUs are added to or removed from a GPU, the share of the GPU’s processing cycles allocated to each vGPU changes accordingly. As a result, the performance of a vGPU might increase as other vGPUs on the same GPU are stopped, or decrease as other vGPUs are started on the same GPU.
- Fixed share scheduler
Each vGPU is given a fixed share of the physical GPU’s processing cycles. As vGPUs are added to or removed from a GPU, the share of the GPU’s processing cycles allocated to each vGPU remains constant. As a result, the performance of a vGPU remains unchanged as other vGPUs are stopped or started on the same GPU.
The size of the share given to a vGPU depends on whether the GPU is in equal-size or mixed-size mode.
In equal-size mode, the size of the share given to a vGPU is inversely proportional to the maximum number of vGPUs of its type allowed per physical GPU.
For example, a maximum of three A10-8Q vGPUs are allowed on a physical GPU. Therefore, an A10-8Q vGPU is given one third (33%) of the physical GPU’s processing cycles.
In mixed-size mode, the size of the share given to a vGPU is proportional to the percentage of nominal GPU frame buffer memory allocated to the vGPU, which in turn depends on the vGPU type.
For example, in a configuration with an A10-8Q and an A10-4Q on the same GPU, the physical GPU’s processing cycles are shared as follows:
The A10-8Q vGPU is given one half (50%) of the physical GPU’s processing cycles.
The A10-4Q vGPU is given one quarter (25%) of the physical GPU’s processing cycles.
Note
If you use the equal share or fixed share vGPU scheduler, the frame-rate limiter (FRL) is disabled.
The equal share and fixed share schedulers impose a strict round-robin scheduling policy. This policy ensures that the schedulers maintain scheduling fairness by adjusting the time slice for each VM that is configured with NVIDIA vGPU. The strict round-robin scheduling policy ensures more consistent scheduling of the work for VMs that are configured with NVIDIA vGPU and restricts the impact of GPU-intensive applications running in one VM on applications running in other VMs.
Note
Unlike in earlier NVIDIA vGPU software releases, strict round robin policy cannot be disabled.
For schedulers that impose a strict round-robin scheduling policy, the adjustment to the time slice is based on the scheduling frequency and an averaging factor.
- Scheduling frequency
The number of times per second that work for a specific vGPU is scheduled. The default scheduling frequency depends on the number of vGPUs that reside on the physical GPU:
If fewer than eight vGPUs reside on the physical GPU, the default is 480 Hz.
If eight or more vGPUs reside on the physical GPU, the default is 960 Hz.
- Averaging factor
A number that determines the moving average of time-slice overshoots accrued for each vGPU. This average controls the strictness with which the scheduling frequency is enforced. A high value for the averaging factor enforces the scheduling frequency less strictly than a low value.
Deviations from the specified scheduling frequency occur because the actual amount of time that a scheduler allocates to a VM might exceed, or overshoot, the time slice specified for the VM. A scheduler enforces the scheduling frequency by shortening the next time slice for each vGPU VM to compensate for the accrued overshoot time of the VM.
To calculate the amount by which to shorten the next time slice for a vGPU VM, the scheduler maintains a running total of the accrued overshoot time for each vGPU VM. This amount is equal to the running total divided by the averaging factor that you specify. The calculated amount is also subtracted from the accrued overshoot time. A high value for the averaging factor enforces the scheduling frequency less strictly by spreading the compensation for the accrued overshoot time over a longer period.
Scheduler Time Slice for Time-Sliced vGPUs#
When multiple VMs access the vGPUs on a single GPU, the GPU performs the work for each VM serially. The vGPU scheduler time slice represents the amount of time that the work of a VM is allowed to run on the GPU before it is preempted and the work of the next VM is performed.
For the equal share and fixed share vGPU schedulers, you can set the length of the time slice. The length of the time slice affects latency and throughput. The optimal length of the time slice depends the workload that the GPU is handling.
For workloads that require low latency, a shorter time slice is optimal. Typically, these workloads are applications that must generate output at a fixed interval, such as graphics applications that generate output at a frame rate of 60 FPS. These workloads are sensitive to latency and should be allowed to run at least once per interval. A shorter time slice reduces latency and improves responsiveness by causing the scheduler to switch more frequently between VMs.
For workloads that require maximum throughput, a longer time slice is optimal. Typically, these workloads are applications that must complete their work as quickly as possible and do not require responsiveness, such as CUDA applications. A longer time slice increases throughput by preventing frequent switching between VMs.
Getting Information about the Scheduling Behavior of Time-Sliced vGPUs#
On all supported hypervisors, the nvidia-smi command provides options for getting detailed information about the scheduling behavior of time-sliced vGPUs. On Linux-based hypervisors, you can also use the hypervisor’s dmesg command to get the current time-sliced vGPU scheduling policy for all GPUs.
Getting Time-Sliced vGPU Scheduler Capabilities#
The scheduler capabilities of a time-sliced vGPU are a set of values that define how you can configure the vGPU to allocate the work for each VM that is configured with NVIDIA vGPU. These capability values depend on the vGPU engine type and, for vGPUs that support multiple scheduling policies, whether the vGPU supports and enforces a strict round-robin scheduling policy.
If the vGPU engine type is graphics, the vGPU scheduler capability values consist of the supported scheduling policies and other values that affect how the work for each VM that is configured with NVIDIA vGPU is allocated. The capability values that are applicable depend on whether the vGPU supports and enforces a strict round-robin scheduling policy.
If the vGPU supports and enforces a strict round-robin scheduling policy, the values for the scheduling frequency and averaging factor are applicable.
Otherwise, the values for the supported time slice range applicable.
If the vGPU engine type is any type other than graphics, the only vGPU scheduler capability value indicates support for the best effort scheduling policy. All other capability values are zero.
To get the scheduler capabilities of all time-sliced vGPUs on the platform, run nvidia-smi vgpu with the –sc or --schedulercaps option.
To limit the information retrieved to a subset of the GPUs on the platform, use the –i or --id option to select one or more GPUs.
[root@vgpu ~]# nvidia-smi vgpu -sc
vGPU scheduler capabilities
Supported Policies : Best Effort
Equal Share
Fixed Share
ARR Mode : Supported
Supported Timeslice Range
Maximum Timeslice : 30000000 ns
Minimum Timeslice : 1000000 ns
Supported Scheduling Frequency
Maximum Frequency : 960
Minimum Frequency : 63
Supported ARR Averaging Factor
Maximum Avg Factor : 60
Minimum Avg Factor : 1
Getting Time-Sliced vGPU Scheduler State Information#
The scheduler state information for a time-sliced vGPU consists of the scheduling policy set for the vGPU and the values of properties that control how the work for the VM that is configured with the vGPU is allocated. The properties available depend on the scheduling policy that is set for the vGPU.
The scheduler state information that can be retrieved for a vGPU depends on whether the VM that is configured with the vGPU is running.
To get scheduler state information for all time-sliced vGPUs on the platform, run nvidia-smi vgpu with the –ss or --schedulerstate option.
To limit the information retrieved to a subset of the GPUs on the platform, use the –i or --id option to select one or more GPUs.
For a MIG-enabled GPU, you must use the –gi or --gpu-instance-id option with the –i or --id option to select one or more GPU instances. If you use the –i or --id option without the –gi or --gpu-instance-id option, the nvidia-smi displays the following error message:
vGPU scheduler state not supported on device.
The following examples show the scheduler state information that is retrieved for a vGPU when the VM that is configured with the vGPU is not running and is running. In these examples, the scheduling policy is equal share scheduler, which imposes a strict round-robin scheduling policy, with the default scheduling frequency and averaging factor.
vGPU Scheduler State Information for a VM that Is Not Running#
Note
For a VM that is not running, Average Factor and Time Slice are not listed.
[root@vgpu ~]# nvidia-smi vgpu -ss
GPU 00000000:65:00.0
Active vGPUs : 0
Scheduler Policy : Equal Share
vGPU Scheduler State Information for a Running VM#
[root@vgpu ~]# nvidia-smi vgpu -ss
GPU 00000000:65:00.0
Active vGPUs : 1
Scheduler Policy : Equal Share
Average Factor : 33
Time Slice(ns) : 2083333
vGPU Scheduler State Information for a GPU Instance on a VM that Is Not Running#
In this example, the scheduling policy is equal share scheduler with a strict round-robin scheduling policy and the default time slice length, scheduling frequency, and averaging factor.
$ nvidia-smi vgpu -ss -gi 1 -i 0
GPU 00000000:C7:00.0
GPU Instance ID : 1
Active vGPUs : 0
Scheduler Policy : Equal Share
vGPU Scheduler State Information for a GPU Instance on a Running VM#
$ nvidia-smi vgpu -ss -gi 1 -i 0
GPU 00000000:C7:00.0
GPU Instance ID : 1
Active vGPUs : 1
Scheduler Policy : Equal Share
Average Factor : 33
Time Slice(ns) : 2083333
Getting Time-Sliced vGPU Scheduler Work Logs#
The scheduler work logs for a time-sliced vGPU provide information about the allocation at runtime of the work for the VM that is configured with the vGPU.
The information in the scheduler work logs that can be retrieved for a vGPU depends on whether the VM that is configured with the vGPU is running.
To get scheduler work logs for all time-sliced vGPUs on the platform, run nvidia-smi vgpu with the –sl or --schedulerlogs option.
To limit the information retrieved to a subset of the GPUs on the platform, use the –i or --id option to select one or more GPUs.
For a MIG-enabled GPU, you must use the –gi or --gpu-instance-id option with the –i or --id option to select one or more GPU instances. If you use the –i or --id option without the –gi or --gpu-instance-id option, the nvidia-smi displays the following error message:
Not supported on the device
To get the scheduler work logs in comma-separated values (CSV) format, run nvidia-smi vGPU --query-gpu-instance-vgpu-scheduler-logs with its options.
vGPU Scheduler Work Logs for a VM that Is Not Running#
[root@vgpu ~]# nvidia-smi vgpu -sl
+---------------------------------------------------------------------------------------------------------+
Engine Id 1
Scheduler Policy Equal Share
GPU at deviceIndex 0 has no active VM runlist.
+---------------------------------------------------------------------------------------------------------+
vGPU Scheduler Work Logs for a Running VM with a GPU in Equal-Size Mode#
[root@vgpu ~]# nvidia-smi vgpu -sl
+------------------------------------------------------------------------------------------------------------------+
GPU Id 0
Engine Id 1
Scheduler Policy Equal Share
Avg Factor 33
Time Slice 2083333
+------------------------------------------------------------------------------------------------------------------+
GPU SW Runlist Weight Time Cumulative Prev Timeslice Target Time Cumulative
Idx Id (0x) Stamp Run Time Runtime Slice Preempt Time
0 0 5555555 1770095639091125856 79042283264 2085184 1999627 2764156
0 1 5555555 1770095639093203904 79039958880 2078048 1998198 2804176
0 2 5555555 1770095639095287040 79038201344 2083136 1998560 2797323
0 0 5555555 1770095639097371008 79044367232 2083968 1999571 2764791
0 1 5555555 1770095639099454656 79042042528 2083648 1998358 2804491
0 2 5555555 1770095639101535744 79040282432 2081088 1998566 2795078
0 0 5555555 1770095639103621696 79046453184 2085952 1999552 2767410
vGPU Scheduler Work Logs for a Running VM with a GPU in Mixed-Size Mode#
[root@vgpu ~]# nvidia-smi vgpu -sl
+------------------------------------------------------------------------------------------------------------------+
GPU Id 0
Engine Id 1
Scheduler Policy Fixed Share
Avg Factor 33
Time Slice 1041666
+------------------------------------------------------------------------------------------------------------------+
GPU SW Runlist Weight Time Cumulative Prev Timeslice Target Time Cumulative
Idx Id (0x) Stamp Run Time Runtime Slice Preempt Time
0 2 4000000 1770095265490639840 12658108992 1044928 1016752 825426
0 33 7555556 1770095265491677824 506677753280 1037984 1019889 714966
0 33 7555556 1770095265492707232 506678782688 1029408 1020001 702708
0 33 7555556 1770095265493751456 506679826912 1044224 1020372 705266
0 2 4000000 1770095265494795328 12659152864 1043872 1016654 827632
0 1 2aaaaaa 1770095265495834656 8583022176 1039328 1016386 831912
0 33 7555556 1770095265496888416 506680880672 1053760 1020295 717360
vGPU Scheduler Work Logs for a GPU Instance on a VM that Is Not Running#
$ nvidia-smi vgpu -sl -gi 1 -i 0
+---------------------------------------------------------------------------------------------------------+
Engine Id 1
GPU Instance ID 1
Scheduler Policy Equal Share
GPU at deviceIndex 0 has no active VM runlist.
+---------------------------------------------------------------------------------------------------------+
vGPU Scheduler Work Logs for a GPU Instance on a Running VM#
$ nvidia-smi vgpu -sl -gi 1 -i 0
+---------------------------------------------------------------------------------------------------------+
GPU Id 0
GPU Instance ID 1
Engine Id 1
Scheduler Policy Equal Share
Avg Factor 33
Time Slice 2083333
+------------------------------------------------------------------------------------------------------------------+
GPU SW Runlist Weight Time Cumulative Prev Timeslice Target Time Cumulative
Idx Id (0x) Stamp Run Time Runtime Slice Preempt Time
0 0 4000000 1770095769748761120 3658665120 2080448 2057586 846768
0 33 4000000 1770095769750841248 534416170560 2080128 2056502 882227
0 2 4000000 1770095769752920160 3661329728 2078912 2055513 913647
0 1 4000000 1770095769754997600 3661458880 2077440 2054029 961163
0 0 4000000 1770095769757134976 3660802496 2137376 2057674 900811
0 33 4000000 1770095769759217088 534418252672 2082112 2056599 881006
0 2 4000000 1770095769761296672 3663409312 2079584 2055647 909898
vGPU Scheduler Work Logs in CSV Format for a GPU Instance on a VM that Is Not Running#
Note
Line breaks have been added to this example to enhance readability.
$ nvidia-smi vgpu --query-gpu-instance-vgpu-scheduler-logs=gpu_bus_id,gi_id,engine_id,scheduler_policy,
time_slice,sw_runlist_id,time_stamp,cumulative_run_time,previous_timeslice_runtime,target_time_slice,
cumulative_preempt_time -gi 1 -i 0 --format=csv
gpu_bus_id, gi_id, engine_id, scheduler_policy, time_slice, sw_runlist_id, time_stamp, cumulative_run_time,
previous_timeslice_runtime, target_time_slice, cumulative_preempt_time
00000000:C7:00.0, 1, 1, Equal Share, -, -, -, -, -, -, -
vGPU Scheduler Work Logs in CSV Format for a GPU Instance on a Running VM#
Note
Line breaks have been added to this example to enhance readability.
$ nvidia-smi vgpu --query-gpu-instance-vgpu-scheduler-logs=gpu_bus_id,gi_id,engine_id,scheduler_policy,
time_slice,sw_runlist_id,time_stamp,cumulative_run_time,previous_timeslice_runtime,target_time_slice,
cumulative_preempt_time -gi 1 -i 0 --format=csv
gpu_bus_id, gi_id, engine_id, scheduler_policy, time_slice, sw_runlist_id, time_stamp, cumulative_run_time,
previous_timeslice_runtime, target_time_slice, cumulative_preempt_time
00000000:C7:00.0, 1, 1, Equal Share, 2083333, 0, 1751920964898612528, 481321948480, 2084800, 2047252, 1192158
00000000:C7:00.0, 1, 1, Equal Share, 2083333, 0, 1751920964900693296, 481324029248, 2080768, 2047207, 1189593
00000000:C7:00.0, 1, 1, Equal Share, 2083333, 0, 1751920964902778160, 481326114112, 2084864, 2047285, 1191124
00000000:C7:00.0, 1, 1, Equal Share, 2083333, 0, 1751920964904863024, 481328198976, 2084864, 2047239, 1192655
Linux-Based Hypervisors Only: Getting the Current Time-Sliced vGPU Scheduling Policy for All GPUs#
On Linux-based hypervisors, you can use the dmesg command to get the current time-sliced vGPU scheduling policy for all GPUs. Get this information before changing the scheduling behavior of one or more GPUs to determine if you need to change it or after changing it to confirm the change.
Note
You cannot perform this task on Windows hypervisors because the dmesg command is available only on Linux-based hypervisors.
Perform this task in your hypervisor command shell.
Open a command shell on your hypervisor host machine.
On all supported hypervisors, you can use secure shell (SSH) for this purpose. Individual hypervisors may provide additional means for logging in. For details, refer to the documentation for your hypervisor.
Use the
dmesgcommand to display messages from the kernel that contain the stringsNVRMandscheduler.$ dmesg | grep NVRM | grep scheduler
The scheduling policy is indicated in these messages by the following strings:
BEST_EFFORTEQUAL_SHAREFIXED_SHARE
If the scheduling policy is equal share or fixed share, the scheduler time slice in ms is also displayed.
This example gets the scheduling policy of the GPUs in a system in which the policy of one GPU is set to best effort, one GPU is set to equal share, and one GPU is set to fixed share.
$ dmesg | grep NVRM | grep scheduler 2020-10-05T02:58:08.928Z cpu79:2100753)NVRM: GPU at 0000:3d:00.0 has software scheduler DISABLED with policy BEST_EFFORT. 2020-10-05T02:58:09.818Z cpu79:2100753)NVRM: GPU at 0000:5e:00.0 has software scheduler ENABLED with policy EQUAL_SHARE. NVRM: Software scheduler timeslice set to 1 ms. 2020-10-05T02:58:12.115Z cpu79:2100753)NVRM: GPU at 0000:88:00.0 has software scheduler ENABLED with policy FIXED_SHARE. NVRM: Software scheduler timeslice set to 1 ms.
Tools for Changing Scheduling Behavior for Time-Sliced vGPUs#
To change the scheduling behavior for time-sliced vGPUs, you can use the nvidia-smi command or the RmPVMRL registry key. The tool to use depends on whether you require the changes to be applied immediately or whether you require the changes to be persistent.
If you require the changes to be applied immediately, use the
nvidia-smicommand.If you use the
nvidia-smicommand, you do not need to reload the driver or reboot the hypervisor host to apply your changes. However, your changes are volatile and do not persist in the following circumstances:The driver is reloaded.
The hypervisor host is rebooted.
The
sriov-managescript is run to enable the virtual functions for the physical GPU in thesysfsfile system.
If you require the changes to be persistent, use the
RmPVMRLregistry key.However, if you use the
RmPVMRLregistry key, you must reload the driver or reboot the hypervisor host to apply your changes.
For information about how to use these tools to change the scheduling behavior for time-sliced vGPUs, refer to the following topics:
Changing Scheduling Behavior for Time-Sliced vGPUs by Using the nvidia-smi Command
Changing Scheduling Behavior for Time-Sliced vGPUs by Using the RmPVMRL Registry Key
Note
Unlike in earlier NVIDIA vGPU software releases, strict round robin policy cannot be disabled.
Changing Scheduling Behavior for Time-Sliced vGPUs by Using the nvidia-smi Command#
The nvidia-smi command provides the vgpu set-scheduler-state subcommand and associated options for changing the scheduling behavior of time-sliced vGPUs. Because the nvidia-smi command is available on all supported hypervisors, you can use this method for Linux-style hypervisors and for Windows hypervisors.
Note
You cannot use the nvidia-smi command to change the scheduling behavior of MIG-backed, time-sliced vGPUs in VMware vSphere ESXi. You must use the RmPVMRL registry key for this purpose. Furthermore, when you use the RmPVMRL registry key for this purpose, you can change the scheduling behavior only for all GPU instances on a GPU. You cannot change the scheduling behavior for select GPU instances.
Ensure that no vGPUs exist on any physical GPU for which you want to change the scheduling behavior for time-sliced vGPUs. Any change that you make affects vGPUs that will be created on the physical GPU after you make the change.
If you try to change the scheduling behavior for time-sliced vGPUs on a physical GPU or GPU instance on which a vGPU already exists, the attempt to change the scheduling behavior fails.
For a physical GPU, the
nvidia-smicommand displays an error message similar to the following example:Unable to set the vGPU scheduler state on GPU "00000000:1A:00.0". vGPU scheduler state cannot be configured, if vGPU instance is currently active on the device.
For a GPU instance, the
nvidia-smicommand displays an error message similar to the following example:Unable to set the vGPU scheduler state on GI Id 1. vGPU scheduler state cannot be configured, if vGPU instance is currently active on the GI.
To change the scheduling behavior for time-sliced vGPUs, run nvidia-smi vgpu set-scheduler-state with its associated options.
For more information about these options, refer to Scheduling Policies for Time-Sliced vGPUs.
–i<gpu-id>,--id<gpu-id><gpu-id>is the identifier of the GPU on which you want to change the scheduling behavior of time-sliced vGPUs in one of the following formats:The GPU’s 0-based index in the natural enumeration returned by the driver
The GPU’s universally unique identifier (UUID)
The GPU’s PCI bus ID in the form
<domain>:<bus>:<device>.<function>in hexadecimal.
This option is not mandatory. If it is omitted, the scheduling behavior of time-sliced vGPUs for all GPUs on the platform is changed.
–gi<gpu-instance-id>,--gpu-instance-id<gpu-instance-id><gpu-instance-id>is the identifier of the GPU instance on which you want to change the scheduling behavior of time-sliced vGPUs.For a MIG-enabled GPU, you must use this option with the
–ior--idoption to select one or more GPU instances. If you use the–ior--idoption without this option, thenvidia-smidisplays the following error message:Unable to set the vGPU scheduler state. Not supported on device "<device-id>".Note
This option is not supported by VMware vSphere ESXi.
-p<S>,--policy<S><S>is a decimal integer in the range 1-3 that sets the scheduler to use:1: Best effort scheduler (default)2: Equal share scheduler3: Fixed share scheduler
If
<S>is not a decimal integer in the range 1-3, the attempt to set the scheduler to use fails and thenvidia-smicommand displays the following error message:Unable to set the vGPU scheduler state. Not supported-asf<frequency>,--arr-sched-frequency<frequency><frequency>is a decimal integer in the range 63 to 960 that sets the scheduling frequency in Hz for the equal share and fixed share schedulers with a strict round-robin scheduling policy.If
<frequency>is outside the range 63 to 960, the scheduling frequency is set as follows:If
<frequency>is not set, the scheduling frequency is set to the default scheduling frequency for the vGPU type as listed in Table 1.If
<frequency>is less than 63, the scheduling frequency is raised to 63.If
<frequency>is greater than 960, the scheduling frequency is capped at 960.
-aavg<averaging-factor>,--arr-avg-factor<averaging-factor><averaging-factor>is a decimal integer in the range 1 to 60 that sets the averaging factor to ensure scheduling fairness for the equal share and fixed share schedulers with a strict round-robin scheduling policy.The number of time slices over which the compensation for the accrued overshoot time is applied depends on the value of
<averaging-factor>:If
<averaging-factor>is 1, the compensation for the accrued overshoot time is applied in a single time slice.If
<averaging-factor>is 60, the compensation for the accrued overshoot time is spread over 60 time slices.If
<averaging-factor>is not set, the default value of 33 is used.If
<averaging-factor>is greater than 60, the number of time slices over which the compensation is applied is capped at 60.
Note
The following options from earlier NVIDIA vGPU software releases have been removed:
--arr-mode<R>. Adaptive Round Robin (ARR) scheduling cannot be disabled.--time-slice<time-slice-length>. The time-slice length could be set only when ARR scheduling was disabled but ARR scheduling can no longer be disabled.
Setting the Scheduling Policy for a Single GPU#
This example sets the scheduling policy of the GPU at PCI domain 0000 and BDF 15:00.0 to fixed share scheduler with a strict round-robin scheduling policy and with the default time slice length.
# nvidia-smi vgpu set-scheduler-state –i 0000:15:00.0 -p 3
Restoring Default Time-Sliced vGPU Scheduler Settings#
This example restores default time-sliced vGPU scheduler settings by setting the vGPU scheduler to best effort scheduler.
# nvidia-smi vgpu set-scheduler-state -p 1
Setting the Scheduling Policy for a Single GPU Instance#
This example sets the vGPU scheduler to equal share scheduler with a strict round-robin scheduling policy and the default time slice length, scheduling frequency, and averaging factor for GPU instance 1 on GPU 0.
$ nvidia-smi vgpu set-scheduler-state -p 2 -gi 1 -i 0
Restoring Default Time-Sliced vGPU Scheduler Settings for a Single GPU Instance#
This example restores default time-sliced vGPU scheduler settings for GPU instance 1 on GPU 0 by setting the vGPU scheduler to best effort scheduler.
$ nvidia-smi vgpu set-scheduler-state -p 1 -gi 1 -i 0
Changing Scheduling Behavior for Time-Sliced vGPUs by Using the RmPVMRL Registry Key#
The RmPVMRL registry key controls the scheduling behavior for NVIDIA vGPUs by setting the scheduling policy, the averaging factor and scheduling frequency for schedulers with a strict round-robin scheduling policy. How to use the RmPVMRL registry key for this purpose depends on whether you are using a Linux-based hypervisor or a Windows hypervisor.
For detailed instructions, refer to the following topics:
Linux-Based Hypervisors: Changing Scheduling Behavior for Time-Sliced vGPUs by Using the RmPVMRL Registry Key#
To use the RmPVMRL registry key to change the scheduling behavior of time-sliced vGPUs, use the standard interfaces of your hypervisor to set the RmPVMRL registry key value.
Linux-Based Hypervisors: Changing the Time-Sliced vGPU Scheduling Behavior for All GPUs by Using the RmPVMRL Registry Key#
Note
You cannot perform this task on Windows hypervisors because it relies on interfaces that are available only on Linux-based hypervisors.
Perform this task in your hypervisor command shell.
Open a command shell on your hypervisor host machine.
On all supported hypervisors, you can use secure shell (SSH) for this purpose. Individual hypervisors may provide additional means for logging in. For details, refer to the documentation for your hypervisor.
Set the
RmPVMRLregistry key to the value that sets the GPU scheduling policy and the length of the time slice that you want.On XenServer or Red Hat Enterprise Linux KVM, add the following entry to the
/etc/modprobe.d/nvidia.conffile.options nvidia NVreg_RegistryDwords="RmPVMRL=<value>"If the
/etc/modprobe.d/nvidia.conffile does not already exist, create it.On VMware vSphere, use the
esxcli setcommand.# esxcli system module parameters set -m <module> -p "NVreg_RegistryDwords=RmPVMRL=<value>"
For the NVIDIA Ada Lovelace GPU architecture and later GPU architectures,
<module>isnvidia-gpu.For GPU architectures before NVIDIA Ada Lovelace,
<module>isnvidia.
<value>The value that sets the GPU scheduling policy and the length of the time slice that you want, for example:
0x01Sets the vGPU scheduling policy to equal share scheduler with the default time slice length.
0x3c3c0001Sets the GPU scheduling policy to equal share scheduler with a strict round-robin scheduling policy, an averaging factor of 60 (0x3C), and a scheduling frequency of 960 (0x3C0) Hz.
0x11Sets the vGPU scheduling policy to fixed share scheduler with the default time slice length.
0x3c3c0011Sets the GPU scheduling policy to fixed share scheduler with a strict round-robin scheduling policy, an averaging factor of 60 (0x3C), and a scheduling frequency of 960 (0x3C0) Hz.
For all supported values, see RmPVMRL Registry Key.
Reboot your hypervisor host machine.
Confirm that the scheduling behavior was changed as required as explained in Linux-Based Hypervisors Only: Getting the Current Time-Sliced vGPU Scheduling Policy for All GPUs.
Linux-Based Hypervisors: Changing the Time-Sliced vGPU Scheduling Behavior for Select GPUs by Using the RmPVMRL Registry Key#
Perform this task in your hypervisor command shell.
Open a command shell on your hypervisor host machine.
On all supported hypervisors, you can use secure shell (SSH) for this purpose. Individual hypervisors may provide additional means for logging in. For details, refer to the documentation for your hypervisor.
Use the
lspcicommand to obtain the PCI domain and bus/device/function (BDF) of each GPU for which you want to change the scheduling behavior.On XenServer or Red Hat Enterprise Linux KVM, add the
-Doption to display the PCI domain and the-d 10de:option to display information only for NVIDIA GPUs.# lspci -D -d 10de:
On VMware vSphere, pipe the output of
lspcito thegrepcommand to display information only for NVIDIA GPUs.# lspci | grep NVIDIA
The NVIDIA GPU listed in this example has the PCI domain
0000and BDF86:00.0.0000:86:00.0 3D controller: NVIDIA Corporation GP104GL [Tesla P4] (rev a1)Use the module parameter
NVreg_RegistryDwordsPerDeviceto set thepciandRmPVMRLregistry keys for each GPU.On XenServer or Red Hat Enterprise Linux KVM, add the following entry to the
/etc/modprobe.d/nvidia.conffile.options nvidia NVreg_RegistryDwordsPerDevice="pci=<pci-domain>:<pci-bdf>;RmPVMRL=<value> [;pci=<pci-domain>:<pci-bdf>;RmPVMRL=<value>...]"
If the
/etc/modprobe.d/nvidia.conffile does not already exist, create it.On VMware vSphere, use the
esxcli setcommand.# esxcli system module parameters set -m <module>\ -p "NVreg_RegistryDwordsPerDevice=pci=<pci-domain>:<pci-bdf>;RmPVMRL=<value>\ [;pci=<pci-domain>:<pci-bdf>;RmPVMRL=<value>...]"
For the NVIDIA Ada Lovelace GPU architecture and later GPU architectures,
<module>isnvidia-gpu.For GPU architectures before NVIDIA Ada Lovelace,
<module>isnvidia.
For each GPU, provide the following information:
<pci-domain>The PCI domain of the GPU.
<pci-bdf>The PCI device BDF of the GPU.
<value>The value that sets the GPU scheduling policy and the length of the time slice that you want, for example:
0x01Sets the GPU scheduling policy to equal share scheduler with the default time slice length.
0x00030001Sets the GPU scheduling policy to equal share scheduler with a time slice that is 3 ms long.
0x11Sets the GPU scheduling policy to fixed share scheduler with the default time slice length.
0x00180011Sets the GPU scheduling policy to fixed share scheduler with a time slice that is 24 (0x18) ms long.
For all supported values, see RmPVMRL Registry Key.
This example adds an entry to the
/etc/modprobe.d/nvidia.conffile to change the scheduling behavior of a single GPU. The entry sets the GPU scheduling policy of the GPU at PCI domain0000and BDF86:00.0to fixed share scheduler with the default time slice length.options nvidia NVreg_RegistryDwordsPerDevice= "pci=0000:86:00.0;RmPVMRL=0x11"
This example adds an entry to the
/etc/modprobe.d/nvidia.conffile to change the scheduling behavior of a single GPU. The entry sets the scheduling policy of the GPU at PCI domain0000and BDF86:00.0to fixed share scheduler with a time slice that is 24 (0x18) ms long.options nvidia NVreg_RegistryDwordsPerDevice= "pci=0000:86:00.0;RmPVMRL=0x00180011"
This example changes the scheduling behavior of a single GPU on a hypervisor host that is running VMware vSphere. The command sets the scheduling policy of the GPU at PCI domain
0000and BDF15:00.0to fixed share scheduler with the default time slice length.# esxcli system module parameters set -m nvidia -p \ "NVreg_RegistryDwordsPerDevice=pci=0000:15:00.0;RmPVMRL=0x11[;pci=0000:15:00.0;RmPVMRL=0x11]"
This example changes the scheduling behavior of a single GPU on a hypervisor host that is running VMware vSphere. The command sets the scheduling policy of the GPU at PCI domain
0000and BDF15:00.0to fixed share scheduler with a time slice that is 24 (0x18) ms long.# esxcli system module parameters set -m nvidia -p \ "NVreg_RegistryDwordsPerDevice=pci=0000:15:00.0;RmPVMRL=0x11[;pci=0000:15:00.0;RmPVMRL=0x00180011]"
Reboot your hypervisor host machine.
Confirm that the scheduling behavior was changed as required as explained in Linux-Based Hypervisors Only: Getting the Current Time-Sliced vGPU Scheduling Policy for All GPUs.
Linux-Based Hypervisors: Restoring Default Time-Sliced vGPU Scheduler Settings by Using the RmPVMRL Registry Key#
Perform this task in your hypervisor command shell.
Open a command shell on your hypervisor host machine.
On all supported hypervisors, you can use secure shell (SSH) for this purpose. Individual hypervisors may provide additional means for logging in. For details, refer to the documentation for your hypervisor.
Unset the
RmPVMRLregistry key.On XenServeror Red Hat Enterprise Linux KVM, comment out the entries in the
/etc/modprobe.d/nvidia.conffile that setRmPVMRLby prefixing each entry with the#character.On VMware vSphere, set the module parameter to an empty string.
# esxcli system module parameters set -m <module> -p "<module-parameter>="
<module>For the NVIDIA Ada Lovelace GPU architecture and later GPU architectures,
<module>isnvidia-gpu.For GPU architectures before NVIDIA Ada Lovelace,
<module>isnvidia.
<module-parameter>The module parameter to set, which depends on whether the scheduling behavior was changed for all GPUs or select GPUs:
For all GPUs, set the
NVreg_RegistryDwordsmodule parameter.For select GPUs, set the
NVreg_RegistryDwordsPerDevicemodule parameter.
For example, to restore default vGPU scheduler settings after they were changed for all GPUs, enter this command:
# esxcli system module parameters set -m nvidia -p "NVreg_RegistryDwords="
Reboot your hypervisor host machine.
Windows Hypervisors: Changing Scheduling Behavior for Time-Sliced vGPUs by Using the RmPVMRL Registry Key#
To use the RmPVMRL registry key to change the scheduling behavior of time-sliced vGPUs, use the Windows Registry Editor to set the RmPVMRL registry key value.
Windows Hypervisors: Getting the Driver Key of a GPU#
The RmPVMRL registry key of a GPU on a host running a Windows hypervisor is located in the GPU’s driver key. When setting the RmPVMRL registry key value for a GPU, you need the GPU’s driver key to identify the Windows registry key in which information about the GPU is stored.
Perform this task in a Windows PowerShell window on the hypervisor host.
Get the
InstanceIDproperty of all available NVIDIA GPUs in your system.PS C:\> Get-PnpDevice -PresentOnly | >> Where-Object {$_.InstanceId -like "PCI\VEN_10DE*" } | >> Select-Object -Property FriendlyName,InstanceId | >> Format-List ... FriendlyName : NVIDIA A40 InstanceId : PCI\VEN_10DE&DEV_2235&SUBSYS_145A10DE&REV_A1\6&2D4D2F51&0&00800018 ...
Note
If multiple NVIDIA GPUs are available on the system, analyze the output from this command to get the
InstanceIDproperty of the GPU on which you want to set the allowed vGPU series.Get the
DEVPKEY_Device_Driverproperty of the GPU from theInstanceIDproperty that you got in the previous step.PS C:\> Get-PnpDeviceProperty -InstanceId "<instance-id>" | >> where {$_.KeyName -eq "DEVPKEY_Device_Driver"} | >> Select-Object -Property Data Data ---- {4d36e968-e325-11ce-bfc1-08002be10318}\0001
<instance-id>The
InstanceIDproperty of the GPU that you got in the previous step, for example,PCI\VEN_10DE&DEV_2235&SUBSYS_145A10DE&REV_A1\6&2D4D2F51&0&00800018.
Windows Hypervisors: Changing the Time-Sliced vGPU Scheduling Behavior for Select GPUs by Using the RmPVMRL Registry Key#
Perform this task on the hypervisor host.
Get the driver key of the GPU on which you want to change the scheduling behavior.
For instructions, refer to Windows Hypervisors: Getting the Driver Key of a GPU.
You will need this information in the next step to identify the Windows registry key in which information about the GPU is stored.
In the Windows Registry Editor, navigate to the Windows registry key
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\<driver-key>.<driver-key>The driver key for the GPU that you got in the previous step, for example:
{4d36e968-e325-11ce-bfc1-08002be10318}\0001
Set the
RmPVMRLDWord (REG_DWORD) registry value to a hexadecimal integer that specifies the GPU scheduling policy and the length of the time slice that you want.For example:
0x01Sets the GPU scheduling policy to equal share scheduler with the default time slice length.
0x00030001Sets the GPU scheduling policy to equal share scheduler with a time slice that is 3 ms long.
0x11Sets the GPU scheduling policy to fixed share scheduler with the default time slice length.
0x00180011Sets the GPU scheduling policy to fixed share scheduler with a time slice that is 24 (0x18) ms long.
For all supported values, refer to RmPVMRL Registry Key.
Reboot your hypervisor host machine.
Confirm that the scheduling behavior was changed as required as explained in Getting Time-Sliced vGPU Scheduler State Information.
Windows Hypervisors: Restoring Default Time-Sliced vGPU Scheduler Settings by Using the RmPVMRL Registry Key#
Perform this task on the hypervisor host.
Get the driver key of the GPU on which you want to change the scheduling behavior.
For instructions, refer to Windows Hypervisors: Getting the Driver Key of a GPU.
You will need this information in the next step to identify the Windows registry key in which information about the GPU is stored.
In the Windows Registry Editor, navigate to the Windows registry key
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\<driver-key>.<driver-key>The driver key for the GPU that you got in the previous step, for example:
{4d36e968-e325-11ce-bfc1-08002be10318}\0001
Remove the
RmPVMRLDWord (REG_DWORD) registry value.Reboot your hypervisor host machine.
RmPVMRL Registry Key#
The RmPVMRL registry key controls the scheduling behavior for NVIDIA vGPUs by setting the scheduling policy, the averaging factor and scheduling frequency for schedulers with a strict round-robin scheduling policy.
Note
Unlike in earlier NVIDIA vGPU software releases, strict round robin policy cannot be disabled.
Type#
Dword
Contents#
Value |
Meaning |
|---|---|
|
Best effort scheduler |
|
Equal share scheduler with a strict round-robin scheduling policy and the default time slice length, scheduling frequency, and averaging factor |
|
Unlike in earlier NVIDIA vGPU software releases, this value is invalid because strict round robin policy cannot be disabled. |
|
Equal share scheduler with a strict round-robin scheduling policy and a user-defined averaging factor |
|
Unlike in earlier NVIDIA vGPU software releases, this value is invalid because strict round robin policy cannot be disabled and a user-defined time slice length |
|
Fixed share scheduler with a strict round-robin scheduling policy and the default time slice length, scheduling frequency, and averaging factor |
|
Unlike in earlier NVIDIA vGPU software releases, this value is invalid because strict round robin policy cannot be disabled. |
|
Fixed share scheduler with a strict round-robin scheduling policy and a user-defined averaging factor |
|
Unlike in earlier NVIDIA vGPU software releases, this value is invalid because strict round robin policy cannot be disabled and a user-defined time slice length |
The default time slice length and scheduling frequency depend on the maximum number of vGPUs per physical GPU allowed for the vGPU type.
Maximum Number of vGPUs |
Default Time Slice Length |
Default Scheduling Frequency |
|---|---|---|
Less than or equal to 8 |
2 ms |
480 Hz |
Greater than 8 |
1 ms |
960 Hz |
<AA>Two hexadecimal digits in the range 0x01 to 0x3C (decimal 1-60) that set the averaging factor for the equal share and fixed share schedulers with a strict round-robin scheduling policy.
The number of time slices over which the compensation for the accrued overshoot time is applied depends on the value of
<AA>:If
<AA>is 0x01, the compensation for the accrued overshoot time is applied in a single time slice.If
<AA>is 0x3C, the compensation for the accrued overshoot time is spread over 60 (0x3C) time slices.If
<AA>is 0x00, the default value of 33 is used.If
<AA>is greater than 0x3C, the value is capped at 0x3C.
<FFF>Three hexadecimal digits in the range 0x3F to 0x3C0 (decimal 63-960) that set the scheduling frequency for the equal share and fixed share schedulers with a strict round-robin scheduling policy. The time slice is the inverse of scheduling frequency. For example, a frequency of 0x3F (63 Hz) yields a time slice of 1/63 s, or 15.873 ms.
A value of 0x100 for
<FFF>sets the scheduling frequency to 256.If
<FFF>is outside the range 0x3F to 0x3C0, the scheduling frequency is set as follows:If
<FFF>is 000, the scheduling frequency is set to the default scheduling frequency for the vGPU type as listed in Table 1.If
<FFF>is greater than 000 but less than 0x3F, the scheduling frequency is raised to 0x3F (decimal 63).If
<FFF>is greater than 0x3C0, the scheduling frequency is capped at 0x3C0 (decimal 960).
Examples#
This example sets the vGPU scheduler to equal share scheduler with a strict round-robin scheduling policy and the default time slice length, scheduling frequency, and averaging factor.
RmPVMRL=0x01
This example sets the vGPU scheduler to fixed share scheduler with a strict round-robin scheduling policy and the default time slice length, scheduling frequency, and averaging factor.
RmPVMRL=0x11
This example sets the vGPU scheduler to equal share scheduler with a strict round-robin scheduling policy, an averaging factor of 60 (0x3C), and a scheduling frequency of 960 (0x3C0) Hz.
RmPVMRL=0x3c3c0001
This example sets the vGPU scheduler to fixed share scheduler with a strict round-robin scheduling policy, an averaging factor of 60 (0x3C), and a scheduling frequency of 960 (0x3C0) Hz.
RmPVMRL=0x3c3c0011