Changing Scheduling Behavior for Time-Sliced vGPUs and MIG-Backed, Time-Sliced vGPUs#
NVIDIA GPUs based on the NVIDIA Maxwell™ graphic architecture implement a best effort vGPU scheduler that aims to balance performance across vGPUs. The best effort scheduler allows a vGPU to use GPU processing cycles that are not being used by other vGPUs. Under some circumstances, a VM running a graphics-intensive application may adversely affect the performance of graphics-light applications running in other VMs.
GPUs based on NVIDIA GPU architectures after the Maxwell architecture additionally support equal share and fixed share vGPU schedulers. These schedulers impose a limit on GPU processing cycles used by a vGPU, which prevents graphics-intensive applications running in one VM from affecting the performance of graphics-light applications running in other VMs. On GPUs that support multiple vGPU schedulers, you can select the vGPU scheduler to use. You can also set the length of the time slice for the equal share and fixed share vGPU schedulers.
Note
If you use the equal share or fixed share vGPU scheduler, the frame-rate limiter (FRL) is disabled.
On GPUs that support the Multi-Instance GPU (MIG) feature, you can change the scheduling behavior for MIG-backed, time-sliced vGPUs within a GPU instance. The scheduling behavior for each GPU instance is set independently of the behavior for other GPU instances on the same GPU. The scheduling behavior that is set for a GPU instance applies to all the vGPUs that reside on the GPU instance.
The best effort scheduler is the default scheduler for all supported GPU architectures.
If you are unsure of the NVIDIA GPU architecture of your GPU, consult the release notes for your hypervisor at NVIDIA Virtual GPU Software Documentation.
Scheduling Policies for Time-Sliced vGPUs#
In addition to the default best effort scheduler, GPUs based on NVIDIA GPU architectures after the Maxwell architecture support equal share and fixed share vGPU schedulers.
- Equal share scheduler
The physical GPU is shared equally amongst the running vGPUs that reside on it. As vGPUs are added to or removed from a GPU, the share of the GPU’s processing cycles allocated to each vGPU changes accordingly. As a result, the performance of a vGPU may increase as other vGPUs on the same GPU are stopped, or decrease as other vGPUs are started on the same GPU.
- Fixed share scheduler
Each vGPU is given a fixed share of the physical GPU’s processing cycles, the amount of which depends on the vGPU type, which in turn determines the maximum number of vGPUs per physical GPU. For example, the maximum number of T4-4Q vGPUs per physical GPU is 4. When the scheduling policy is fixed share, each T4-4Q vGPU is given one quarter, or 25%, the physical GPU’s processing cycles. As vGPUs are added to or removed from a GPU, the share of the GPU’s processing cycles allocated to each vGPU remains constant. As a result, the performance of a vGPU remains unchanged as other vGPUs are stopped or started on the same GPU.
Note
For time-sliced vGPUs with different amounts of frame buffer on the same physical GPU, only the best effort and equal share schedulers are supported. The fixed share scheduler is not supported.
By default, these schedulers impose a strict round-robin scheduling policy. When this policy is enforced, the schedulers maintain scheduling fairness by adjusting the time slice for each VM that is configured with NVIDIA vGPU. The strict round-robin scheduling policy ensures more consistent scheduling of the work for VMs that are configured with NVIDIA vGPU and restricts the impact of GPU-intensive applications running in one VM on applications running in other VMs.
Instead of a strict round-robin scheduling policy, you can ensure scheduling fairness by scheduling the work for the vGPU that has spent the least amount of time in the scheduled state. This behavior was the default scheduling behavior in NVIDIA vGPU software releases before 15.0.
Note
Disabling strict round robin policy is deprecated and NVIDIA vGPU software 19 is the last release branch to support it. Support for this feature is planned to be removed in the next major release of NVIDIA vGPU software.
When a strict round-robin scheduling policy is enforced, the adjustment to the time slice is based on the scheduling frequency and an averaging factor.
- Scheduling frequency
The number of times per second that work for a specific vGPU is scheduled. The default scheduling frequency depends on the number of vGPUs that reside on the physical GPU:
If fewer than eight vGPUs reside on the physical GPU, the default is 480 Hz.
If eight or more vGPUs reside on the physical GPU, the default is 960 Hz.
- Averaging factor
A number that determines the moving average of time-slice overshoots accrued for each vGPU. This average controls the strictness with which the scheduling frequency is enforced. A high value for the averaging factor enforces the scheduling frequency less strictly than a low value.
Deviations from the specified scheduling frequency occur because the actual amount of time that a scheduler allocates to a VM might exceed, or overshoot, the time slice specified for the VM. A scheduler enforces the scheduling frequency by shortening the next time slice for each vGPU VM to compensate for the accrued overshoot time of the VM.
To calculate the amount by which to shorten the next time slice for a vGPU VM, the scheduler maintains a running total of the accrued overshoot time for each vGPU VM. This amount is equal to the running total divided by the averaging factor that you specify. The calculated amount is also subtracted from the accrued overshoot time. A high value for the averaging factor enforces the scheduling frequency less strictly by spreading the compensation for the accrued overshoot time over a longer period.
Scheduler Time Slice for Time-Sliced vGPUs#
When multiple VMs access the vGPUs on a single GPU, the GPU performs the work for each VM serially. The vGPU scheduler time slice represents the amount of time that the work of a VM is allowed to run on the GPU before it is preempted and the work of the next VM is performed.
For the equal share and fixed share vGPU schedulers, you can set the length of the time slice. The length of the time slice affects latency and throughput. The optimal length of the time slice depends the workload that the GPU is handling.
For workloads that require low latency, a shorter time slice is optimal. Typically, these workloads are applications that must generate output at a fixed interval, such as graphics applications that generate output at a frame rate of 60 FPS. These workloads are sensitive to latency and should be allowed to run at least once per interval. A shorter time slice reduces latency and improves responsiveness by causing the scheduler to switch more frequently between VMs.
For workloads that require maximum throughput, a longer time slice is optimal. Typically, these workloads are applications that must complete their work as quickly as possible and do not require responsiveness, such as CUDA applications. A longer time slice increases throughput by preventing frequent switching between VMs.
Getting Information about the Scheduling Behavior of Time-Sliced vGPUs#
On all supported hypervisors, the nvidia-smi command provides options for getting detailed information about the scheduling behavior of time-sliced vGPUs. On Linux-based hypervisors, you can also use the hypervisor’s dmesg command to get the current time-sliced vGPU scheduling policy for all GPUs.
Getting Time-Sliced vGPU Scheduler Capabilities#
The scheduler capabilities of a time-sliced vGPU are a set of values that define how you can configure the vGPU to allocate the work for each VM that is configured with NVIDIA vGPU. These capability values depend on the vGPU engine type and, for vGPUs that support multiple scheduling policies, whether the vGPU supports and enforces a strict round-robin scheduling policy.
If the vGPU engine type is graphics, the vGPU scheduler capability values consist of the supported scheduling policies and other values that affect how the work for each VM that is configured with NVIDIA vGPU is allocated. The capability values that are applicable depend on whether the vGPU supports and enforces a strict round-robin scheduling policy.
If the vGPU supports and enforces a strict round-robin scheduling policy, the values for the scheduling frequency and averaging factor are applicable.
Otherwise, the values for the supported time slice range applicable.
If the vGPU engine type is any type other than graphics, the only vGPU scheduler capability value indicates support for the best effort scheduling policy. All other capability values are zero.
To get the scheduler capabilities of all time-sliced vGPUs on the platform, run nvidia-smi vgpu with the –sc or --schedulercaps option.
To limit the information retrieved to a subset of the GPUs on the platform, use the –i or --id option to select one or more GPUs.
[root@vgpu ~]# nvidia-smi vgpu -sc
vGPU scheduler capabilities
Supported Policies : Best Effort
Equal Share
Fixed Share
ARR Mode : Supported
Supported Timeslice Range
Maximum Timeslice : 30000000 ns
Minimum Timeslice : 1000000 ns
Supported Scheduling Frequency
Maximum Frequency : 960
Minimum Frequency : 63
Supported ARR Averaging Factor
Maximum Avg Factor : 60
Minimum Avg Factor : 1
Getting Time-Sliced vGPU Scheduler State Information#
The scheduler state information for a time-sliced vGPU consists of the scheduling policy set for the vGPU and the values of properties that control how the work for the VM that is configured with the vGPU is allocated. The properties available depend on the scheduling policy that is set for the vGPU.
The scheduler state information that can be retrieved for a vGPU depends on whether the VM that is configured with the vGPU is running.
To get scheduler state information for all time-sliced vGPUs on the platform, run nvidia-smi vgpu with the –ss or --schedulerstate option.
To limit the information retrieved to a subset of the GPUs on the platform, use the –i or --id option to select one or more GPUs.
For a MIG-enabled GPU, you must use the –gi or --gpu-instance-id option with the –i or --id option to select one or more GPU instances. If you use the –i or --id option without the –gi or --gpu-instance-id option, the nvidia-smi displays the following error message:
vGPU scheduler state not supported on device.
The following examples show the scheduler state information that is retrieved for a vGPU when the VM that is configured with the vGPU is not running and is running. In these examples, the scheduling policy is equal share scheduler with a strict round-robin scheduling policy and the default time slice length, scheduling frequency, and averaging factor.
vGPU Scheduler State Information for a VM that Is Not Running#
Note
For a VM that is not running, ARR Mode, Average Factor, and Time Slice are not listed.
[root@vgpu ~]# nvidia-smi vgpu -ss
GPU 00000000:65:00.0
Active vGPUs : 0
Scheduler Policy : Equal Share
vGPU Scheduler State Information for a Running VM#
[root@vgpu ~]# nvidia-smi vgpu -ss
GPU 00000000:65:00.0
Active vGPUs : 1
Scheduler Policy : Equal Share
ARR Mode : Enabled
Average Factor : 33
Time Slice(ns) : 2083333
vGPU Scheduler State Information for a GPU Instance on a VM that Is Not Running#
In this example, the scheduling policy is equal share scheduler with a strict round-robin scheduling policy and the default time slice length, scheduling frequency, and averaging factor.
$ nvidia-smi vgpu -ss -gi 1 -i 0
GPU 00000000:C7:00.0
GPU Instance ID : 1
Active vGPUs : 0
Scheduler Policy : Equal Share
vGPU Scheduler State Information for a GPU Instance on a Running VM#
$ nvidia-smi vgpu -ss -gi 1 -i 0
GPU 00000000:C7:00.0
GPU Instance ID : 1
Active vGPUs : 1
Scheduler Policy : Equal Share
ARR Mode : Enabled
Average Factor : 33
Time Slice(ns) : 2083333
Getting Time-Sliced vGPU Scheduler Work Logs#
The scheduler work logs for a time-sliced vGPU provide information about the allocation at runtime of the work for the VM that is configured with the vGPU.
The information in the scheduler work logs that can be retrieved for a vGPU depends on whether the VM that is configured with the vGPU is running.
To get scheduler work logs for all time-sliced vGPUs on the platform, run nvidia-smi vgpu with the –sl or --schedulerlogs option.
To limit the information retrieved to a subset of the GPUs on the platform, use the –i or --id option to select one or more GPUs.
For a MIG-enabled GPU, you must use the –gi or --gpu-instance-id option with the –i or --id option to select one or more GPU instances. If you use the –i or --id option without the –gi or --gpu-instance-id option, the nvidia-smi displays the following error message:
Not supported on the device
To get the scheduler work logs in comma-separated values (CSV) format, run nvidia-smi vGPU --query-gpu-instance-vgpu-scheduler-logs with its options.
vGPU Scheduler Work Logs for a VM that Is Not Running#
[root@vgpu ~]# nvidia-smi vgpu -sl
+---------------------------------------------------------------------------------------------------------+
Engine Id 1
Scheduler Policy Equal Share
GPU at deviceIndex 0 has no active VM runlist.
+---------------------------------------------------------------------------------------------------------+
vGPU Scheduler Work Logs for a Running VM#
[root@vgpu ~]# nvidia-smi vgpu -sl
+---------------------------------------------------------------------------------------------------------+
GPU Id 0
Engine Id 1
Scheduler Policy Equal Share
ARR Mode Enabled
Avg Factor 33
Time Slice 2083333
+---------------------------------------------------------------------------------------------------------+
GPU SW Runlist Time Cumulative Prev Timeslice Target Time Cumulative
Idx Id Stamp Run Time Runtime Slice Preempt Time
0 0 1673362687729708384 2619237216 2083840 2005425 2493060
0 0 1673362687731793472 2621322304 2085088 2005372 2494762
0 0 1673362687733877664 2623406496 2084192 2005346 2495595
vGPU Scheduler Work Logs for a GPU Instance on a VM that Is Not Running#
$ nvidia-smi vgpu -sl -gi 1 -i 0
+---------------------------------------------------------------------------------------------------------+
Engine Id 1
GPU Instance ID 1
Scheduler Policy Equal Share
GPU at deviceIndex 0 has no active VM runlist.
+---------------------------------------------------------------------------------------------------------+
vGPU Scheduler Work Logs for a GPU Instance on a Running VM#
$ nvidia-smi vgpu -sl -gi 1 -i 0
+---------------------------------------------------------------------------------------------------------+
GPU Id 0
GPU Instance ID 1
Engine Id 1
Scheduler Policy Equal Share
ARR Mode Enabled
Avg Factor 33
Time Slice 2083333
+---------------------------------------------------------------------------------------------------------+
GPU SW Runlist Time Cumulative Prev Timeslice Target Time Cumulative
Idx Id Stamp Run Time Runtime Slice Preempt Time
0 0 1751920518677853712 35101189664 2105216 2047776 1195280
0 0 1751920518679935952 35103271904 2082240 2047113 1194187
0 0 1751920518682024144 35105360096 2088192 2047146 1199046
0 0 1751920518684108816 35107444768 2084672 2046999 1200385
vGPU Scheduler Work Logs in CSV Format for a GPU Instance on a VM that Is Not Running#
Note
Line breaks have been added to this example to enhance readability.
$ nvidia-smi vgpu --query-gpu-instance-vgpu-scheduler-logs=gpu_bus_id,gi_id,engine_id,scheduler_policy,
arr_mode,time_slice,sw_runlist_id,time_stamp,cumulative_run_time,previous_timeslice_runtime,target_time_slice,
cumulative_preempt_time -gi 1 -i 0 --format=csv
gpu_bus_id, gi_id, engine_id, scheduler_policy, arr_mode, time_slice, sw_runlist_id, time_stamp, cumulative_run_time,
previous_timeslice_runtime, target_time_slice, cumulative_preempt_time
00000000:C7:00.0, 1, 1, Equal Share, -, -, -, -, -, -, -, -
vGPU Scheduler Work Logs in CSV Format for a GPU Instance on a Running VM#
Note
Line breaks have been added to this example to enhance readability.
$ nvidia-smi vgpu --query-gpu-instance-vgpu-scheduler-logs=gpu_bus_id,gi_id,engine_id,scheduler_policy,
arr_mode,time_slice,sw_runlist_id,time_stamp,cumulative_run_time,previous_timeslice_runtime,target_time_slice,
cumulative_preempt_time -gi 1 -i 0 --format=csv
gpu_bus_id, gi_id, engine_id, scheduler_policy, arr_mode, time_slice, sw_runlist_id, time_stamp, cumulative_run_time,
previous_timeslice_runtime, target_time_slice, cumulative_preempt_time
00000000:C7:00.0, 1, 1, Equal Share, Enabled, 2083333, 0, 1751920964898612528, 481321948480, 2084800, 2047252, 1192158
00000000:C7:00.0, 1, 1, Equal Share, Enabled, 2083333, 0, 1751920964900693296, 481324029248, 2080768, 2047207, 1189593
00000000:C7:00.0, 1, 1, Equal Share, Enabled, 2083333, 0, 1751920964902778160, 481326114112, 2084864, 2047285, 1191124
00000000:C7:00.0, 1, 1, Equal Share, Enabled, 2083333, 0, 1751920964904863024, 481328198976, 2084864, 2047239, 1192655
Linux-Based Hypervisors Only: Getting the Current Time-Sliced vGPU Scheduling Policy for All GPUs#
On Linux-based hypervisors, you can use the dmesg command to get the current time-sliced vGPU scheduling policy for all GPUs. Get this information before changing the scheduling behavior of one or more GPUs to determine if you need to change it or after changing it to confirm the change.
Note
You cannot perform this task on Windows hypervisors because the dmesg command is available only on Linux-based hypervisors.
Perform this task in your hypervisor command shell.
Open a command shell on your hypervisor host machine.
On all supported hypervisors, you can use secure shell (SSH) for this purpose. Individual hypervisors may provide additional means for logging in. For details, refer to the documentation for your hypervisor.
Use the
dmesgcommand to display messages from the kernel that contain the stringsNVRMandscheduler.$ dmesg | grep NVRM | grep scheduler
The scheduling policy is indicated in these messages by the following strings:
BEST_EFFORTEQUAL_SHAREFIXED_SHARE
If the scheduling policy is equal share or fixed share, the scheduler time slice in ms is also displayed.
This example gets the scheduling policy of the GPUs in a system in which the policy of one GPU is set to best effort, one GPU is set to equal share, and one GPU is set to fixed share.
$ dmesg | grep NVRM | grep scheduler 2020-10-05T02:58:08.928Z cpu79:2100753)NVRM: GPU at 0000:3d:00.0 has software scheduler DISABLED with policy BEST_EFFORT. 2020-10-05T02:58:09.818Z cpu79:2100753)NVRM: GPU at 0000:5e:00.0 has software scheduler ENABLED with policy EQUAL_SHARE. NVRM: Software scheduler timeslice set to 1 ms. 2020-10-05T02:58:12.115Z cpu79:2100753)NVRM: GPU at 0000:88:00.0 has software scheduler ENABLED with policy FIXED_SHARE. NVRM: Software scheduler timeslice set to 1 ms.
Tools for Changing Scheduling Behavior for Time-Sliced vGPUs#
To change the scheduling behavior for time-sliced vGPUs, you can use the nvidia-smi command or the RmPVMRL registry key. The tool to use depends on whether you require the changes to be applied immediately or whether you require the changes to be persistent.
If you require the changes to be applied immediately, use the
nvidia-smicommand.If you use the
nvidia-smicommand, you do not need to reload the driver or reboot the hypervisor host to apply your changes. However, your changes are volatile and do not persist in the following circumstances:The driver is reloaded.
The hypervisor host is rebooted.
The
sriov-managescript is run to enable the virtual functions for the physical GPU in thesysfsfile system.
If you require the changes to be persistent, use the
RmPVMRLregistry key.However, if you use the
RmPVMRLregistry key, you must reload the driver or reboot the hypervisor host to apply your changes.
For information about how to use these tools to change the scheduling behavior for time-sliced vGPUs, refer to the following topics:
Changing Scheduling Behavior for Time-Sliced vGPUs by Using the nvidia-smi Command
Changing Scheduling Behavior for Time-Sliced vGPUs by Using the RmPVMRL Registry Key
Note
Disabling strict round robin policy is deprecated and NVIDIA vGPU software 19 is the last release branch to support it. Support for this feature is planned to be removed in the next major release of NVIDIA vGPU software.
Changing Scheduling Behavior for Time-Sliced vGPUs by Using the nvidia-smi Command#
The nvidia-smi command provides the vgpu set-scheduler-state subcommand and associated options for changing the scheduling behavior of time-sliced vGPUs. Because the nvidia-smi command is available on all supported hypervisors, you can use this method for Linux-style hypervisors and for Windows hypervisors.
Ensure that no vGPUs exist on any physical GPU for which you want to change the scheduling behavior for time-sliced vGPUs. Any change that you make affects vGPUs that will be created on the physical GPU after you make the change.
If you try to change the scheduling behavior for time-sliced vGPUs on a physical GPU or GPU instance on which a vGPU already exists, the attempt to change the scheduling behavior fails.
For a physical GPU, the
nvidia-smicommand displays an error message similar to the following example:Unable to set the vGPU scheduler state on GPU "00000000:1A:00.0". vGPU scheduler state cannot be configured, if vGPU instance is currently active on the device.
For a GPU instance, the
nvidia-smicommand displays an error message similar to the following example:Unable to set the vGPU scheduler state on GI Id 1. vGPU scheduler state cannot be configured, if vGPU instance is currently active on the GI.
To change the scheduling behavior for time-sliced vGPUs, run nvidia-smi vgpu set-scheduler-state with its associated options.
For more information about these options, refer to Scheduling Policies for Time-Sliced vGPUs.
–i<gpu-id>,--id<gpu-id><gpu-id>is the identifier of the GPU on which you want to change the scheduling behavior of time-sliced vGPUs in one of the following formats:The GPU’s 0-based index in the natural enumeration returned by the driver
The GPU’s universally unique identifier (UUID)
The GPU’s PCI bus ID in the form
<domain>:<bus>:<device>.<function>in hexadecimal.
This option is not mandatory. If it is omitted, the scheduling behavior of time-sliced vGPUs for all GPUs on the platform is changed.
–gi<gpu-instance-id>,--gpu-instance-id<gpu-instance-id><gpu-instance-id>is the identifier of the GPU instance on which you want to change the scheduling behavior of time-sliced vGPUs.For a MIG-enabled GPU, you must use this option with the
–ior--idoption to select one or more GPU instances. If you use the–ior--idoption without this option, thenvidia-smidisplays the following error message:Unable to set the vGPU scheduler state. Not supported on device "<device-id>".-p<S>,--policy<S><S>is a decimal integer in the range 1-3 that sets the scheduler to use:1: Best effort scheduler (default)2: Equal share scheduler3: Fixed share scheduler
If
<S>is not a decimal integer in the range 1-3, the attempt to set the scheduler to use fails and thenvidia-smicommand displays the following error message:Unable to set the vGPU scheduler state. Not supported-a<R>,--arr-mode<R><R>is a Boolean parameter that enables or disables a strict round-robin scheduling policy for the scheduler:0: Disables a strict round-robin scheduling policy for the scheduler1: Enables a strict round-robin scheduling policy for the scheduler
If a strict round-robin scheduling policy for the scheduler is enabled, the
-asfand-aavgoptions can also be used to set the scheduling frequency and averaging factor.Note
Disabling strict round robin policy is deprecated and NVIDIA vGPU software 19 is the last release branch to support it. Support for this feature is planned to be removed in the next major release of NVIDIA vGPU software.
For equal share and fixed share schedulers, this parameter is optional. If omitted,
--arr-modeis set to1to enable a strict round-robin scheduling policy for the scheduler. For best effort schedulers, this parameter is not applicable.If
<R>is not0or1, the attempt to enable or disable a strict round-robin scheduling policy fails and thenvidia-smicommand displays the following error message:Option passed to set Adaptive Round Robin scheduler is invalid.-asf<frequency>,--arr-sched-frequency<frequency><frequency>is a decimal integer in the range 63 to 960 that sets the scheduling frequency in Hz for the equal share and fixed share schedulers with a strict round-robin scheduling policy.If
<frequency>is outside the range 63 to 960, the scheduling frequency is set as follows:If
<frequency>is not set, the scheduling frequency is set to the default scheduling frequency for the vGPU type as listed in Table 1.If
<frequency>is less than 63, the scheduling frequency is raised to 63.If
<frequency>is greater than 960, the scheduling frequency is capped at 960.
-aavg<averaging-factor>,--arr-avg-factor<averaging-factor><averaging-factor>is a decimal integer in the range 1 to 60 that sets the averaging factor to ensure scheduling fairness for the equal share and fixed share schedulers with a strict round-robin scheduling policy.The number of time slices over which the compensation for the accrued overshoot time is applied depends on the value of
<averaging-factor>:If
<averaging-factor>is 1, the compensation for the accrued overshoot time is applied in a single time slice.If
<averaging-factor>is 60, the compensation for the accrued overshoot time is spread over 60 time slices.If
<averaging-factor>is not set, the default value of 33 is used.If
<averaging-factor>is greater than 60, the number of time slices over which the compensation is applied is capped at 60.
-ts<time-slice-length>,--time-slice<time-slice-length><time-slice-length>is a decimal integer in the range 1,000,000 to 30,000,000 that sets the length of the time slice in nanoseconds (ns) for equal share and fixed share schedulers without a strict round-robin scheduling policy. Set this parameter only if--arr-modeis set to0to disable a strict round-robin scheduling policy for the scheduler.The minimum length is 1,000,000 ns (1 ms) and the maximum length is 30,000,000 ns (30 ms). If
<time-slice-length>is outside the range 1,000,000 to 30,000,000, the length is set as follows:If
<time-slice-length>is not set, the length is set to the default time slice length for the vGPU type as listed in Table 1.If
<time-slice-length>is less than 1,000,000, the length is raised to 1,000,000 ns (1 ms).If
<time-slice-length>is greater than 30,000,000, the length is capped at 30,000,000 ns (30 ms).
Setting the Scheduling Policy for a Single GPU#
This example sets the scheduling policy of the GPU at PCI domain 0000 and BDF 15:00.0 to fixed share scheduler without a strict round-robin scheduling policy and with the default time slice length.
# nvidia-smi vgpu set-scheduler-state –i 0000:15:00.0 -p 3 -a 0
Setting the Scheduling Policy and Time Slice for a Single GPU#
This example sets the scheduling policy of the GPU at PCI domain 0000 and BDF 86:00.0 to fixed share scheduler without a strict round-robin scheduling policy and with a time slice that is 24 ms (24,000,000 ns) long.
# nvidia-smi vgpu set-scheduler-state -i 0000:86:00.0 -p 3 -a 0 -ts 24,000,000
Setting the Scheduling Policy and Time Slice for All GPUs#
This example sets the vGPU scheduler to equal share scheduler without a strict round-robin scheduling policy and with a time slice that is 3 ms (3,000,000 ns) long for all GPUs on the platform.
# nvidia-smi vgpu set-scheduler-state -p 2 -a 0 -ts 3,000,000
Restoring Default Time-Sliced vGPU Scheduler Settings#
This example restores default time-sliced vGPU scheduler settings by setting the vGPU scheduler to best effort scheduler.
# nvidia-smi vgpu set-scheduler-state -p 1
Setting the Scheduling Policy for a Single GPU Instance#
This example sets the vGPU scheduler to equal share scheduler with a strict round-robin scheduling policy and the default time slice length, scheduling frequency, and averaging factor for GPU instance 1 on GPU 0.
$ nvidia-smi vgpu set-scheduler-state -p 2 -gi 1 -i 0
Restoring Default Time-Sliced vGPU Scheduler Settings for a Single GPU Instance#
This example restores default time-sliced vGPU scheduler settings for GPU instance 1 on GPU 0 by setting the vGPU scheduler to best effort scheduler.
$ nvidia-smi vgpu set-scheduler-state -p 1 -gi 1 -i 0
Changing Scheduling Behavior for Time-Sliced vGPUs by Using the RmPVMRL Registry Key#
The RmPVMRL registry key controls the scheduling behavior for NVIDIA vGPUs by setting the scheduling policy, the averaging factor and scheduling frequency for schedulers with a strict round-robin scheduling policy, and the length of the time slice for schedulers without a strict round-robin scheduling policy. How to use the RmPVMRL registry key for this purpose depends on whether you are using a Linux-based hypervisor or a Windows hypervisor.
For detailed instructions, refer to the following topics:
Linux-Based Hypervisors: Changing Scheduling Behavior for Time-Sliced vGPUs by Using the RmPVMRL Registry Key#
To use the RmPVMRL registry key to change the scheduling behavior of time-sliced vGPUs, use the standard interfaces of your hypervisor to set the RmPVMRL registry key value.
Linux-Based Hypervisors: Changing the Time-Sliced vGPU Scheduling Behavior for All GPUs by Using the RmPVMRL Registry Key#
Note
You can change the vGPU scheduling behavior only on GPUs that support multiple vGPU schedulers, that is, GPUs based on NVIDIA GPU architectures after the Maxwell architecture.
Note
You cannot perform this task on Windows hypervisors because it relies on interfaces that are available only on Linux-based hypervisors.
Perform this task in your hypervisor command shell.
Open a command shell on your hypervisor host machine.
On all supported hypervisors, you can use secure shell (SSH) for this purpose. Individual hypervisors may provide additional means for logging in. For details, refer to the documentation for your hypervisor.
Set the
RmPVMRLregistry key to the value that sets the GPU scheduling policy and the length of the time slice that you want.On XenServer or Red Hat Enterprise Linux KVM, add the following entry to the
/etc/modprobe.d/nvidia.conffile.options nvidia NVreg_RegistryDwords="RmPVMRL=<value>"If the
/etc/modprobe.d/nvidia.conffile does not already exist, create it.On VMware vSphere, use the
esxcli setcommand.# esxcli system module parameters set -m <module> -p "NVreg_RegistryDwords=RmPVMRL=<value>"
For the NVIDIA Ada Lovelace GPU architecture and later GPU architectures,
<module>isnvidia-gpu.For GPU architectures before NVIDIA Ada Lovelace,
<module>isnvidia.
<value>The value that sets the GPU scheduling policy and the length of the time slice that you want, for example:
0x01Sets the vGPU scheduling policy to equal share scheduler with the default time slice length.
0x00030001Sets the GPU scheduling policy to equal share scheduler with a time slice that is 3 ms long.
0x11Sets the vGPU scheduling policy to fixed share scheduler with the default time slice length.
0x00180011Sets the GPU scheduling policy to fixed share scheduler with a time slice that is 24 (0x18) ms long.
For all supported values, see RmPVMRL Registry Key.
Reboot your hypervisor host machine.
Confirm that the scheduling behavior was changed as required as explained in Linux-Based Hypervisors Only: Getting the Current Time-Sliced vGPU Scheduling Policy for All GPUs.
Linux-Based Hypervisors: Changing the Time-Sliced vGPU Scheduling Behavior for Select GPUs by Using the RmPVMRL Registry Key#
Note
You can change the vGPU scheduling behavior only on GPUs that support multiple vGPU schedulers, that is, GPUs based on NVIDIA GPU architectures after the Maxwell architecture.
Perform this task in your hypervisor command shell.
Open a command shell on your hypervisor host machine.
On all supported hypervisors, you can use secure shell (SSH) for this purpose. Individual hypervisors may provide additional means for logging in. For details, refer to the documentation for your hypervisor.
Use the
lspcicommand to obtain the PCI domain and bus/device/function (BDF) of each GPU for which you want to change the scheduling behavior.On XenServer or Red Hat Enterprise Linux KVM, add the
-Doption to display the PCI domain and the-d 10de:option to display information only for NVIDIA GPUs.# lspci -D -d 10de:
On VMware vSphere, pipe the output of
lspcito thegrepcommand to display information only for NVIDIA GPUs.# lspci | grep NVIDIA
The NVIDIA GPU listed in this example has the PCI domain
0000and BDF86:00.0.0000:86:00.0 3D controller: NVIDIA Corporation GP104GL [Tesla P4] (rev a1)Use the module parameter
NVreg_RegistryDwordsPerDeviceto set thepciandRmPVMRLregistry keys for each GPU.On XenServer or Red Hat Enterprise Linux KVM, add the following entry to the
/etc/modprobe.d/nvidia.conffile.options nvidia NVreg_RegistryDwordsPerDevice="pci=<pci-domain>:<pci-bdf>;RmPVMRL=<value> [;pci=<pci-domain>:<pci-bdf>;RmPVMRL=<value>...]"
If the
/etc/modprobe.d/nvidia.conffile does not already exist, create it.On VMware vSphere, use the
esxcli setcommand.# esxcli system module parameters set -m <module>\ -p "NVreg_RegistryDwordsPerDevice=pci=<pci-domain>:<pci-bdf>;RmPVMRL=<value>\ [;pci=<pci-domain>:<pci-bdf>;RmPVMRL=<value>...]"
For the NVIDIA Ada Lovelace GPU architecture and later GPU architectures,
<module>isnvidia-gpu.For GPU architectures before NVIDIA Ada Lovelace,
<module>isnvidia.
For each GPU, provide the following information:
<pci-domain>The PCI domain of the GPU.
<pci-bdf>The PCI device BDF of the GPU.
<value>The value that sets the GPU scheduling policy and the length of the time slice that you want, for example:
0x01Sets the GPU scheduling policy to equal share scheduler with the default time slice length.
0x00030001Sets the GPU scheduling policy to equal share scheduler with a time slice that is 3 ms long.
0x11Sets the GPU scheduling policy to fixed share scheduler with the default time slice length.
0x00180011Sets the GPU scheduling policy to fixed share scheduler with a time slice that is 24 (0x18) ms long.
For all supported values, see RmPVMRL Registry Key.
This example adds an entry to the
/etc/modprobe.d/nvidia.conffile to change the scheduling behavior of a single GPU. The entry sets the GPU scheduling policy of the GPU at PCI domain0000and BDF86:00.0to fixed share scheduler with the default time slice length.options nvidia NVreg_RegistryDwordsPerDevice= "pci=0000:86:00.0;RmPVMRL=0x11"
This example adds an entry to the
/etc/modprobe.d/nvidia.conffile to change the scheduling behavior of a single GPU. The entry sets the scheduling policy of the GPU at PCI domain0000and BDF86:00.0to fixed share scheduler with a time slice that is 24 (0x18) ms long.options nvidia NVreg_RegistryDwordsPerDevice= "pci=0000:86:00.0;RmPVMRL=0x00180011"
This example changes the scheduling behavior of a single GPU on a hypervisor host that is running VMware vSphere. The command sets the scheduling policy of the GPU at PCI domain
0000and BDF15:00.0to fixed share scheduler with the default time slice length.# esxcli system module parameters set -m nvidia -p \ "NVreg_RegistryDwordsPerDevice=pci=0000:15:00.0;RmPVMRL=0x11[;pci=0000:15:00.0;RmPVMRL=0x11]"
This example changes the scheduling behavior of a single GPU on a hypervisor host that is running VMware vSphere. The command sets the scheduling policy of the GPU at PCI domain
0000and BDF15:00.0to fixed share scheduler with a time slice that is 24 (0x18) ms long.# esxcli system module parameters set -m nvidia -p \ "NVreg_RegistryDwordsPerDevice=pci=0000:15:00.0;RmPVMRL=0x11[;pci=0000:15:00.0;RmPVMRL=0x00180011]"
Reboot your hypervisor host machine.
Confirm that the scheduling behavior was changed as required as explained in Linux-Based Hypervisors Only: Getting the Current Time-Sliced vGPU Scheduling Policy for All GPUs.
Linux-Based Hypervisors: Restoring Default Time-Sliced vGPU Scheduler Settings by Using the RmPVMRL Registry Key#
Perform this task in your hypervisor command shell.
Open a command shell on your hypervisor host machine.
On all supported hypervisors, you can use secure shell (SSH) for this purpose. Individual hypervisors may provide additional means for logging in. For details, refer to the documentation for your hypervisor.
Unset the
RmPVMRLregistry key.On XenServeror Red Hat Enterprise Linux KVM, comment out the entries in the
/etc/modprobe.d/nvidia.conffile that setRmPVMRLby prefixing each entry with the#character.On VMware vSphere, set the module parameter to an empty string.
# esxcli system module parameters set -m <module> -p "<module-parameter>="
<module>For the NVIDIA Ada Lovelace GPU architecture and later GPU architectures,
<module>isnvidia-gpu.For GPU architectures before NVIDIA Ada Lovelace,
<module>isnvidia.
<module-parameter>The module parameter to set, which depends on whether the scheduling behavior was changed for all GPUs or select GPUs:
For all GPUs, set the
NVreg_RegistryDwordsmodule parameter.For select GPUs, set the
NVreg_RegistryDwordsPerDevicemodule parameter.
For example, to restore default vGPU scheduler settings after they were changed for all GPUs, enter this command:
# esxcli system module parameters set -m nvidia -p "NVreg_RegistryDwords="
Reboot your hypervisor host machine.
Windows Hypervisors: Changing Scheduling Behavior for Time-Sliced vGPUs by Using the RmPVMRL Registry Key#
To use the RmPVMRL registry key to change the scheduling behavior of time-sliced vGPUs, use the Windows Registry Editor to set the RmPVMRL registry key value.
Windows Hypervisors: Getting the Driver Key of a GPU#
The RmPVMRL registry key of a GPU on a host running a Windows hypervisor is located in the GPU’s driver key. When setting the RmPVMRL registry key value for a GPU, you need the GPU’s driver key to identify the Windows registry key in which information about the GPU is stored.
Perform this task in a Windows PowerShell window on the hypervisor host.
Get the
InstanceIDproperty of all available NVIDIA GPUs in your system.PS C:\> Get-PnpDevice -PresentOnly | >> Where-Object {$_.InstanceId -like "PCI\VEN_10DE*" } | >> Select-Object -Property FriendlyName,InstanceId | >> Format-List ... FriendlyName : NVIDIA A40 InstanceId : PCI\VEN_10DE&DEV_2235&SUBSYS_145A10DE&REV_A1\6&2D4D2F51&0&00800018 ...
Note
If multiple NVIDIA GPUs are available on the system, analyze the output from this command to get the
InstanceIDproperty of the GPU on which you want to set the allowed vGPU series.Get the
DEVPKEY_Device_Driverproperty of the GPU from theInstanceIDproperty that you got in the previous step.PS C:\> Get-PnpDeviceProperty -InstanceId "<instance-id>" | >> where {$_.KeyName -eq "DEVPKEY_Device_Driver"} | >> Select-Object -Property Data Data ---- {4d36e968-e325-11ce-bfc1-08002be10318}\0001
<instance-id>The
InstanceIDproperty of the GPU that you got in the previous step, for example,PCI\VEN_10DE&DEV_2235&SUBSYS_145A10DE&REV_A1\6&2D4D2F51&0&00800018.
Windows Hypervisors: Changing the Time-Sliced vGPU Scheduling Behavior for Select GPUs by Using the RmPVMRL Registry Key#
Perform this task on the hypervisor host.
Get the driver key of the GPU on which you want to change the scheduling behavior.
For instructions, refer to Windows Hypervisors: Getting the Driver Key of a GPU.
You will need this information in the next step to identify the Windows registry key in which information about the GPU is stored.
In the Windows Registry Editor, navigate to the Windows registry key
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\<driver-key>.<driver-key>The driver key for the GPU that you got in the previous step, for example:
{4d36e968-e325-11ce-bfc1-08002be10318}\0001
Set the
RmPVMRLDWord (REG_DWORD) registry value to a hexadecimal integer that specifies the GPU scheduling policy and the length of the time slice that you want.For example:
0x01Sets the GPU scheduling policy to equal share scheduler with the default time slice length.
0x00030001Sets the GPU scheduling policy to equal share scheduler with a time slice that is 3 ms long.
0x11Sets the GPU scheduling policy to fixed share scheduler with the default time slice length.
0x00180011Sets the GPU scheduling policy to fixed share scheduler with a time slice that is 24 (0x18) ms long.
For all supported values, refer to RmPVMRL Registry Key.
Reboot your hypervisor host machine.
Confirm that the scheduling behavior was changed as required as explained in Getting Time-Sliced vGPU Scheduler State Information.
Windows Hypervisors: Restoring Default Time-Sliced vGPU Scheduler Settings by Using the RmPVMRL Registry Key#
Perform this task on the hypervisor host.
Get the driver key of the GPU on which you want to change the scheduling behavior.
For instructions, refer to Windows Hypervisors: Getting the Driver Key of a GPU.
You will need this information in the next step to identify the Windows registry key in which information about the GPU is stored.
In the Windows Registry Editor, navigate to the Windows registry key
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\<driver-key>.<driver-key>The driver key for the GPU that you got in the previous step, for example:
{4d36e968-e325-11ce-bfc1-08002be10318}\0001
Remove the
RmPVMRLDWord (REG_DWORD) registry value.Reboot your hypervisor host machine.
RmPVMRL Registry Key#
The RmPVMRL registry key controls the scheduling behavior for NVIDIA vGPUs by setting the scheduling policy, the averaging factor and scheduling frequency for schedulers with a strict round-robin scheduling policy, and the length of the time slice for schedulers without a strict round-robin scheduling policy.
Note
You can change the vGPU scheduling behavior only on GPUs that support multiple vGPU schedulers, that is, GPUs based on NVIDIA GPU architectures after the Maxwell architecture.
Note
Disabling strict round robin policy is deprecated and NVIDIA vGPU software 19 is the last release branch to support it. Support for this feature is planned to be removed in the next major release of NVIDIA vGPU software.
Type#
Dword
Contents#
Value |
Meaning |
|---|---|
|
Best effort scheduler |
|
Equal share scheduler with a strict round-robin scheduling policy and the default time slice length, scheduling frequency, and averaging factor |
|
Equal share scheduler without a strict round-robin scheduling policy and the default time slice length |
|
Equal share scheduler with a strict round-robin scheduling policy and a user-defined averaging factor |
|
Equal share scheduler without a strict round-robin scheduling policy and with a user-defined time slice length |
|
Fixed share scheduler with a strict round-robin scheduling policy and the default time slice length, scheduling frequency, and averaging factor Note This value cannot be set for time-sliced vGPUs on a physical GPU in mixed-size mode. |
|
Fixed share scheduler without a strict round-robin scheduling policy and with the default time slice length Note This value cannot be set for time-sliced vGPUs on a physical GPU in mixed-size mode. |
|
Fixed share scheduler with a strict round-robin scheduling policy and a user-defined averaging factor Note This value cannot be set for time-sliced vGPUs on a physical GPU in mixed-size mode. |
|
Fixed share scheduler without a strict round-robin scheduling policy and with a user-defined time slice length Note This value cannot be set for time-sliced vGPUs on a physical GPU in mixed-size mode. |
The default time slice length and scheduling frequency depend on the maximum number of vGPUs per physical GPU allowed for the vGPU type.
Maximum Number of vGPUs |
Default Time Slice Length |
Default Scheduling Frequency |
|---|---|---|
Less than or equal to 8 |
2 ms |
480 Hz |
Greater than 8 |
1 ms |
960 Hz |
<AA>Two hexadecimal digits in the range 0x01 to 0x3C (decimal 1-60) that set the averaging factor for the equal share and fixed share schedulers with a strict round-robin scheduling policy.
The number of time slices over which the compensation for the accrued overshoot time is applied depends on the value of
<AA>:If
<AA>is 0x01, the compensation for the accrued overshoot time is applied in a single time slice.If
<AA>is 0x3C, the compensation for the accrued overshoot time is spread over 60 (0x3C) time slices.If
<AA>is 0x00, the default value of 33 is used.If
<AA>is greater than 0x3C, the value is capped at 0x3C.
<FFF>Three hexadecimal digits in the range 0x3F to 0x3C0 (decimal 63-960) that set the scheduling frequency for the equal share and fixed share schedulers with a strict round-robin scheduling policy. The time slice is the inverse of scheduling frequency. For example, a frequency of 0x3F (63 Hz) yields a time slice of 1/63 s, or 15.873 ms.
A value of 0x100 for
<FFF>sets the scheduling frequency to 256.If
<FFF>is outside the range 0x3F to 0x3C0, the scheduling frequency is set as follows:If
<FFF>is 000, the scheduling frequency is set to the default scheduling frequency for the vGPU type as listed in Table 1.If
<FFF>is greater than 000 but less than 0x3F, the scheduling frequency is raised to 0x3F (decimal 63).If
<FFF>is greater than 0x3C0, the scheduling frequency is capped at 0x3C0 (decimal 960).
<TT>Two hexadecimal digits in the range 0x01 to 0x1E (decimal 1-30) that set the length of the time slice in milliseconds (ms) for the equal share and fixed share schedulers. The minimum length is 1 ms and the maximum length is 30 ms.
If
<TT>is outside the range 01 to 1E, the length is set as follows:If
<TT>is 00, the length is set to the default time slice length for the vGPU type as listed in Table 1.If
<TT>is greater than 0x1E (decimal 30), the length is capped at 30 ms.
Examples#
This example sets the vGPU scheduler to equal share scheduler with a strict round-robin scheduling policy and the default time slice length, scheduling frequency, and averaging factor.
RmPVMRL=0x01
This example sets the vGPU scheduler to equal share scheduler without a strict round-robin scheduling policy and with a time slice that is 3 ms long.
RmPVMRL=0x00030003
This example sets the vGPU scheduler to fixed share scheduler with a strict round-robin scheduling policy and the default time slice length, scheduling frequency, and averaging factor.
RmPVMRL=0x11
This example sets the vGPU scheduler to fixed share scheduler without a strict round-robin scheduling policy and with a time slice that is 24 (0x18) ms long.
RmPVMRL=0x00180011
This example sets the vGPU scheduler to equal share scheduler with a strict round-robin scheduling policy, an averaging factor of 60 (0x3C), and a scheduling frequency of 960 (0x3C0) Hz.
RmPVMRL=0x3c3c0001
This example sets the vGPU scheduler to fixed share scheduler with a strict round-robin scheduling policy, an averaging factor of 60 (0x3C), and a scheduling frequency of 960 (0x3C0) Hz.
RmPVMRL=0x3c3c0011