Benchmarking#

Follow these steps to measure the best possible time to first token (TTFT) and throughput of NIMs on Windows. These steps are necessary to enable a faster response in the system due to the power savings characteristics of Windows.

Run these steps as administrator from Windows Command Prompt or PowerShell in the host OS. These commands have no effect if run inside the WSL2 instance.

Run the following command:
```
nvidia-smi -q -d SUPPORTED_CLOCKS;
```
Pick the first two reported values for maximum memory clock and maximum core clock.
Run the following commands to lock the memory and core clocks. Replace the values with those reported in the list from step 2.
```
nvidia-smi -lgc <clock> # e.g., 3105
nvidia-smi -lmc <clock> # e.g., 9001
```
Run the benchmark. Refer to the specific NIM documentation for benchmarking information.
When finished, run the following commands to restore the default system values:
```
nvidia-smi -rgc
nvidia-smi -rmc
```

For more information, see the command help for nvidia-smi.