Is this page helpful?

Use Multiple GPUs#

Applications that are developed with the NVIDIA Audio Effects SDK can be used with multiple GPUs. By default, the SDK assumes that the application will set the GPU. Optionally, the SDK can select the best GPU to run the effects.

Select the GPU for Audio Effects Processing in a Multi-GPU Environment#

The GPU to be used to run audio effects in a multi-GPU environment can be controlled by using the cudaSetDevice() and cudaGetDevice() CUDA functions.

The device should be set before NvAFX_Load() is called because NvAFX_Load() will succeed only when the currently selected GPU supports the SDK:

int chosenGPU = 0; // or whichever GPU you want to use
cudaSetDevice(chosenGPU);
NvAFX_Handle effect = nullptr;
NvAFX_Status err = NvAFX_CreateEffect(code, &effect);
err = NvAFX_Set...; // set parameters
...
err = NvAFX_Load(effect);
...
err = NvAFX_Run(effect, ...);

Select GPUs for Chained Audio Effects on Linux#

When using chained effects in a multi-GPU environment on Linux, the SDK can optionally run the effects in the chain on separate GPUs. For example, in a Denoiser 16k + Superres 16k to 48k chain, the denoiser effect can be run entirely on one GPU and the Superres effect on another GPU.

To use this feature, create the chained effect and set NVAFX_PARAM_CHAINED_EFFECT_GPU_LIST to an array that specifies the GPU IDs by using NvAFX_SetU32List. This parameter must be set before you call NvAFX_Load on the effect.

The following sample demonstrates use of this parameter:

NvAFX_Handle effect = nullptr;
NvAFX_Status err = NvAFX_CreateChainedEffect(code, &effect);
...

// Run first effect on GPU id 3, second on GPU id 4
uint32_t gpus[] = {3, 4};
err = NvAFX_SetU32List(effect, NVAFX_PARAM_CHAINED_EFFECT_GPU_LIST, gpus, 2);
...
err = NvAFX_Load(effect);
...

Offload GPU Selection to the SDK for Audio Effects Processing in a Multi-GPU Environment#

In a multi-GPU environment, the SDK can optionally determine the optimal GPU on which to run the audio effects. To use this feature, call NvAFX_SetU32 with the parameters NvAFX_SetU32(effect, NVAFX_PARAM_USE_DEFAULT_GPU, 1) before loading effects. If called after an audio effect is loaded, this function has no effect.

Note

NVAFX_PARAM_USE_DEFAULT_GPU and NVAFX_PARAM_USER_CUDA_CONTEXT (Windows SDK only) cannot be used at the same time.

If the application sets NVAFX_PARAM_USE_DEFAULT_GPU to 0 (or does not set this parameter), the SDK does not explicitly select the GPU to run the effect. The application can set the device on which SDK calls are executed by using cudaSetDevice() to set the device. If this parameter is not set or is set to 0, the SDK uses the default device (device 0).

Note

This parameter is not supported for chained effects.

If the application sets NVAFX_PARAM_USE_DEFAULT_GPU to 1, the application should not call cudaSetDevice(), and all other effects or multiple instances of an effect will use the GPU determined by the SDK. If the application does explicitly call cudaSetDevice() before calling:ref:NvAFX_Load() <nvafx-load>, the SDK might override the application’s device preference. If the client calls cudaSetDevice() to set the GPU to a different GPU just before calling NvAFX_Run(), the NvAFX_Run() call will fail:

NvAFX_Handle effect = nullptr;
NvAFX_Status err = NvAFX_CreateEffect(code, &effect);
err = NvAFX_SetU32(effect, NVAFX_PARAM_USE_DEFAULT_GPU, 1);
...
err = NvAFX_Load(effect);
...

Select GPUs for Various Tasks#

The applications that use the SDK might be designed to perform multiple tasks in a multi-GPU environment in addition to applying the audio effect filter. In this situation, the best GPU for each task should be selected before calling NvAFX_Load() and be set before each NvAFX_Run() call.

Switching to the appropriate GPU is the responsibility of the application. If the application does not switch to the appropriate GPU before calling NvAFX_Run(), the call will fail with an error.

The following steps demonstrate how to complete CUDA tasks and SDK calls on different GPUs.

Call cudaGetDeviceCount() to determine the number of available GPUs:
```
cuErr = cudaGetDeviceCount(&deviceCount);
```
Determine the best GPU for the task.

For example, this can be determined by iterating over the available GPUs and selecting the GPU with the highest number of SMs by using cudaGetDeviceProperties().
In the loop that completes the application’s tasks, call cudaSetDevice(gpuOtherTask) before the other CUDA workload, and then call cudaSetDevice(gpuAFX) immediately before NvAFX_Run(). This sequence is illustrative application logic; the application supplies its own loop, audio buffers, error handling, and task implementation.

CUDA Graph Support on Windows#

The Windows SDK supports using CUDA graphs, which improve performance by reducing the CPU overheads seen with short-lived CUDA kernels.

By default, graphs are enabled in the Windows SDK, but this can cause issues if the SDK runs in parallel with other applications that are using CUDA graphs. The following example shows how to disable CUDA graphs:

// Call before loading model (NvAFX_Load) on Windows
err = NvAFX_SetU32(effect, NVAFX_PARAM_DISABLE_CUDA_GRAPH, 1);
...