Multi-GPU Overview

Applications developed with the Video Effects SDK can be used with multiple GPUs.

By default, the SDK determines which GPU to use based on the capability of the currently selected GPU: If the currently selected GPU supports the Video Effects SDK, the SDK uses it. Otherwise, the SDK chooses the best GPU. You can control which GPU is used in a multi-GPU environment by using the NVIDIA CUDA Toolkit functions cudaSetDevice(int whichGPU) and cudaGetDevice(int whichGPU) and the Video Effects set function NvVFX_SetS32(NULL, NVVFX_GPU, whichGPU). The set function is intended to be called only once for the Video Effects SDK before any effects are created. Images that are allocated on one GPU cannot be transparently passed to another GPU, so you must ensure that the same GPU is used for all video effects.

NvCV_Status err;
int chosenGPU = 0; // or whatever GPU you want to use
err = NvVFX_SetS32(NULL, NVVFX_GPU, chosenGPU);
if (NVCV_SUCCESS != err) {
    printf(“Error choosing GPU %d: %s\\n”, chosenGPU,
          NvCV_GetErrorStringFromCode(err));
}
cudaSetDevice(chosenGPU);
NvCVImage \*dst = new NvCVImage(…);
NvVFX_Handle eff;
err = NvVFX_API NvVFX_CreateEffect(code, &eff);
err = NvVFX_API NvVFX_SetImage(eff, NVVFX_OUTPUT_IMAGE, dst);
…
err = NvVFX_API NvVFX_Load(eff);
err = NvVFX_API NvVFX_Run(eff, true);
// switch GPU for other task; switch back for next frame

Buffers need to be allocated on the selected GPU, so before you allocate images on this GPU, call cudaSetDevice(). Neural networks need to be loaded on the selected GPU, so before NvVFX_Load() is called, set this GPU as the current device.

To use the buffers and models, before you call NvVFX_Run(), the GPU device needs to be current. A previous call to NvVFX_SetS32(NULL, NVVFX_GPU, whichGPU) helps enforce this requirement.

For performance concerns, switching to the appropriate GPU is the responsibility of the application.