Select a Batch Model

The Video Effects SDK comprises several runtime engines that not only implement an effect but are tuned to take the best advantage of a particular GPU architecture and are optimized to simultaneously process multiple inputs.

These multiple inputs, also known as a batch, and the engine that is optimized to process n images simultaneously is called a batch-n model. An effect on a GPU architecture might have one, two, or more batch models.

Other than for efficiency, the batch size of images that are submitted to Video Effects is unrelated to the maximum batch size of a batch model. The maximum efficiency is achieved for image batch sizes that are integral multiples of the model batch size. For example, a batch-4 model will be most efficient when passed image batches of size 4, 8, 12, and so on, although you can also feed the models in batches of 3, 5, 10, or even 1.

Each effect comes with a model that is optimized for one image, a batch-1 model. Some effects might have other models that can simultaneously and efficiently process large batches of images. If your application can take advantage of the higher efficiency of these larger batches, you can specify the batch model you want before loading it.

By default, a batch-1 model is loaded when NvVFX_Load() is called. To select another model, first specify the model batch size:

NvVFX_SetU32(effect, NVVFX_MODEL_BATCH, modelBatch);

Then call:

NvVFX_Load(effect);

If the model with the batch size you want is available, NVCV_SUCCESS is returned. Otherwise, an appropriate substitution is made, and the NVCV_ERR_MODELSUBSTITUTION status is returned. Although it might not be the result you wanted, the most efficient batch model for your specified batch size is loaded.

Note

The NVCV_ERR_MODELSUBSTITUTION status is a notification and not an error.

The batch size of the loaded model can be subsequently queried by running the following command:

NvVFX_GetU32(effect, NVVFX_MODEL_BATCH, &modelBatch);

To find the available model batch sizes, query the INFO string:

NvVFX_GetString(effect, NVVFX_INFO, &infoStr);

Note

The INFO string is designed to be humanly parsable.

Programmatically, you can repeatedly call the following with increasing sizes until the returned size is smaller than the requested size:

NvVFX_SetU32(effect, NVVFX_MODEL_BATCH, modelBatch);
NvVFX_Load(effect);
NvVFX_GetU32(effect, NVVFX_MODEL_BATCH, &modelBatch);

The values that are returned by NvVFX_GetU32() are identical to the available model batch sizes. This information might be useful at runtime startup to maximize throughput and minimize latency.

NvCV_Load() is a heavyweight operation (on the order of seconds), so we recommend that you avoid repeated calls when the service is online. The preceding code example is just a suggestion for offline querying purposes, during startup or during quarterly tune-ups.