VFX SDK#

Creating a Triton Object and Connecting to an ARSDK Enabled Triton Server#

NvVFX_TritonServer is a predefined type in the SDK, which represents a connection to the server. The first step is to create an instance of this type and call the NvVFX_ConnectTritonServer to connect to a server. The NvVFX_ConnectTritonServer function accepts a string indicating the URL (IP and the port) of the gRPC service on the server. In the same application, it is possible to connect to multiple servers by creating multiple instances.

NvCV_Status nvcv_err = NVCV_SUCCESS;

NvVFX_TritonServer server_handle;

char server_url[] = "127.0.0.1:8001";

nvcv_err = NvVFX_ConnectTritonServer(server_url, &server_handle);

Creating a Feature Instance#

NvVFX_Handle is a predefined type in the SDK, which represents a feature. To instantiate a handle for a feature to run on a server, NvVFX_CreateEffectTriton API is called. This is different from the NvVFX_CreateEffect API which creates a feature to be run locally. Only feature instances created using NvVFX_CreateEffectTriton can run features on the server and use the batching APIs. If a feature handle is instantiated using NvVFX_CreateEffect, its usage is limited to exactly the same as the non-Triton enabled SDKs.

The NvVFX_CreateEffectTriton API takes feature ID as input which indicates what feature to instantiate. For AI Green Screen, the feature ID is NVVFX_FX_GREEN_SCREEN. The next step is to assign the feature instance to a server instance using NvVFX_SetTritonServer. Several feature instances may be assigned to a single server instance, however one feature instance may be associated with only one server instance.

NvVFX_Handle feature_handle;

nvcv_err = NvVFX_CreateEffectTriton(NVVFX_FX_GREEN_SCREEN, &feature_handle);

nvcv_err = NvVFX_SetTritonServer(feature_handle, server_handle);

Getting and Setting Feature Properties#

After the feature instance is instantiated and assigned to a server, feature configurations can be set using the setters and verified using the getters. For example, the following code sets the mode. The features support the same configurations as the non-Triton SDKs.

nvcv_err = NvVFX_SetU32(feature_handle, NVVFX_MODE, 0);

Loading the Feature#

NvVFX_Load is called to initialize the feature after the configurations have been set.

nvcv_err = NvVFX_Load(feature_handle);

Creating State Objects#

The SDK uses state objects to identify and track the input video stream in a batched input. One state object is associated with one input video stream. Therefore, if the application intends to process N video streams, N state objects must be created.

The SDK specifies NvVFX_StateObjectHandle type for the state objects which are instantiated using NvVFX_AllocateState as shown below. Note that even if the application is processing one video stream, it is necessary to create one state object to represent that video stream.

int num_input_video_streams = 10;

std::vector<NvVFX_StateObjectHandle>
state_handles(num_input_video_streams);

for (int i=0; i<num_input_video_streams; i++) {

   nvcv_err = NvVFX_AllocateState(feature_handle, &state_handles[i]);

}

It is not necessary to create all the state objects at once, new state objects may be created during runtime as needed.

Setting Input and Output#

The process of setting input and output buffers for the features is the same as the non-Triton enabled SDKs. The only difference is that when the application is trying to send a batch of input (more than one video stream concurrently) to the server, it is necessary to provide batched buffers.

Batched NvCVImage Object Buffers#

A batched NvCVImage points to a contiguous buffer that contains multiple images with the same structure. The Triton enabled SDK library will transfer NvCVImage to and back from the server using gRPC if the NvCVImage buffer is allocated on the CPU and using CUDA shared memory if the NvCVImage buffer is allocated on the GPU.

There are tighter restrictions on the type of images supplied to the Triton Client, relative to the native SDK:

  • The pixel format is restricted to BGR, BGRA, and A; RGB and RGBA are not accommodated.

  • The images must have a zero-gap pitch. This can be achieved by specifying 1 as the alignment, when calling the NvCVImage constructor or NvCVImage_Alloc, NvCVImage_Realloc.

Batch Utilities#

The following utility functions in BatchUtilities.cpp help you to work with image batches:

  • AllocateBatchBuffer() can be used to allocate a buffer for a batch of images.

  • NthImage() can be used to set a view into the nth image in a batched buffer.

  • ComputeImageBytes() can be used to determine the number of bytes for each image to advance the pixel pointer from one image to the next.

  • TransferToNthImage() makes it easy to call NvCVImage_Transfer() to set one of the images in a batch.

  • TransferFromNthImage() makes it easy to call NvCVImage_Transfer() to copy one of the images in a batch to a regular image.

  • TransferToBatchImage() transfers multiple images from different locations to a batched image.

  • TransferFromBatchImage() can be used to retrieve the images in a batch to different images in different locations.

  • TransferBatchImage() transfers all images in a batch to another compatible batch of images.

The last three functions can also be accomplished by repeatedly calling the Nth image APIs, but the source code illustrates an alternative method of accessing images in a batch.

Allocation of Batched Buffers#

To allocate batched buffers, call the AllocateBatchBuffer() function, which will allocate an image that is N times taller than the prototypical image.

Note

The allocation cannot always be interpreted this way, especially if the pixels are planar.

The purpose of this function is mainly to provide storage and then dispose of this storage when the NvCVImage goes out of scope or its destructor is called.

You can use your own method to allocate the storage for the batched images. The image that the AllocateBatchBuffer() yields is only used for bookkeeping and is never used in any of the SDK APIs.

Note

The SDK APIs require an NvCVImage descriptor for only the first image.

Setting the Batched Images#

The API takes the image descriptors for only the first image in a batch. The following sample allocates the src and dst batch buffers and sets the input and outputs via the views of the first image in each batch buffer.

NvCVImage srcBatch, dstBatch, nthSrc, nthDst;
AllocateBatchBuffer(&srcBatch, batchSize, srcWidth, srcHeight,
                    ...);
AllocateBatchBuffer(&dstBatch, batchSize, dstWidth, dstHeight,
                    ...);
NthImage(0, srcHeight, &srcBatch, &nthSrc);
NthImage(0, dstHeight, &dstBatch, &nthDst);
NvVFX_SetImage(effect, NVVFX_INPUT_IMAGE, &nthSrc);
NvVFX_SetImage(effect, NVVFX_OUTPUT_IMAGE, &nthDst);

Because the image descriptors are copied into the SDK, this can be simplified to the following:

NvCVImage srcBatch, dstBatch, nth;
AllocateBatchBuffer(&srcBatch, batchSize, srcWidth, srcHeight,
                    ...);
AllocateBatchBuffer(&dstBatch, batchSize, dstWidth, dstHeight,
                    ...);
NvVFX_SetImage(effect, NVVFX_INPUT_IMAGE, NthImage(0, srcHeight, &srcBatch, &nth));
NvVFX_SetImage(effect, NVVFX_OUTPUT_IMAGE, NthImage(0, dstHeight, &dstBatch, &nth));

The other images in the batch are computed by advancing the pixel pointer by the size of each image.

The other aspect of setting the batched images is determining how to set the pixel values. Each image in the batch is accessible by calling the NthImage() function:

NthImage(n, imageHeight, &batchImage, &nthImage);

You can then use the same techniques that were used for other NvCVImages on the recently initialized nthImage view. As previously suggested, NthImage() is just a thin wrapper around NvCVImage_InitView() and can be used instead. The NvCVImage_Transfer() functions can be used to copy pixels to and from the batch.

Setting a Batch of State Objects#

The SDK uses batch size parameter, NVVFX_BATCH_SIZE, to determine the number of items in the input/output buffers and the batch of states parameter, NVVFX_STATE, to determine which item in the batched input and output buffer belongs to which input video stream. It is necessary to set these parameters before the NvVFX_Run call.

A batch of states array is an array of NvVFX_StateObjectHandle pointers. In the following example, a batch size of 3 is used and a batch of states is created with pointers to the p-th, the q-th and the r-th items in the state_handles array. This signifies that all input/output buffers contain data in the same order. Since each state corresponds to an input video stream, it signifies that the 0-th, the 1-st, and the 2-nd items in the buffers belong to the p-th, the q-th and the r-th input video streams.

A batch can not contain more than one item from a single input video stream, so a batch of states should not have duplicate state objects.

Even when processing only one video, it is necessary to create a batch of states array with one item and set the batch size to 1.

unsigned batch_size = 3;

NvVFX_StateObjectHandle* batch_of_states[] = {&state_handles[p],
&state_handles[q], &state_handles[r]};

nvcv_err = NvVFX_SetU32(feature_handle, NVVFX_BATCH_SIZE, batch_size);

nvcv_err = NvVFX_SetStateObjectHandleArray(feature_handle, NVVFX_STATE,
batch_of_states);

The NVVFX_BATCH_SIZE and the NVVFX_STATE can be changed before every NvVFX_Run call.

Note

The size of the batch need not be equal to the total number of input video streams.

Running the Feature#

NvVFX_Run runs the feature on the server. The call is asynchronous and non-blocking.

nvcv_err = NvVFX_Run(feature_handle);

Waiting for the Result#

NvVFX_SynchronizeTriton will block the execution and wait until the results are available after the NvVFX_Run call. When this function exits, the results will be populated in the output buffers.

nvcv_err = NvVFX_SynchronizeTriton(feature_handle);

Destroying the State Objects#

NvVFX_DeallocateState destroys the state objects.

for (int i=0; i<num_input_video_streams; i++) {

   nvcv_err = NvVFX_DeallocateState(feature_handle, state_handles[i]);

}

Destroying the Feature Instance#

NvVFX_Destroy destroys the feature instance.

nvcv_err = NvVFX_Destroy(feature_handle);

Destroying the Triton Object#

NvVFX_DisconnectTritonServer disconnects from the server and destroys the server object.

nvcv_err = NvVFX_DisconnectTritonServer(server_handle);