NVIDIA Tegra
NVIDIA DRIVE OS 5.1 Linux SDK

Developer Guide
5.1.6.0 Release


 
EGLStream Support for Multiple Consumers/Producers
 
Multiple Acquire and Consumer Support
Using NvMedia EGLStream to Send an Image to Multiple Consumers
Buffer Flow with Multiple Consumers
Sending and Returning in Single Threads
Sending in a Single and Returning in Multiple Threads
The NVIDIA® NvMedia library supports EGLStream functionality on the producer and consumer sides, including multiple producers and consumers. This topic describes support for different combinations of producers and consumers. It also explains how to use APIs to implement this support.
Multiple Acquire and Consumer Support
Multiple acquire support means a consumer can hold multiple frames before releasing the first one to the producer.
For example, when a producer posts 3 frames (0, 1, 2), the consumer can acquire 3 frames (0, 1, 2) continuously before it releases frame 0 to the producer.
Multiple Acquire Support
The following combinations of producer and consumer are supported for multiple acquire functionality.
Producer Side API
Consumer Side API
OpenGL ES
NvMedia
NvMedia
OpenGL ES
NvMedia
NvMedia
NvMedia
NVIDIA® CUDA®
CUDA
NvMedia
Multiple Consumer Support
The following combinations of producer and consumer are supported for multiple consumer functionality.
Producer Side API
Consumer Side API
NvMedia
OpenGL ES
NvMedia
NvMedia (2)
NvMedia
CUDA
Using NvMedia EGLStream to Send an Image to Multiple Consumers
This section describes how to send an image using NvMedia EGLStream to multiple EGL consumers. Applications must enable multiple sending mode (multiSend) on all producers used to send an image to multiple EGLStreams.
For an example, see the NvMediaEglStreamProducerAttributes in NvMedia API.
To send an image using NvMedia EGLStream to multiple EGL consumers
The following example is for two producers and consumers.
1. Create eglstream1 and consumer1 and connect them to eglstream1.
2. Create producer1 and connect to eglstream1.
3. Call NvMediaEglStreamProducerSetAttributes (producer1, attribmask, attributes) with attributes > multiSend = NVMEDIA_TRUE and attribmask = NVMEDIA_EGL_STREAM_PRODUCER_ATTRIBUTE_MULTISEND.
4. Create eglstream2 and consumer2, and connect them to eglstream2.
5. Create producer2 and it connect to eglstream2.
6. Call NvMediaEglStreamProducerSetAttributes (producer2, attribmask, attributes) with attributes > multiSend = NVMEDIA_TRUE and attribmask = NVMEDIA_EGL_STREAM_PRODUCER_ATTRIBUTE_MULTISEND.
Even though the producer is actually one component, two NvMediaEglStream producers are created. However, the same image is posted to two different eglstreams.
Buffer Flow with Multiple Consumers
Sending images to multiple consumers is similar to sending them to a single consumer. However, the image can be reused, for example, to write the next NvMedia IPP output, when all EGLStream consumers have finished processing the image and returned to the producers.
It is the responsibility of the application to ensure that all the consumers have returned the image back to the respective producers before reusing the image.
Sending and Returning in Single Threads
Applications send and return to multiple EGLSteams in a single send and single return thread. This is accomplished by sending the image to multiple EGLStream producers in a sequence. Similarly, the return thread can retrieve the image from multiple streams in a sequence and reuse it. For example, return it to the NvMedia IPP.
Example Send Thread
while(!exit) {
NvMediaIPPComponentGetOutput(&output);
// Send image to multiple streams
NvMediaEglStreamProducerPostImage(producer1, output.image);
NvMediaEglStreamProducerPostImage(producer2, output.image);
}
Example Return Thread
while(!exit) {
// Receive image from multiple streams
NvMediaEglStreamProducerGetImage(producer1, &returnImage1);
NvMediaEglStreamProducerGetImage(producer2, &returnImage2);
if(returnImage1 != returnImage2) {
// This should not happen
// In this case the two streams got out of sync
}
output.image = returnImage1;
NvMediaIPPComponentReturnOutput(&output);
}
Sending in a Single and Returning in Multiple Threads
If there are multiple return threads for the different EGLStreams the return threads must ensure that the image is returned only once to NvMedia IPP. The consumers return the images asynchronously, so a reference counting logic must be added and when the image is no longer referenced it can be returned to NvMedia IPP.
Example Send Thread
while(!exit) {
NvMediaIPPComponentGetOutput(&output);
// Increment ref count twice for two EGLStream producers
IncrementRefCount(output.image);
IncrementRefCount(output.image);
// Send the image to multiple streams
NvMediaEglStreamProducerPostImage(producer1, output.image);
NvMediaEglStreamProducerPostImage(producer2, output.image);
}
Example Return Thread 1
while(!exit) {
// Get the image back from the first stream
NvMediaEglStreamProducerGetImage(producer1, &returnImage);
// Decrement ref count and check that the image is no loger referenced
if(DecrementRefCount(returnImage) == 1) {
// Image is no longer referenced and can be returned
output.image = returnImage1;
NvMediaIPPComponentReturnOutput(&output);
}
}
Example Return Thread 2
while(!exit) {
// Get the image back from the second stream
NvMediaEglStreamProducerGetImage(producer2, &returnImage);
// Decrement ref count and check that the image is no loger referenced
if(DecrementRefCount(returnImage) == 1) {
// Image is no longer referenced and can be returned
output.image = returnImage;
NvMediaIPPComponentReturnOutput(&output);
}
}
Example Reference Counting Logic
The following is a sample implementation of a reference counting logic:
// The number should be equal or greater than the maximum number of buffers expected
// from the Output component
#define MAX_HANDLES 8
 
typedef struct {
void *handle;
int refCount;
} RefCountStruct;
 
RefCountStruct handleList[MAX_HANDLES];
 
// Mutex for atomic list operations
pthread_mutex_t handleListMutex = PTHREAD_MUTEX_INITIALIZER;
 
// Increment reference count
// Returns:
// 0 - OK
// -1 - Error, No room for handle in list
//
static int IncrementRefCount(void *handle)
{
int i;
 
// Lock list
pthread_mutex_lock(&handleListMutex);
 
// Find handle in the current list
for(i = 0; i < MAX_HANDLES; i++) {
if(handleList[i].handle == handle) {
// If handle found increment reference
handleList[i].refCount++;
// Unlock list
pthread_mutex_unlock(&handleListMutex);
return 0;
}
}
 
// Not in the list do find an empty slot
for(i = 0; i < MAX_HANDLES; i++) {
if(handleList[i].handle == NULL) {
// Save handle and set initial reference count
handleList[i].handle = handle;
handleList[i].refCount = 1;
// Unlock list
pthread_mutex_unlock(&handleListMutex);
return 0;
}
}
 
// Unlock list
pthread_mutex_unlock(&handleListMutex);
 
// No empty slot in the list
return -1;
}
 
// Decrement reference count
// Returns:
// 0 - OK, handle still being used
// 1 - OK, handle is no longer used
// -1 - Error, Handle is not found in l;ist
//
static int DecrementRefCount(void *handle)
{
int i;
 
// Lock list
pthread_mutex_lock(&handleListMutex);
 
// Find handle in the current list
for(i = 0; i < MAX_HANDLES; i++) {
if(handleList[i].handle == handle) {
// Decrement reference count
if(handleList[i].refCount > 0) {
handleList[i].refCount--;
}
// Check for no reference
if(handleList[i].refCount == 0) {
// Free handle
handleList[i].handle = NULL;
// Unlock list
pthread_mutex_unlock(&handleListMutex);
// Indicate no reference
return 1;
} else {
// Unlock list
pthread_mutex_unlock(&handleListMutex);
// Still being used
return 0;
}
}
}
 
// Unlock list
pthread_mutex_unlock(&handleListMutex);
 
// Handle is not in list
return -1;
}