The NvMedia Raw sample application nvmipp_raw demonstrates how to create NvMedia IPP components to build one or more IPP pipelines to process raw image input from one or more cameras. It provides an example on how to build multiple concurrent IPP pipelines and render aggregate stream output. The output of IPP is sent through EGL Stream to an OpenGL consumer that applies local tone mapping. It operates by parsing command line arguments and a configuration file.

The application requires you to provide a command line input with:

The configuration file contains multiple capture parameter sets, one of which is used in the command line. The application parses configuration settings and other command line parameters, and builds the IPP pipeline with the required components. It starts the IPP pipeline to keep capturing frames and sends them to the EGL stream output for display.

The following diagram provides an overview of the process.

The NvMedia RAW IPP sample application support requirements are as follows:

The NvMedia IPP Raw sample application provides an example on how to build IPP pipelines and process raw data from cameras. The diagram below describes typical IPP pipelines built for processing two cameras. One IPP pipeline is needed for each camera. Each IPP Raw pipeline consists of:

EGL producer and GL consumer are attached to the IPP pipeline output to provide streams for display.

The following diagram shows the system when two IPP pipelines are constructed.

When only one ICP is created and it is attached to all pipelines, each IPP component, except the Output component, is an individual thread. Besides IPP threads, three more threads are alive:

The main thread:

After IPP pipelines are started, the main thread processes user input in interactive mode. When the user quits, the main thread stops the pipelines, and releases resources.

The thread of the EGL Producer is defined as ipp_GetIPPOutputThreadFunc, which runs in a loop that performs the following functions:

Like EGL Producer, GL Consumer also runs in a loop. It receives images in semi-planar YUV 4:2:0 format through EGL Stream. The metadata of the local tone mapping is received for each image and used to generate a three-dimensional table. The GL fragment shader applies local tone mapping by looking up the table through a 3D texture. After rendering is finished, the image is displayed. The consumer releases one image for each EGL Stream at the end of one iteration.

The Auto Control Plugin library provides a set of algorithms to estimate exposure settings and white balance gains for each frame to feed back to the sensor control. The NvMedia IPP pipeline must bind this library during the pipeline creation. An example simple plugin is provided as part of nvmipp_raw application, which can be used with --plugin as an argument in the command line

NvMedia Auto Control Plugin (libnvmedia_acp.so) provides 2 types of behavior:

Each configuration file is required to start with a list of configuration parameters. These parameters are used by the application to determine the capture settings. The required parameters are:

You can also provide optional configuration parameters:

The following is an example configuration for the p2360 board:

This example is for OV10640 cameras. If you have OV10635 cameras, use nvmipp_yuv in place of nvmipp_raw.

 

The following table show the parameters map to the Camera CSI physical group on the DRIVE PX 2 (P3407) for the supported camera types.

The following tabsle show the parameters map to the Camera CSI physical group on the DRIVE PX 2 for Tegra A for the supported camera types.

The following table show the parameters map to the Camera CSI physical group on the DRIVE PX 2 for Tegra B for the supported camera types.

The command line options to run the script is as follows.

The following table describes the required command line arguments.

The following table describes optional command line arguments.

Users of this application have encountered the following issues.

Several situations can cause generic video link errors.

Test Command:

Error Messages:

This is image quality issue could be because of not using an auto-control plugin.

Example Test Command:

—Or—

Warning Message:

This issue may be caused because no auto-control plugin is used with the application. Because there is no currently-defined auto control algorithm, fixed exposure and gains are applied for the sensor and thus video may look dark. To fix this issue, update the test command to include --nvplugin, and run again.