The NvMedia Raw sample application, nvmipp_raw, demonstrates how to create NvMedia IPP components to build one or more IPP pipelines to process raw image input from one or more cameras. It provides an example of how to build multiple concurrent IPP pipelines and render aggregate stream output. The output of IPP is sent through EGL Stream to an OpenGL consumer that applies local tone mapping. It operates by parsing command line arguments and a configuration file.

The application requires you to provide a command line input with:

The configuration file contains multiple capture parameter sets, one of which is used in the command line. The application parses configuration settings and other command line parameters and builds the IPP pipeline with the required components. It starts the IPP pipeline to capture frames and send them to the EGL stream output for display.

The following diagram provides an overview of the process.

The NvMedia RAW IPP sample application features are:

The NvMedia IPP Raw sample application provides an example on how to build IPP pipelines and process raw data from cameras. The diagram below describes a typical IPP pipelines built for processing two cameras. One IPP pipeline is used for each camera. Each IPP Raw pipeline consists of:

An EGL producer and a GL consumer are attached to the IPP pipeline output to provide streams for display.

The following diagram shows the system after two IPP pipelines are constructed.

When only one ICP is created and it is attached to all pipelines, each IPP component except the Output component is an individual thread. Besides IPP threads, three more threads are alive:

The main thread:

After IPP pipelines are started, the main thread processes user input in interactive mode. When the user quits, the main thread stops the pipelines and releases resources.

The thread of the EGL producer is defined as ipp_GetIPPOutputThreadFunc. It runs in a loop that performs the following functions:

Like EGL producer, the GL consumer runs in a loop. It receives images in semi-planar YUV 4:2:0 format through EGL Stream. The metadata of the local tone mapping is received for each image and is used to generate a 3D table. The GL fragment shader applies local tone mapping by looking up the table through a 3D texture. After rendering is finished, the image is displayed. The consumer releases one image for each EGL Stream at the end of one iteration.

The Auto Control Plugin library (nvmedia_acp.so provides a set of algorithms to estimate exposure settings and white balance gains for each frame to feed back to the sensor control. During pipeline creation, the IPP raw sample application normally binds this library. To disable the NvMedia Auto Control Plugin, specify --noplugin as an argument in the command line.

When slave capture is enabled, processor B captures the images from the sensors that are already programmed by the master (processor A).

To enable this feature, use the --slave option when running the sample application on processor B.

For information on configuration parameters, see “Configuration File Parameters” in Image Camera Capture (nvmimg_cap).

This topic provides instructions for running the sample application nvmipp_raw.

The following sections contain tables that explain how to call nvmipp_raw, depending on the following factors:

To understand the limitations described in these tables, you must know the camera interface module (CIM) SKU version for your platform. For guidance on determining that version, see Parameter Map in Image Camera Capture (mvmimg_cap).

The following subsections present parameter maps for the E3550 board’s processors A and B with DPHY 1.1 and CPHY 1.1, two and four lanes.

The following table shows the commands for mapping to the Camera CSI physical group on the NVIDIA DRIVE™ AGX Platform processor A for DPHY1.1, two lanes.

This table shows the commands for mapping to the Camera CSI physical group on the DRIVE AGX Platform processor A for DPHY1.1, four lanes.

The following table shows the commands for mapping to the Camera CSI physical group on the DRIVE AGX Platform processor B for DPHY1.1, two lanes.

 

The following table shows the commands for mapping to the Camera CSI physical group on the DRIVE AGX Platform processor B for DPHY1.1, four lanes.

 

The following table shows the commands for mapping to the Camera CSI physical group on the DRIVE AGX Platform processor A for CPHY1.1, four lanes.

The following table shows the commands for mapping to the Camera CSI physical group on the DRIVE AGX Platform processor B for CPHY1.1, two lanes.

 

The following table shows the commands for mapping to the Camera CSI physical group on the DRIVE AGX Platform processor B for CPHY1.1, four lanes.

 

The following subsections present parameter maps for the P3479 board with DPHY 1.1 and CPHY 1.1, four lanes.

This table shows the commands for mapping to the Camera CSI physical group on the P3479 for DPHY1.1, four lanes.

The following table shows the commands for mapping to the Camera CSI physical group on the DRIVE AGX Platform P3479 for CPHY1.1, four lanes.

The command line syntax to run the script is as follows.

The following table describes the required command line options.

The following table describes optional command line options.

 

The following examples specify the ISP output1 or output 2 formats.