The ipp_capture_encode application demonstrates how to use the NvMedia IPP framework to process raw image input from one or more cameras and subsequently encode the IPP output using the NvMedia IEP API. The encoded bit stream is eventually written to a file.

Each instance of the application can exercise up to four cameras on a camera group port. To exercise more than four cameras or cameras on different camera group ports, multiple instances of the application are required.

Each camera input is handled by an IPP pipeline, an encoder, and a file writer component, working in tandem. Additionally, the output of the first pipeline can be redirected to a display. The following diagram shows the different modules of the application.

The application has four major modules:

The streaming engine uses the NvMedia IPP framework to present a consolidated view of the capture, ISP processing, algorithm control functionality and sensor control functionality for multiple camera inputs.

Multiple camera inputs can be aggregated into a CSI port. A pipe is created for each camera input. The pipe consolidates the camera capture, ISP processing, algorithm control and sensor control functionality for the corresponding camera input. An application pipe and the different IPP components that it instantiates is shown in the following diagram.

In context of the application, each streaming engine pipe can generate up to two outputs. These outputs can be channeled to different consumers of different latencies. The reference counting is managed internally by the IPP. If two outputs are enabled for a pipe, then it signifies that the pipe will generate two parallel output streams for the same camera. In context of the application, pipe0 will have up to two output ports feeding into encoder and display. All other pipes have only one output port.

Each of the streaming engine pipes internally use the IPP algorithm and sensor control modules to facilitate real time processing of statistics generated by the ISP.

In context of the application, each pipe of the streaming engine can generate up to two outputs. The output stream handler module ensures that two consumers can consume the frames generated by a streaming engine pipe in a safe manner. There are as many instances of the output stream handler as there are instances of pipes in the streaming engine. Each instance of the output stream handler is bound to the corresponding pipe of the streaming engine by way of pipe number.

In context of the application, the consumers are the display and the encoder->writer. They have different processing latencies and the output stream handler makes no assumptions about the time each consumer takes to process a frame.

Each instance of the output stream handler has as many execution threads as outputs from the corresponding streaming engine pipe. In its execution context, the output stream handler ‘waits’ on availability of frames on the streaming engine pipe and if they are available, sends them out to the consumer queues.

The Image Encoder module uses the NvMedia IEP API. The streaming engine has the capability to generate multiple image streams (through multiple pipes). The application instantiates one image encoder instance each for each of the active pipes. Each instance of the encoder is bound to an instance of the output stream handler, which in turn is bound to a streaming engine pipe. The encoder uses the NvMedia IEP APIs for exercising the underlying encoder hardware. Figure 3 shows the Image Encoder module with its input and output interfaces and the NvMedia IEP component.

Each instance of the image encoder module instantiates two threads. The first thread (encoder feed thread) extracts uncompressed frame from its input queue and feeds it to the NvMedia IEP encoder. The second thread (encoder output thread) extracts encoded frames from the encoder and pushes them to writer feed queue.

The file writer component is the consumer of the encoded streams coming from the image encoder. The application instantiates one instance of the file writer each for each of the active streaming engine pipes. Each instance of the file writer is bound to an instance of the encoder, which in turn is bound to an output stream handler and streaming engine pipe. Each instance of the file writer implements an execution thread. The thread extracts encoded frame buffer from the writer feed queue and writes it onto an output file.

The following shows the command syntax.

The following table describes the required command line arguments.

The application can provide --plugin or --nvplugin, but not both. The --cam_enable and --aggregate options are complementary in nature. Either may be used, but not both.

The following are examples of Tegra A use cases.

4 AR0231 on camera group A with 4 cameras using virtual channel and H.264 encode. The encode preset is LOW_LATENCY_CONSTANT_BITRATE. The display is on port 0, using nvplugin.

4 Continental OV10640 cameras on camera group B, with 4 cameras using virtual channel and H.265 encode. The encode preset is HIGH_QUALITY_CONSTANT_BITRATE. No display is used. Sample plugin is used.

A Maxim OV10640 on camera group C, with 4 cameras using virtual channel and H.265 encode. The encode preset is CONSTANT_QUALITY. No display is used. Nvplugin is used.

The following are examples of Tegra B use cases.

An AR0231 on camera group A, with 4 cameras using virtual channel and H.264 encode. The encode preset is LOW_LATENCY_CONSTANT_BITRATE on display port 0 using nvplugin.

4 Continental OV10640 cameras on camera group B, with 4 cameras using virtual channel and H.265 encode. The encode preset is HIGH_QUALITY_CONSTANT_BITRATE. No display is used. Sample plugin is used.

A Maxim OV10640 on camera group C, with 4 cameras using virtual channel and H.265 encode. The encode preset is CONSTANT_QUALITY. No display is used. Nvplugin is used.

Press q on the console to terminate a launched application.

The following are troubleshooting tips for common error messages.

The error messages are as follows:

They manifest as warnings only if the application is launched with at least logging level 1 (i.e., --v 1). The warnings are as follows:

Each instance of the application works with a maximum of 4 cameras on a given camera group. If multiple instances of the application are launched, exercising more than 4 cameras, then these warnings may appear, indicating that the system is running at its maximum limit. Another possible cause of these messages is when the display rate is slower than the capture rate (e.g., displaying on a 4K monitor with display rate of 30Hz with capture running at 36fps).