NVIDIA Tegra
DRIVE 5.0 Linux Open Source Software

Development Guide
5.0.10.3 Release


 
NvMedia Sample Applications
 
Building the NvMedia Sample Applications
Running the NvMedia Sample Applications
IPP Raw (nvmipp_raw)
IPP YUV (nvmipp_yuv)
IPP File Processing (nvmipp_file)
Image Camera Capture (nvmimg_cap)
Image Camera Capture and EGL Producer (nvm_imgcap_eglprod)
Image USB Camera Capture (nvmimg_usbcap)
Image Capture Calibrate (nvmimg_cc)
Image Encoder (nvmimg_enc)
Image JPEG Encode (nvmimg_jpgenc)
Image JPEG Decode (nvmimg_jpgdec)
Image Play (nvmimg_play)
Video Encode (nvmvid_enc)
NvMediaImage from RAW Image Sensor to CUDA Consumer on dGPU (nvmipp_bayerdgpu)
IPP Capture (nvmipp_capture)
nvmipp_capture_encode
Troubleshooting Common Error Messages
IPP Sensor Statistics Control (nvmipp_ssc)
EGL Stream (nvm_eglstream)
NvMediaImage to CUDA Consumer on dGPU (nvm_egldgpu)
Video Demo (nvmvid_play)
Image 2D Processing (nvmimg_weave2d)
Creating NvMedia Capture and Display Surfaces
The NvMedia sample applications demonstrate how to use the NvMedia API to perform multimedia tasks.
Multimedia based samples are built for single windowing compatibility with X11.
Graphics (OpenGL ES) samples are built and organized for compatibility with both Wayland and X11 windowing systems with subdirectories for both.
Tips to Using Sample Applications
When developing drivers for camera sensors 0V10640 or AR0231, use the nvmipp_raw application as an IPP example and test application.
Errors occur when using the wrong configuration file for the two Tegras. For example, if you use a Tegra B configuration file in Tegra A, or vice versa, errors display.
Ensure that the Ethernet-based communication used for camera power control is configured correctly. For more information, see Library Usage and Configuration in Tegra-AURIX Communication on DRIVE PX 2.
For the applications that display video, sluggish playback may be observed in the following cases:
With the -s option: YUV dump is enabled.
With the -v 3 option: Logging is enabled.
The following table lists the NvMedia sample applications.
NvMedia Application
Description
IPP Raw
Demonstrates how to create the NvMedia IPP components to build one or more IPP pipelines to process raw image input from one or more cameras. It provides an example on how to build multiple concurrent IPP pipelines and render aggregate stream output. The output of IPP is sent through the EGL Stream to an OpenGL consumer that applies local tone mapping. It operates by parsing command line arguments and a configuration file.
IPP YUV
Demonstrates how to create the NvMedia IPP components to build one or more IPP pipelines to process YUV image input from one or more cameras. It provides an example on how to build multiple concurrent IPP pipelines and render aggregate stream output. The output of IPP is sent through the EGL Stream to an OpenGL consumer that applies local tone mapping. It operates by parsing command line arguments and a configuration file.
IPP File
Demonstrates how to use the NvMedia Image Processing Pipeline (IPP) framework to:
Read RAW images from the filesystem
Process the RAW images through the Image Signal Processor (ISP)
Write the processed images to the filesystem
It also demonstrates how different IPP components can be connected to form an IPP pipeline.
Image Camera Capture
Captures frames from sensors using the NvMedia Image Sensor Control API to display them to a screen and/or save them to a file. It can also be used to test the error detection functionality of the ISC.
Image USB Camera Capture
Captures frames from UVC compliant USB camera nodes and renders to a display using EGLStreams.
Image Capture Calibrate
Used to bring up or validate the camera sensor and capture sensor data to raw files for camera calibration. It also
parses a script file with the necessary I2C commands and sends them to the designated sensor(s).
If you run the application with any calibration command option, you must specify a sensor name.
Image Encode
Demonstrates the usage of the NvMedia image encode API to encode H.264/H.265 bitstreams based on the NVENC hardware engine.
Image JPEG Encode
Demonstrates how to encode image surfaces into JPEG bitstreams using the NvMedia JPEG encode APIs.
Video Encode
Demonstrates using the NvMedia video encode API to encode H.264/H.265 bitstreams based on the NVIDIA Encoder (NVENC)
hardware engine. The application accepts raw YUV/RGB frames as input and produces compressed elementary stream as output. It supports encoding of:
H.264 Baseline
Main profiles up to Level 4.1
High profiles up to Level 4.1
H.265 Main profiles up to level 6.0 with various configure parameters
Test cases are also provided to cover all the feature testing.
Image JPEG Decode
Demonstrates how to decode a set of JPEG bitstreams into image surfaces using the NvMedia JPEG decode APIs.
IPP Capture
Captures raw and processed outputs simultaneously to a file from the NvMedia Image Processing Pipleine (NvMedia IPP).
It also demonstrates how to use NvMedia IPP to capture raw, ISP-processed and LTM-processed output from a camera with the auto-control algorithms turned on or off.
It can be used to re-process the captured raw-stream through the NvMedia image processing pipeline using the File Reader component, instead of the Capture component, as input into the IPP pipeline.
IPP Sensor Statistics Control
Demonstrates how to use the NvMedia IPP components to build IPP pipelines and embedded sensor statistics for image sensor control. The application features multiple IPP pipelines for concurrent frame capture from multiple sensors, as well as image display and recording.
IPP SSC is compatible with the AR0231 RCCB sensor.
EGL Stream
EGL is an interface between Khronos rendering APIs such as OpenGL and the underlying native platform window system. It handles:
Graphics context management
Surface/buffer binding
Rendering synchronization
It also enables high-performance, accelerated, mixed-mode 2D and 3D rendering using other Khronos APIs.
Video Demo
Describes how to use the sample application for stream decoding and playback.
Image 2D Processing
nvmimg_weave2d
Describes the Weave2D sample application, nvmimg_weave2d, demonstrating the Image 2D Processing API.
The following table shows the association between categories of test samples and the hardware those samples exercise.
Test Sample Category
Hardware Used
Capture
Video Input (VI)
Capture and image signal processing
VI, Image Signal Processor (ISP)
Decode
NVIDIA Decoder (NVDEC), Video Image Compositor (VIC)
Encode
NVIDIA Encoder (NVENC), VIC
The applications are categorized as follows:
Capture—video or image capture
Image signal processing—Performs computer vision algorithms
EGLstream—backend processing that routes NvMedia data directly to EGLstream, which in turn converts the data to display format
Decode—Decodes frames from a file
Encode—Encodes frames to a file
Building the NvMedia Sample Applications
All the samples come with source code and a makefile. For example, to build the image capture (nvmimg_cap) sample, use the following procedure.
To build a sample application
1. If you are running the sample on a non-standard windowing system like EGLdevice, Wayland, or X11, enter:
export NV_WINSYS=<egldevice|wayland|x11>
Every platform has a default windowing system. Most samples are set up to compile on 5 different windowing systems, depending on the platform.
2. On the host system enter the following commands.
cd <top>/drive-t186ref-linux/samples/nvmedia/<sample>
make clean
make
Where <sample> is the sample name, for example nvmipp_raw.
3. Re-run the copytarget script passing in the path to the targetfs to ensure that the new executable is copied over to the target.
<top>/drive-t186ref-linux/utils/scripts/copytarget <targetfs_dir>
 
Caution:
If the copytarget script is invoked without specifying a <targetfs_dir>, the script corrupts the host system files.
Running the NvMedia Sample Applications
This topic provides instructions on running NvMedia sample applications.
Prerequisites
You have the shared libraries and executables needed, configuration files or scripts to configure the cameras. For detailed help, please refer to the specific sample application section under appendix.
You have the video files in case of decode/encode. For more information, see Supported Video Media Formats.
To run a sample application
1. On the target system, navigate to the folder with the sample application.
/home/nvidia/drive-t186ref-linux/samples/nvmedia/<sample>
2. Copy the default configuration file and modify your copy for your needs.
3. Run the sample application.