Developer Guide Release

NvMedia Sample Applications
Building and Running the NvMedia Samples
EGL Stream (nvm_eglstream)
EGLStream Support for Multiple Consumers/Producers
Vision Programming Interface (nvm_vpi)
IPP Raw (nvmipp_raw)
Image Camera Capture (nvmimg_cap)
Image Capture Calibrate (nvmimg_cc)
Image Play (nvmimg_play)
Image Encoder (nvmimg_enc)
Image JPEG Encode (nvmimg_jpgenc)
Image JPEG Decode (nvmimg_jpgdec)
Image LDC (nvmimg_ldc)
Image 2D (nvmimg_2d)
Video Encode (nvmvid_enc)
Video Demo (nvmvid_play)
RAW Data to CUDA Consumer on dGPU (nvmipp_bayerdgpu)
IPP File Processing (nvmipp_file)
NvMediaImage Surfaces to CUDA Consumer on dGPU (nvm_egldgpu)
Image OpticalFlow/StereoDisparity (nvmedia_imageofst)
Image 2D Processing (nvmimg_weave2d)
SIPL (Sensor Input Processing Library) Camera
SIPL (Sensor Input Processing Library) Reprocess
Creating NvMedia Capture and Display Surfaces
Deep Learning Accelerator Programming Interface (nvm_dlaSample)
The NvMedia sample applications demonstrate how to use the NvMedia API to perform multimedia tasks.
Multimedia based samples are built for single windowing compatibility that depends on the platform operating system:
NVIDIA DRIVE™ 5.0 QNX uses the Screen windowing system.
NVIDIA DRIVE™ 5.0 Linux uses the X11 windowing system.
Graphics OpenGL ES samples are built and organized for compatibility with both Wayland and X11 windowing systems with subdirectories for both. For information on the graphics OpenGL ES samples, see Building and Running Samples in Graphics Programming.
Tips for Using Sample Applications
When developing drivers for camera sensors AR0231, use the nvmipp_raw application as an IPP example and test application.
Ensure that the Ethernet-based communication used for camera power control is configured correctly.
NvMedia Sample Applications and Tools
NvMedia Application
EGL Stream
EGL is an interface between Khronos rendering APIs such as OpenGL and the underlying native platform window system. It handles:
Graphics context management
Surface/buffer binding
Rendering synchronization
It also enables high-performance, accelerated, mixed-mode 2D and 3D rendering using other Khronos APIs.
Vision Programming Interface
Performs computer vision operations using algorithms such as Convolution and BoxFilter.
Demonstrates how to create the NvMedia IPP components to build one or more IPP pipelines to process raw image input from one or more cameras. It provides an example on how to build multiple concurrent IPP pipelines and render aggregate stream output. The output of IPP is sent through the EGL Stream to an OpenGL consumer that applies local tone mapping. It operates by parsing command line arguments and a configuration file.
Image Camera Capture
Captures frames from sensors using the NvMedia Image Sensor Control API to display them to a screen and/or save them to a file. It can also be used to test the error detection functionality of the ISC.
Image Capture Calibrate
Used to bring up or validate the camera sensor and capture sensor data to raw files for camera calibration. It also
parses a script file with the necessary I2C commands and sends them to the designated sensor(s).
If you run the application with any calibration command option, you must specify a sensor name.
Image Play
Performs stream decoding.
Image Encode
Demonstrates the usage of the NvMedia image encode API to encode H.264/H.265 bitstreams based on the NVENC hardware engine.
Image JPEG Encode
Demonstrates how to encode image surfaces into JPEG bitstreams using the NvMedia JPEG encode APIs.
Image JPEG Decode
Demonstrates how to decode a set of JPEG bitstreams into image surfaces using the NvMedia JPEG decode APIs.
Lens Distortion Correction (LDC)
Performs Geometric Transformation (GEOTRANS) and Temporal Noise Reduction (TNR) on input YUV files.
Video Encode
Demonstrates using the NvMedia video encode API to encode H.264/H.265 bitstreams based on the NVIDIA Encoder (NVENC)
hardware engine. The application accepts raw YUV/RGB frames as input and produces compressed elementary stream as output. It supports encoding of:
H.264 Baseline
Main profiles up to Level 4.1
High profiles up to Level 4.1
H.265 Main profiles up to level 6.0 with various configure parameters
Test cases are also provided to cover all the feature testing.
Video Demo
Describes how to use the sample application for stream decoding and playback.
NvMediaImage from RAW Image Sensor to CUDA Consumer on dGPU nvmipp_bayerdgpu
Demonstrates how to post RAW data coming from an image sensor to a CUDA consumer running on dGPU.
IPP File Processing
Demonstrates how to use the NvMedia Image Processing Pipeline (IPP) framework.
NvMediaImage to CUDA Consumer on dGPU
Demonstrates how to use NvMediaEglstream APIs to post NvMediaImage surfaces to dGPU, which can be processed by CUDA APIs running on dGPU.
Image OpticalFlow / StereoDisparity
Demonstrates how to use the NvMedia Optical Flow or Stereo Disparity (OFST) APIs to perform motion estimation/stereo matching. The sample uses the NVIDIA® Tegra® NVENC hardware engine.
NvMediaImage Weave2D API and Sample Application
Merges odd lines from one CSI port with even lines from another for a frame captured over NVIDIA Tegra CSI ports.
Sensor Input Processing Library
Demonstrates how to use the NvMedia SIPL APIs to select and initialize a camera platform configuration.
The association between categories of test samples and the hardware those samples exercise is as follows.
Test Sample Category
NVIDIA® Tegra® Hardware Used
Tegra Video Input (VI)
Capture and image signal processing
Tegra VI, Tegra Image Signal Processor (ISP)
NVIDIA Decoder (NVDEC), Tegra Video Image Compositor (VIC)
Vision Programming
Tegra Programmable Vision Accelerator (PVA)
The applications are categorized as follows:
Capture—video or image capture
Image signal processing—Performs computer vision algorithms
EGLstream—backend processing that routes NvMedia data directly to EGLstream, which in turn converts the data to display format
Decode—Decodes frames from a file
Encode—Encodes frames to a file