WebRTC is a free open source project that provides real-time communication capabilities to browsers and mobile apps.
A major feature of WebRTC is the ability to send and receive interactive HD videos. Fast processing of such videos requires hardware accelerated video encoding.
Currently the open source WebRTC project framework supports various software encoder types: VP8, VP9, and H264. NVIDIA integrates hardware accelerated H.264 encoding into the WebRTC encoding framework. This document uses the name NvEncoder to denote this feature.
Typical Buffer Flow
The following diagram shows a typical buffer flow in the hardware accelerated WebRTC encoding framework.
A source delivers YUV frames, which the framework converts to I420 format and queues on the output plane of the encoder. The output plane buffers are sent to NvEncoder using proprietary NVIDIA low-level encoding libraries. NvEncoder returns filled encoded buffers on its capture plane.
The encoded buffers are sent for further processing as required by the application.
Application and Unit Test Setup
A README file in the WebRTC_r32.2.0_aarch64.tbz2 package contains additional information about for application usage and setup.
To set up and test the NvEncoder sample application
1. Video Loopback application: To run the application, enter this command:
• --codec specifies the encoding format. Hardware encoding currently supports only H.264.
• --width specifies the frame width.
• --height specifies the frame height.
• --capture_device_index specifies the index of /dev/video<x>.
If the WebRTC framework works correctly, the application displays the camera stream with the desired width and height. It scales if WebRTC detects frame drops.
2. Peer connection Client/Server application:
1. Camera setup: Connect a USB camera to the Jetson device.
Server setup: To start the peerconnection_client application on the Jetson platform, enter the command:
$ ./peerconnection_server
This command starts the server on port 8888 with the default configuration.
Client setup: To start two instances of the peerconnection_client application on the Jetson platform, enter the commands:
Each command starts an instance that connects a client to the server automatically. The second command uses the ‑‑autocall option to call the first available other client on the server without user intervention.
3. H.264 unit test for NvEncoder: Enter the command:
Currently hardware acceleration supports only the H.264 encoder.
The video_loopback works only with the H.264 encoder.
USB Camera outputs buffers in YUY2 color format, but NVEncoder expects input in I420 or NV12 format, requiring a color conversion. This is currently done using the WebRTC framework on the CPU.