Release Notes v1.1

Release Highlights

• Developer Preview of Python Bindings. It allows using VPI directly in Python scripts.
  • All algorithms are exposed, except for KLT Template Tracker.
  • Sample applications were ported to Python, except where corresponding Python functionality isn't present.
  • Algorithm usage snippets in documentation were ported to Python.
• New algorithms:
  • Dense Optical Flow on NVENC backend (Tegra only)
  • Laplacian Pyramid Generator on CUDA and CPU backends
  • Image Histogram on CUDA and CPU backends
  • Equalize Histogram on CUDA and CPU backends
  • Background Subtractor on CUDA and CPU backends
• Improved Stereo Disparity Estimator on CUDA
  • ~2.2x speed up, as measured on Jetson AGX Xavier.
  • Better quality overall, less noisy output.
  • Masks out invalid disparities, mostly due to occlusions.
  • Optionally outputs a disparity confidence map.
  • Maximum disparity increased to 256.
• New PVA+NVENC+VIC backend for Stereo Disparity Estimator
  • Allows masking out of invalid disparity
  • Optionally outputs a disparity confidence map.
  • Up to 256 disparity levels.
  • Allows 1920x1080 inputs.
• Added Stereo Disparity GUI demo application.
  • Allows input from rectified stereo-pair video.
  • Supports ZED stereo cameras.

New Features

• Added NVENC backend, exposing some functionality provided by NVENC processor in Tegra devices.
• New sample applications:
  • Dense Optical Flow
  • Background Subtractor
  • Benchmarking, which replaces the old "Timings with Events" sample.
• New image formats (enabled for selected algorithms/platforms)
  • VPI_IMAGE_FORMAT_Y8
  • VPI_IMAGE_FORMAT_Y8_ER
  • VPI_IMAGE_FORMAT_Y16_ER
  • VPI_IMAGE_FORMAT_Y8_BL
  • VPI_IMAGE_FORMAT_Y8_ER_BL
  • VPI_IMAGE_FORMAT_Y16_ER_BL
  • VPI_IMAGE_FORMAT_NV24
  • VPI_IMAGE_FORMAT_NV24_ER
  • VPI_IMAGE_FORMAT_NV24_BL
  • VPI_IMAGE_FORMAT_NV24_ER_BL
• Rescale (VIC), Remap (VIC), Convert Image Format (VIC, CUDA and CPU) algorithms now supports NV24 image format.
• Remap on VIC now accepts U8 and U16 formats.
• Speed up NvBuffer and EGLImage wrapping, specially if CPU backend is disabled.
• Speed up pipeline processing by minimizing internal memory mapping operations.
• Added NVTX support to VPI calls and internal algorithm processing. This allows the user to use NVIDIA Nsight Systems to gather more detailed information (timing, etc) on how VPI pipeline is being executed.
• Convolution on PVA allows kernel weights to be between -1 (inclusive) and 1 (inclusive). Before weights must be strictly greater than -1 and smaller than 1.
• Increased maximum allowed bounding box size on KLT Feature Tracker to 128x128.

API Updates

• Passing NULL as params argument to vpiSubmitOpticalFlowPyrLK makes the algorithm take default params.
• Added vpiImageDataImportOpenCVMat and vpiImageDataImportOpenCVMatPlanes functions. They allow filling up a VPIImageData structure with memory buffer information of an existing cv::Mat.
• Added new functions:
  • vpiPixelTypeGetBitsPerChannel
  • vpiInitHarrisCornerDetectorParams
  • vpiInitOpticalFlowPyrLKParams
  • vpiPayloadGetFlags
  • vpiArrayTypeGetName
  • vpiArrayTypeGetSize
  • vpiPyramidGetScale
• Removed vpiPayloadGetBackend - use vpiPayloadGetFlags instead.
• Added VPIStereoDisparityEstimatorCreationParams::downscaleFactor attribute.
• Added confidenceThreshold and quality attributes to VPIStereoDisparityEstimatorParams.
• Added scale attribute to VPIPyramidData.
• VPIWarpMap::pitchBytes is now a 32-bit integer.
• Setting VPIStereoDisparityEstimatorParams::maxDisparity to 0 makes the Stereo Disparity Estimator algorithm use the maximum disparity defined during payload creation.
• Memory objects (images, arrays or pyramids) that are guaranteed to be used by only one stream can be created with flag VPI_EXCLUSIVE_STREAM_ACCESS, allowing faster internal memory handling (mapping among available backends, etc).
• Algorithm submissions to a backend are now deferred until the corresponding VPI stream is explicitly flushed via vpiStreamFlush or implicitly. See Streams for more details. For old behavior, create the VPIStream with flag VPI_STREAM_GREEDY. With it, every algorithm submission implicitly flushes the stream.

Other

• Reduce memory footprint when CUDA backend isn't enabled in current VPI context.
• Algorithm submit calls fail immediately in case of parameter validation errors. Before it would succeed, with failure status delivered asynchronously.
• Wrapping user-provided CPU or CUDA memory buffers automatically enables the corresponding backends in the VPIImage or VPIArray object.
• Errors originated in one stream are propagated downwards to other streams in the processing pipeline.
• Global context has only available backends enabled.

Selected Bug Fixes

Note

When installing VPI via the SDK Manager installer, it's advisable to upgrade the installed VPI 1.1.11 to the most recent version to take advantage of the bug fixes made after JetPack 4.6 was released. To do so, please execute the following commands in a terminal:

sudo apt-get update
sudo apt-get install libnvvpi1

v1.1.12

• Fixed performance regression in CUDA implementation Color Names algorithm that led to 10x slowdown in Jetson Nano and Jetson TX2 devices.
• Fixed performance regression in CPU implementation of Box Filter, Gaussian Filter and Separable Convolution that led to 2x slowdown on Jetson AGX Xavier and around 1.2x slowdown in other platforms.
• Passing NULL to outBGImage parameter of vpiSubmitBackgroundSubtractor resulted in a segmentation fault. Now this is accepted as a means of informing VPI that the background image output isn't needed.

v1.1.11

• Fix output buffer corruption in vpiImageFormatGetName.
• Do not segfault when creating VPIWarpMap that is too large.

Known Issues

• Convert Image Format on CUDA might introduce a small error of at most 2 when compared with other backends.
• PVA backend implementation of KLT Feature Tracker doesn't match CUDA and CPU's output. Tracking is lost more frequently.
• PVA backend implementation of vpiSubmitConvolution currently doesn't work with 3264x2448 inputs, it returns an error instead.
• Harris Corner Detector on PVA may return spurious keypoints when input image is larger than 1088x1088.
• CPU to CUDA image shared mapping of wrapped non-CUDA-managed CPU memory had to be disabled due to some rare segfaults. In this case, memory mapping is now done via memory copies.
• Harris Corner Detector result scores/positions might differ among backends.
• Temporal Noise Reduction and Perspective Warp samples' output image extension must be .avi on Ubuntu 16.04. Using .mp4 might lead to failures.
• Stereo Disparity Estimator
  • output differs significantly between CPU and new CUDA backend implementation.
  • CUDA backend still uses the old implementation in existing programs linked against VPI-1.0 ABI. This behavior was chosen due to backward compatibility concerns.
  • On CPU backend and old CUDA backend (VPI-1.0 ABI), no checking on maximum disparity limit is being performed. It's recommended set maximum disparity to at most 64. Using a higher value leads to undefined behavior: too much memory is allocated, which may lead to system running out of memory.

Notices

Disclaimer

ALL NVIDIA DESIGN SPECIFICATIONS, REFERENCE BOARDS, FILES, DRAWINGS, DIAGNOSTICS, LISTS, AND OTHER DOCUMENTS (TOGETHER AND SEPARATELY, "MATERIALS") ARE BEING PROVIDED "AS IS." NVIDIA MAKES NO WARRANTIES, EXPRESSED, IMPLIED, STATUTORY, OR OTHERWISE WITH RESPECT TO THE MATERIALS, AND EXPRESSLY DISCLAIMS ALL IMPLIED WARRANTIES OF NONINFRINGEMENT, MERCHANTABILITY, AND FITNESS FOR A PARTICULAR PURPOSE.

Information furnished is believed to be accurate and reliable. However, NVIDIA Corporation assumes no responsibility for the consequences of use of such information or for any infringement of patents or other rights of third parties that may result from its use. No license is granted by implication of otherwise under any patent rights of NVIDIA Corporation. Specifications mentioned in this publication are subject to change without notice. This publication supersedes and replaces all other information previously supplied. NVIDIA Corporation products are not authorized as critical components in life support devices or systems without express written approval of NVIDIA Corporation.

Trademarks

NVIDIA and the NVIDIA logo are trademarks or registered trademarks of NVIDIA Corporation in the U.S. and other countries. Other company and product names may be trademarks of the respective companies with which they are associated.

Copyright

© 2019-2021 NVIDIA Corporation. All rights reserved.