VPI - Vision Programming Interface

0.4.4 Release

Cross-Compilation Targeting aarch64

Overview

This sample shows how to build your applications in a x86_64 host, targeting Jetson devices that use aarch64 architecture. It uses several features first made available in cmake-3.5. The sample application itself creates an input image, applies a box filter to it and save the result to disk.

This sample shows the following:

  • Create a x86_64 build tree for cross-compilation targeting aarch64 architecture.
  • Creating and destroying a VPI stream.
  • Wrapping an image hosted on CPU (the input) to be used by VPI.
  • Creating a VPI-managed 2D image where output will be written to.
  • Use of Box Filter algorithm.
  • Simple stream synchronization.
  • Image locking to access its contents from CPU side.
  • Error handling.
  • Environment clean up.

Instructions

JetPack's installer already set up the cross-compilation toolchain using gcc, but if some reason it isn't available, install it manually with:

apt-get install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu

Now cmake can be instructed to create a cross-compiling build tree by calling it as follows in the samples directory:

cmake . -DCMAKE_TOOLCHAIN_FILE=Toolchain_aarch64_l4t.cmake

The file Toolchain_aarch64_l4t.cmake is included in the samples directory and defines the cross-compiler that will be used, among other configurations. In particular, it also allows cross-compilation of CUDA applications, provided that the CUDA aarch64 cross-compilation libraries are correctly installed on host.

Note
This sample can also be compiled targeting the host. Just omit the CMAKE_TOOLCHAIN_FILE parameter during cmake invocation.

The usage is:

./vpi_sample_08_cross_aarch64_l4t <backend>

where

  • backend: either cpu, cuda or pva; it defines the backend that will perform the processing.

Source code

For convenience, here's the code that is also installed in the samples directory.

/*
* Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <vpi/Image.h>
#include <vpi/Status.h>
#include <vpi/Stream.h>
#include <cstring> // for memset
#include <fstream>
#include <iostream>
#include <sstream>
#define CHECK_STATUS(STMT) \
do \
{ \
VPIStatus status = (STMT); \
if (status != VPI_SUCCESS) \
{ \
char buffer[VPI_MAX_STATUS_MESSAGE_LENGTH]; \
vpiGetLastStatusMessage(buffer, sizeof(buffer)); \
std::ostringstream ss; \
ss << vpiStatusGetName(status) << ": " << buffer; \
throw std::runtime_error(ss.str()); \
} \
} while (0);
int main(int argc, char *argv[])
{
VPIImage image = NULL;
VPIImage blurred = NULL;
VPIStream stream = NULL;
int retval = 0;
try
{
if (argc != 2)
{
throw std::runtime_error(std::string("Usage: ") + argv[0] + " <cpu|pva|cuda>");
}
std::string strBackend = argv[1];
// Now parse the backend
VPIBackend backendType;
if (strBackend == "cpu")
{
backendType = VPI_BACKEND_CPU;
}
else if (strBackend == "cuda")
{
backendType = VPI_BACKEND_CUDA;
}
else if (strBackend == "pva")
{
backendType = VPI_BACKEND_PVA;
}
else
{
throw std::runtime_error("Backend '" + strBackend +
"' not recognized, it must be either cpu, cuda or pva.");
}
// Create the stream for the given backend.
CHECK_STATUS(vpiStreamCreate(backendType, &stream));
char imgContents[512][512];
for (int i = 0; i < 512; ++i)
{
for (int j = 0; j < 512; ++j)
{
imgContents[i][j] = i * 512 + j + i;
}
}
// We now wrap the loaded image into a VPIImage object to be used by VPI.
{
// First fill VPIImageData with the, well, image data...
VPIImageData imgData;
memset(&imgData, 0, sizeof(imgData));
imgData.numPlanes = 1;
imgData.planes[0].width = 512;
imgData.planes[0].height = 512;
imgData.planes[0].pitchBytes = 512;
imgData.planes[0].data = imgContents[0];
// Wrap it into a VPIImage. VPI won't make a copy of it, so the original
// image must be in scope at all times.
CHECK_STATUS(vpiImageCreateHostMemWrapper(&imgData, 0, &image));
}
// Now create the output image, single unsigned 8-bit channel.
CHECK_STATUS(vpiImageCreate(512, 512, VPI_IMAGE_FORMAT_U8, 0, &blurred));
// Submit it for processing passing the image to be blurred and the result image
CHECK_STATUS(vpiSubmitBoxFilter(stream, backendType, image, blurred, 3, 3, VPI_BOUNDARY_COND_ZERO));
// Wait until the algorithm finishes processing
CHECK_STATUS(vpiStreamSync(stream));
// Now let's retrieve the output image contents and output it to disk
{
// Lock output image to retrieve its data on cpu memory
VPIImageData outData;
CHECK_STATUS(vpiImageLock(blurred, VPI_LOCK_READ, &outData));
std::ofstream fd(("boxfiltered_" + strBackend + ".pgm").c_str());
fd << "P5\n512 512 255\n";
for (int i = 0; i < 512; ++i)
{
fd.write(reinterpret_cast<const char *>(outData.planes[0].data) + outData.planes[0].pitchBytes * i,
512);
}
fd.close();
// Done handling output image, don't forget to unlock it.
CHECK_STATUS(vpiImageUnlock(blurred));
}
}
catch (std::exception &e)
{
std::cerr << e.what() << std::endl;
retval = 1;
}
// Clean up
// Make sure stream is synchronized before destroying the objects
// that might still be in use.
if (stream != NULL)
{
vpiStreamSync(stream);
}
vpiImageDestroy(blurred);
return retval;
}

Here is the cmake toolchain file that is being used.

# Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of NVIDIA CORPORATION nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
set(target_arch aarch64-linux-gnu)
set(CMAKE_LIBRARY_ARCHITECTURE ${target_arch} CACHE STRING "" FORCE)
# Configure cmake to look for libraries, include directories and
# packages inside the target root prefix.
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
set(CMAKE_FIND_ROOT_PATH "/usr/${target_arch}")
# needed to avoid doing some more strict compiler checks that
# are failing when cross-compiling
set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY)
# specify the toolchain programs
find_program(CMAKE_C_COMPILER ${target_arch}-gcc)
find_program(CMAKE_CXX_COMPILER ${target_arch}-g++)
if(NOT CMAKE_C_COMPILER OR NOT CMAKE_CXX_COMPILER)
message(FATAL_ERROR "Can't find suitable C/C++ cross compiler for ${target_arch}")
endif()
set(CMAKE_AR ${target_arch}-ar CACHE FILEPATH "" FORCE)
set(CMAKE_RANLIB ${target_arch}-ranlib)
set(CMAKE_LINKER ${target_arch}-ld)
# Not all shared libraries dependencies are instaled in host machine.
# Make sure linker doesn't complain.
set(CMAKE_EXE_LINKER_FLAGS_INIT -Wl,--allow-shlib-undefined)
# instruct nvcc to use our cross-compiler
set(CMAKE_CUDA_FLAGS "-ccbin ${CMAKE_CXX_COMPILER} -Xcompiler -fPIC" CACHE STRING "" FORCE)

And finally the accompanying CMakeLists.txt. Note that it is just a plain simple CMakeLists.txt. Everything related to cross-compilation is defined in the toolchain file above.

1 # Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
2 #
3 # Redistribution and use in source and binary forms, with or without
4 # modification, are permitted provided that the following conditions
5 # are met:
6 # * Redistributions of source code must retain the above copyright
7 # notice, this list of conditions and the following disclaimer.
8 # * Redistributions in binary form must reproduce the above copyright
9 # notice, this list of conditions and the following disclaimer in the
10 # documentation and/or other materials provided with the distribution.
11 # * Neither the name of NVIDIA CORPORATION nor the names of its
12 # contributors may be used to endorse or promote products derived
13 # from this software without specific prior written permission.
14 #
15 # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
16 # EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
17 # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
18 # PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
19 # CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
20 # EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
21 # PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
22 # PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
23 # OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
24 # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
25 # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
26 
27 cmake_minimum_required(VERSION 3.5)
28 
29 # To cross-compile for aarch64-l4t target,
30 # pass -DCMAKE_TOOLCHAIN_FILE=Toolchain_aarch64_l4t.cmake
31 # to cmake when creating build tree.
32 
33 project(vpi_sample_08_cross_aarch64_l4t)
34 
35 find_package(vpi 0.4.0 REQUIRED)
36 
37 add_executable(${PROJECT_NAME} main.cpp)
38 target_link_libraries(${PROJECT_NAME} vpi)
VPIImagePlane::height
uint32_t height
Height of this plane in pixels.
Definition: Image.h:138
VPIImagePlane::width
uint32_t width
Width of this plane in pixels.
Definition: Image.h:137
vpiStreamCreate
VPIStatus vpiStreamCreate(uint32_t flags, VPIStream *stream)
Create a stream instance.
VPIBackend
VPIBackend
VPI Backend types.
Definition: Types.h:89
BoxFilter.h
Declares functions that implement the Box Filter algorithm.
VPI_LOCK_READ
@ VPI_LOCK_READ
Lock memory only for reading.
Definition: Types.h:447
vpiImageUnlock
VPIStatus vpiImageUnlock(VPIImage img)
Releases the lock on an image object.
vpiStreamSync
VPIStatus vpiStreamSync(VPIStream stream)
Blocks the calling thread until all submitted commands in this stream queue are done (queue is empty)...
VPI_BACKEND_CUDA
@ VPI_BACKEND_CUDA
CUDA backend.
Definition: Types.h:91
VPIImageData
Stores information about image characteristics and content.
Definition: Image.h:159
VPIStream
struct VPIStreamImpl * VPIStream
A handle to a stream.
Definition: Types.h:190
vpiStreamDestroy
void vpiStreamDestroy(VPIStream stream)
Destroy a stream instance and deallocate all HW resources.
VPIImageData::planes
VPIImagePlane planes[VPI_MAX_PLANE_COUNT]
Data of all image planes.
Definition: Image.h:166
vpiImageCreate
VPIStatus vpiImageCreate(uint32_t width, uint32_t height, VPIImageFormat fmt, uint32_t flags, VPIImage *img)
Create an empty image instance with the specified flags.
vpiImageDestroy
void vpiImageDestroy(VPIImage img)
Destroy an image instance.
vpiSubmitBoxFilter
VPIStatus vpiSubmitBoxFilter(VPIStream stream, VPIBackend backend, VPIImage input, VPIImage output, uint32_t kernelSizeX, uint32_t kernelSizeY, VPIBoundaryCond boundary)
Runs a 2D box filter over an image.
Image.h
Functions and structures for dealing with VPI images.
VPI_IMAGE_FORMAT_U8
@ VPI_IMAGE_FORMAT_U8
Single plane with one 8-bit unsigned integer channel.
Definition: ImageFormat.h:99
VPIImagePlane::pitchBytes
uint32_t pitchBytes
Difference in bytes of beginning of one row and the beginning of the previous.
Definition: Image.h:139
VPIImage
struct VPIImageImpl * VPIImage
A handle to an image.
Definition: Types.h:196
VPIImageData::numPlanes
int32_t numPlanes
Number of planes.
Definition: Image.h:161
VPI_BOUNDARY_COND_ZERO
@ VPI_BOUNDARY_COND_ZERO
All pixels outside the image are considered to be zero.
Definition: Types.h:218
vpiImageLock
VPIStatus vpiImageLock(VPIImage img, VPILockMode mode, VPIImageData *hostData)
Acquires the lock on an image object and returns a pointer to the image planes.
VPIImageData::type
VPIImageFormat type
Image type.
Definition: Image.h:160
Status.h
Declaration of VPI status codes handling functions.
VPI_BACKEND_CPU
@ VPI_BACKEND_CPU
CPU backend.
Definition: Types.h:90
vpiImageCreateHostMemWrapper
VPIStatus vpiImageCreateHostMemWrapper(const VPIImageData *hostData, uint32_t flags, VPIImage *img)
Create an image object by wrapping around an existing host memory block.
Stream.h
Declares functions dealing with VPI streams.
VPIImagePlane::data
void * data
Pointer to the first row of this plane.
Definition: Image.h:147
VPI_BACKEND_PVA
@ VPI_BACKEND_PVA
PVA backend.
Definition: Types.h:92