Benchmarking

Overview

This application shows how to properly measure the time taken to process VPI tasks. It follows a simplified version of the procedure described in Benchmarking Method section.

In a nutshell, the sample returns the median of a series of measurements of 5x5 Gaussian Filter algorithm execution time. Each measurement comprises running the algorithm 50 times and taking the average running time. This form of measurement in batches and taking the median leads to a more stable elapsed time value, as it excludes external perturbation factors.

Instructions

The usage is:

./vpi_sample_05_benchmark <backend>

where

• backend: either cpu, cuda or pva; it defines the backend that will perform the processing.

Here's one example:

./vpi_sample_05_benchmark cuda

This is using the CUDA backend. Try other backends to see how the processing time differs between them.

Results

Note

The benchmark results shown below are for demonstration purposes only, therefore, we're not specifying here the hardware used. You should run the sample on the platform where you want to gather benchmarking information from.

CPU Backend

Input size: 1920 x 1080
Image format: VPI_IMAGE_FORMAT_U16
Algorithm: 5x5 Gaussian Filter
Approximated elapsed time per call: 2.368492 ms

CUDA Backend

Input size: 1920 x 1080
Image format: VPI_IMAGE_FORMAT_U16
Algorithm: 5x5 Gaussian Filter
Approximated elapsed time per call: 0.043012 ms

PVA Backend

Input size: 1920 x 1080
Image format: VPI_IMAGE_FORMAT_U16
Algorithm: 5x5 Gaussian Filter
NVMEDIA_ARRAY:   53,  Version 2.1
NVMEDIA_VPI :  172,  Version 2.4
Approximated elapsed time per call: 1.692010 ms

Source Code

For convenience, here's the code that is also installed in the samples directory.

Language: C++

#include <vpi/Event.h>
#include <vpi/Image.h>
#include <vpi/Status.h>
#include <vpi/Stream.h>
#include <vpi/algo/GaussianFilter.h>
 
#include <algorithm>
#include <cstdio>
#include <iostream>
#include <sstream>
#include <vector>
 
#define CHECK_STATUS(STMT)                                    \
    do                                                        \
    {                                                         \
        VPIStatus status = (STMT);                            \
        if (status != VPI_SUCCESS)                            \
        {                                                     \
            char buffer[VPI_MAX_STATUS_MESSAGE_LENGTH];       \
            vpiGetLastStatusMessage(buffer, sizeof(buffer));  \
            std::ostringstream ss;                            \
            ss << "" #STMT "\n";                              \
            ss << vpiStatusGetName(status) << ": " << buffer; \
            throw std::runtime_error(ss.str());               \
        }                                                     \
    } while (0);
 
int main(int argc, char *argv[])
{
    VPIImage image   = NULL;
    VPIImage blurred = NULL;
    VPIStream stream = NULL;
 
    VPIEvent evStart = NULL;
    VPIEvent evStop  = NULL;
 
    int retval = 0;
 
    try
    {
        // 1. Processing of command line parameters -----------
 
        if (argc != 2)
        {
            throw std::runtime_error(std::string("Usage: ") + argv[0] + " <cpu|pva|cuda>");
        }
 
        std::string strBackend = argv[1];
 
        // Parse the backend
        VPIBackend backend;
 
        if (strBackend == "cpu")
        {
            backend = VPI_BACKEND_CPU;
        }
        else if (strBackend == "cuda")
        {
            backend = VPI_BACKEND_CUDA;
        }
        else if (strBackend == "pva")
        {
            backend = VPI_BACKEND_PVA;
        }
        else
        {
            throw std::runtime_error("Backend '" + strBackend +
                                     "' not recognized, it must be either cpu, cuda or pva.");
        }
 
        // 2. Initialization stage ----------------------
 
        // Create the stream for only the target backend.
        uint64_t streamFlags = (uint64_t)backend | VPI_REQUIRE_BACKENDS;
        CHECK_STATUS(vpiStreamCreate(streamFlags, &stream));
 
        int width = 1920, height = 1080;
        VPIImageFormat imgFormat = VPI_IMAGE_FORMAT_U16;
 
        std::cout << "Input size: " << width << " x " << height << '\n'
                  << "Image format: " << vpiImageFormatGetName(imgFormat) << '\n'
                  << "Algorithm: 5x5 Gaussian Filter on " << strBackend << std::endl;
 
        // Memory flags set to guarantee top performance.
        // Only the benchmarked backend is enabled, and memories
        // are guaranteed to be used by only one stream.
        uint64_t memFlags = (uint64_t)backend | VPI_EXCLUSIVE_STREAM_ACCESS;
 
        // Create image with zero content
        CHECK_STATUS(vpiImageCreate(width, height, imgFormat, memFlags, &image));
 
        // Create a temporary image convolved with a low-pass filter.
        CHECK_STATUS(vpiImageCreate(width, height, imgFormat, memFlags, &blurred));
 
        // Create the events we'll need to get timing info
        CHECK_STATUS(vpiEventCreate(backend, &evStart));
        CHECK_STATUS(vpiEventCreate(backend, &evStop));
 
        // 3. Gather timings --------------------
 
        const int BATCH_COUNT     = 20;
        const int AVERAGING_COUNT = 50;
 
        // Collect measurements for each execution batch
        std::vector<float> timingsMS;
        for (int batch = 0; batch < BATCH_COUNT; ++batch)
        {
            // Record stream queue when we start processing
            CHECK_STATUS(vpiEventRecord(evStart, stream));
 
            // Get the average running time within this batch.
            for (int i = 0; i < AVERAGING_COUNT; ++i)
            {
                // Call the algorithm to be measured.
                CHECK_STATUS(vpiSubmitGaussianFilter(stream, backend, image, blurred, 5, 5, 1, 1, VPI_BORDER_ZERO));
            }
 
            // Record stream queue just after blurring
            CHECK_STATUS(vpiEventRecord(evStop, stream));
 
            // Wait until the batch processing is done
            CHECK_STATUS(vpiEventSync(evStop));
 
            float elapsedMS;
            CHECK_STATUS(vpiEventElapsedTimeMillis(evStart, evStop, &elapsedMS));
            timingsMS.push_back(elapsedMS / AVERAGING_COUNT);
        }
 
        // 4. Performance analysis ----------------------
 
        // Get the median of the measurements so that outliers aren't considered.
        nth_element(timingsMS.begin(), timingsMS.begin() + timingsMS.size() / 2, timingsMS.end());
        float medianMS = timingsMS[timingsMS.size() / 2];
 
        printf("Approximated elapsed time per call on %s: %f ms\n", strBackend.c_str(), medianMS);
    }
    catch (std::exception &e)
    {
        std::cerr << e.what() << std::endl;
        retval = 1;
    }
 
    // 4. Clean up -----------------------------------
 
    // Destroy stream first, it'll make sure all processing
    // submitted to it is finished.
    vpiStreamDestroy(stream);
 
    // Now we can destroy other VPI objects, since they aren't being
    // used anymore.
    vpiImageDestroy(image);
    vpiImageDestroy(blurred);
    vpiEventDestroy(evStart);
    vpiEventDestroy(evStop);
 
    return retval;
}