Architecture#

Overview#

The PVA engine is a VLIW SIMD DSP which is optimized for implementing image processing and computer vision algorithms, available on certain NVIDIA SOCs. Key places where PVA’s capabilities are a good match are algorithmic domains where we need to have predictable processing, at low power and low latency. PVA shines on semi-dense or dense regular computation, even on small data sets, which need predictable run-times with low latency and low power.

The PVA SDK provides libraries, headers, and tools for writing, debugging and optimizing applications for the PVA.

A PVA application consists of host code and device code:

  • Host code is used to configure a PVA program. This code runs on the SOC’s CPU. It is responsible for setting up DMA operations via Dataflow abstractions, allocating resources for applications, and handling scheduling with other engines.

  • Device code is the code which runs on the PVA’s vector processing unit (VPU). It is responsible for triggering DMA operations and processing data.

Both host and device code use the cuPVA runtime to interact with the PVA engine.

PVA Processor#

PVA processor architecture contains two Vector Processing Subsystems (VPS) and a Control Subsystem (Cortex-R5). Each VPS may be independently programmed by the user. The Control Subsystem schedules work between the two VPSs and is not available for user programming.

A simplified block diagram is depicted in Figure 1.

PVA Architecture

Figure 1: PVA Architecture#

Each component of the PVA is described in detail below.

VPU#

The Vector Processing Unit (VPU) is the main processing engine of the PVA. VPU is closely tied together with VMEM, DLUT and Instruction cache to form the VPS (Vector Processing Subsystem).

The VPU instruction format is 7-way VLIW, consisting of:

  • 2 scalar ALUs

  • 2 vector ALUs

  • 3 load/store units

Vector ALUs are the heavy lifters for image processing and computer vision kernels. The Wide SIMD vector unit executes up to 2 vector math instructions per cycle. Each VPU SIMD instruction can process 32 pixels (12-bits per pixel), in parallel. An extensive set of general and CV specific Integer arithmetic/logic operations and Floating Point arithmetic and math operations are supported.

VMEM#

VPU vector memory (VMEM) houses local data memory for VPU so that various image processing and computer vision algorithms can be implemented efficiently. VMEM supports various complex memory access patterns, including consecutive read/write of various lengths, transposition, table lookup, histogram and vector addressed stores.

DLUT#

Each VPS includes a decoupled look-up table unit (DLUT). The DLUT can operate concurrently and in parallel with the VPU to offload VMEM lookup operations. DLUT offers high throughput lookup operations with hardware acceleration to minimize memory bank conflicts. The DLUT may be used from VPU via the Sampler API.

DMA Controller#

VPU can only load from or store to its internal memory, VMEM. VPU needs the Direct Memory Access (DMA) engine to move data among external memory, L2 memory and VMEM. Each VPS block within PVA has a dedicated DMA engine and each DMA engine has multiple channels that can move data from multiple source locations to destination locations in parallel. Memory read/write operations can be done in parallel with the VPU processes to hide the data access latency. Each VPU and DMA pair synchronize between themselves on tile granularity. Tiles are typically sub-blocks of the application’s input/output dataset from external memory and there are typically tens to hundreds of tiles per task. The tiles are sized to fit within VMEM, and the DMA engine is used to traffic these sub-blocks to VMEM for VPU processing.

The host-visible DataFlow APIs map onto these channels and descriptors at compile time. cuPVA does not impose a fixed numeric cap on the number of DataFlow objects a single cupva::CmdProgram may contain; what matters is that each DataFlow added with cupva::CmdProgram::addDataFlowHead() consumes a DMA channel for the lifetime of the program, while DataFlows added with cupva::CmdProgram::addDataFlow() chain off an existing head without claiming an additional channel. Programs that request more heads than the target VPS has channels are rejected at cupva::CmdProgram::compileDataFlows() time. cupva::GatherScatterDataFlow adds hardware-defined per-trigger caps (GSDF_MAX_NUM_TILES tiles per trigger and GSDF_MAX_NUM_PLANES image planes per dataflow); see the Dataflow configuration reference for current values.

L2 Buffer#

PVA employs an L2 memory that is shared between the two sets of VPS and DMA. L2 buffer size is larger than VMEM and has a higher bandwidth than the external DRAM. It can be used to fetch data blocks that cannot fit to VMEM or as a scratch buffer.

Debug hardware#

Each VPU has a CoreSight/APB-based debug interface connected to a system-level JTAG interface. It is accessible through JTAG or through CPU software. The VPU supports typical symbolic debug features such as:

  • Read/write VMEM

  • Read/write processor registers

  • Single stepping

  • 24 hardware watch/break points

  • Unlimited software breakpoints

To use the VPU debug hardware, refer to On-Chip debugging.

Control Subsystem (Cortex-R5)#

PVA has a control subsystem, Cortex-R5. Cortex-R5 is the main orchestrator of tasks running on PVA as well as its external interface. It interacts with host CPU, GPU, other CV system engines including ISP and hardware accelerators such as NVENC.

R5 configures, controls and monitors VPU and DMA tasks that are needed for PVA functions. R5 configures DMA, optionally prefetches VPU program into VPU Instruction cache, and kicks off each VPU-DMA pair to process a task that runs for typically hundreds of micro-seconds to a few milli-seconds.

For a cuPVA program, the R5 acts as an opaque scheduler and is not exposed as part of the cuPVA programming model.

Safety Usage#

Hardware#

In accordance with ISO 26262-5; 7.1.2, the ASIL supported by PVA hardware is ASIL-D systematic and ASIL-B random.

Software#

The safety builds of the cuPVA host and device runtimes are candidates for safety certification at ASIL-B level when used as part of a suitable DRIVE OS safety release. Refer to the certification to determine which versions of the cuPVA runtime are suitable for safety use and the appropriate DRIVE OS version.

The majority of the cuPVA host and device APIs are provided in safety builds, meaning applications developed on non-safety DRIVE OS versions can be easily ported to safety builds in most cases. There are a few exceptions to this rule, relating to debugging APIs and features not available in safety builds.

APIs which are not available in safety builds may be identified in the following ways:

  • All such APIs are defined in headers which are prefaced by a guard which requires the preprocessor token PVA_ALLOW_NONSAFETY to be set to 1. If this is not set, and the file is included, the preprocessor generates an error. PVA SDK build scripts set this token to 1 if PVA_SAFETY is not set as a CMake option.

  • All such APIs are clearly marked in the documentation.

Host API#

Overview#

The Host API is designed to assist the user in doing all necessary setup steps on the SOC’s CPU, before handing execution over to the PVA. The following describes typical host code steps to configure of a simple application, synchronous with the CPU:

cupva::Stream::synchronize() reports cupva::Error::InvalidContext if the Context was invalidated.

For more detailed examples of usage, please see the samples distributed with the SDK or the Tutorials.

Context Management#

Many cuPVA APIs require a Context to be current on the calling thread. The Context that is current to a thread can be set by calling cupva::Context::SetCurrent() from the thread. Creating a Context also causes the new Context to become current on the calling thread.

If an API requiring a Context is called from a thread that does not have a current Context, a default process-wide Context is used. This Context is only created when it is needed, and is destroyed when the process exits.

There is a system-wide limit on the number of Contexts that may be created across all processes, including default Contexts. On currently supported platforms, this limit is 4. Exceeding this limit causes future Context creation to fail.

Error Reporting#

cuPVA reports both synchronous API errors and asynchronous errors.

Synchronous errors#

A synchronous error is detected while an API call is executing.

The cuPVA host API is a C++17 API that reports failure via throwing exceptions. The possible exception types are cupva::Exception as well as C++ standard library exceptions. All APIs provide an exception specification for the possible exceptions that they may throw. Users must catch all possible exceptions where process termination is not the desired behavior.

Any cuPVA C API that returns a cupvaError_t may also produce a detailed error message string. Use CupvaGetLastError() to retrieve the error code and read the message left by only the most recent cuPVA C API call made from the current thread. CupvaGetLastError() itself does not modify any error state.

Asynchronous Context errors#

Note

Everything below requires PVA driver version 2009 or newer, which cupva::GetDriverVersion() reports. Prior to 2009:

  • Asynchronous errors do not invalidate a Context. Work submitted after a failure continues to be accepted.

  • cupva::GetStatus() throws, and CupvaGetStatus() returns, InsufficientDriverVersion.

  • Status for asynchronous work is available only where a CmdStatus array was supplied at submit time, read with cupva::CheckCommandStatus() once the corresponding command has completed.

Work submitted to a Stream executes asynchronously, so a failure may be detected after the submission API has returned successfully. A later API call can therefore report an error caused by previously submitted work.

Such a failure leaves the Context in an invalid state, and an invalid Context cannot be recovered. To continue, release the objects created on that Context, destroy it, and start again with a new one.

That applies to a Context the application created. An application that never creates one uses a Context cuPVA creates implicitly, which cannot be destroyed; if it is invalidated, the process must be restarted. An application that needs to recover from asynchronous Context errors should manage its own Contexts and avoid using the cuPVA default Context.

Use cupva::GetStatus(), or CupvaGetStatus() from C, to ask whether the current Context has been invalidated and to read a structured description of the failure. These inspect only the Context, not the error state left by the most recent C API call.

After a submission fails, query the Context. A submission refused because the Context is invalid reports InvalidContext and no other failure does, so any other code belongs to that call alone and can be retried.

Command-status and fence APIs continue to report the status of their corresponding work.

Naming Streams#

A failure is attributed to a Stream by its creation order on the Context: the first Stream created is Stream 1.

cupva::Stream::setName() and CupvaStreamSetName() attach a name to a Stream, which is then reported alongside the number in the streamName field. Naming is optional; an unnamed Stream is still reported by number and streamName is null.

Identifying the failing submission#

Match an asynchronous error to the work that caused it through cupva::Status, or cupvaStatus_t in C. Its fields identify the Stream, how far back the failing submission sits in that Stream’s history, and which command within that submission failed, in the same terms the application used to submit the work.

A detailed error message is also provided, for logs and for NVIDIA support:

Context invalidated by cupva::Error::VpuApplicationError on Stream 1 "detect",
235 submissions before the most recent, command 0. ...

It is not a stable interface and may change between releases, so it must not be parsed or acted on. Use the structure for anything the application decides on.

An application that matches a failure against its own records needs to keep enough of them to cover the submissions it can have in flight at once. A pipeline that submits without waiting runs well ahead of the PVA, so a failure is commonly reported some hundreds of submissions back.

Worked example#

The error_reporting sample provokes a device-side failure and then shows both reporting paths side by side. It creates its own Context, which becomes current for the calling thread, so that it holds a handle it can query.

The C++ sample runs a pipeline the way a real application does: it submits frame after frame without waiting on each one. submit() queues work and returns, so the frame that fails is already well behind by the time the PVA executes it. Nothing is reported at the point it is submitted. Instead a later submit() throws, because the Context has been invalidated in the meantime:

        struct Submitted
        {
            uint32_t frame;
            char_t const *stage;
        };
        // Newest at the front, so an offset into it is what Status::submissionsSinceError reports. Deep enough
        // to cover the submissions this pipeline can have in flight at once, which here is a couple of hundred.
        std::deque<Submitted> recentSubmissions;
        constexpr size_t kHistoryDepth{512U};

        // Long enough that the pipeline is still running when the failure surfaces.
        constexpr uint32_t kFrames{500U};
        constexpr uint32_t kFailingFrame{73U};
        CmdStatus status[2]{};

        bool submitThrew{false};
        for (uint32_t frame{0U}; frame < kFrames; ++frame)
        {
            CmdProgram frameProgram    = CmdProgram::Create(exec);
            frameProgram["shouldFail"] = (frame == kFailingFrame) ? 1 : 0;

            recentSubmissions.push_front(Submitted{frame, "detect"});
            if (recentSubmissions.size() > kHistoryDepth)
            {
                recentSubmissions.pop_back();
            }

            try
            {
                failingStream.submit({&frameProgram, &rf}, status);
            }
            catch (cupva::Exception const &e)
            {
                // InvalidContext means an earlier frame failed on the device and invalidated the Context, and
                // this submission is the first to run into it. Recovery means destroying the Context and building
                // a new one.
                if (e.getErrorCode() == cupva::Error::InvalidContext)
                {
                    std::cout << "submit() threw while submitting frame " << frame << ": " << e.what() << std::endl;
                    submitThrew = true;
                    break;
                }
                // Any other code is a failure of this call alone, which the application can retry.
                std::cout << "submit() failed for frame " << frame << " but the Context is usable: " << e.what()
                          << std::endl;
            }
        }

        if (!submitThrew)
        {
            // Every submission was accepted, so the failure has not surfaced yet. Wait for the queued work to drain
            // so that it does. A real pipeline reaches the same point through its next fence wait.
            try
            {
                fence.wait();
            }
            catch (cupva::Exception const &e)
            {
                std::cout << "fence.wait() threw: " << e.what() << std::endl;
            }
        }

The sample reuses a single CmdStatus array, so what it holds by this point describes the most recent submission attempted rather than the one that failed. An application that kept a distinct CmdStatus for every command could locate the failure from those alone, by retaining one for everything it can have in flight and testing each in turn. The Context query reaches the same answer without any of that: it needs no CmdStatus at all, and names the Stream, the submission and the command directly:

        cupva::Error const statusCode{CheckCommandStatus(status[0])};
        std::cout << "CheckCommandStatus for the most recent submission attempted: " << static_cast<int32_t>(statusCode)
                  << std::endl;
        // [end_CheckCommandStatus]

        // GetStatus reports whether the Context is still usable, and which Stream, submission and command the
        // failure came from. It needs no CmdStatus. The description is for logs; its format is not stable between
        // releases.
        // [start_ContextQuery]
        cupva::Status const info{cupva::GetStatus()};
        cupva::Error const ctxError{info.err};
        std::cout << "GetStatus: " << info.errorMsg << std::endl;
        // The same facts as values, so they can be acted on without parsing the description.
        std::cout << "  structured: Stream " << info.stream << " ("
                  << ((info.streamName != nullptr) ? info.streamName : "unnamed") << "), ";
        if (info.submissionsSinceError == cupva::Status::SUBMISSION_UNKNOWN)
        {
            std::cout << "submission unknown";
        }
        else
        {
            std::cout << info.submissionsSinceError
                      << ((info.submissionsSinceError == 1U) ? " submission back" : " submissions back");
        }
        std::cout << ", command ";
        if (info.command == cupva::Status::COMMAND_UNKNOWN)
        {
            std::cout << "unknown" << std::endl;
        }
        else
        {
            std::cout << info.command << std::endl;
        }

An application that keeps a history of what it recently submitted can look the failure up in its own terms, without parsing the description:

        if ((info.submissionsSinceError != cupva::Status::SUBMISSION_UNKNOWN) &&
            (info.submissionsSinceError < recentSubmissions.size()))
        {
            Submitted const &entry = recentSubmissions[info.submissionsSinceError];
            std::cout << "  traced to frame " << entry.frame << ", stage " << entry.stage << ", command "
                      << info.command << std::endl;
        }
        else
        {
            // Only reachable if the history is shallower than the number of submissions that were in flight.
            std::cout << "  the failing submission is no longer in the recorded history" << std::endl;
        }

The error belongs to the Context rather than the Stream that caused it, so unrelated work on a second Stream is rejected as well:

        CmdProgram healthyProgram    = CmdProgram::Create(exec);
        healthyProgram["shouldFail"] = 0;
        try
        {
            otherStream.submit({&healthyProgram, &rf}, nullptr);
            fence.wait();
            std::cout << "Submission on the second Stream was accepted" << std::endl;
        }
        catch (cupva::Exception const &e)
        {
            std::cout << "Submission on the second Stream was rejected: " << e.what() << std::endl;
        }

A failing submission is checked first per command, then at Context scope:

    cupvaError_t statusCode = CUPVA_ERROR_NONE;
    CHECK_ERROR_GOTO(CupvaCheckCommandStatus(failingStatus[0], &statusCode), err, HealthyProgramCreateFailed);
    printf("CupvaCheckCommandStatus for the failing program: %d\n", (int32_t)statusCode);
    /** [end_CheckCommandStatus] */

    /* CupvaGetStatus reports whether the Context is still usable, and which Stream, submission and command the
     * failure came from. It needs no CmdStatus. The description is for logs; its format is not stable between
     * releases.
     */
    /** [start_ContextQuery] */
    CHECK_ERROR_GOTO(CupvaGetStatus(&status), err, HealthyProgramCreateFailed);
    printf("CupvaGetStatus: %s\n", status.errorMsg);
    /* The same facts as values, so they can be acted on without parsing the description. */
    char const *streamName = (status.streamName != NULL) ? status.streamName : "unnamed";
    /* submissionsSinceError counts back from the most recent submission on that Stream, 0 being the most recent. */
    char submission[32];
    if (status.submissionsSinceError == CUPVA_SUBMISSION_UNKNOWN)
    {
        snprintf(submission, sizeof(submission), "submission unknown");
    }
    else
    {
        snprintf(submission, sizeof(submission), "%u submission%s back", status.submissionsSinceError,
                 (status.submissionsSinceError == 1U) ? "" : "s");
    }
    if (status.command == CUPVA_COMMAND_UNKNOWN)
    {
        printf("  structured: Stream %u (%s), %s, command unknown\n", status.stream, streamName, submission);
    }
    else
    {
        printf("  structured: Stream %u (%s), %s, command %u\n", status.stream, streamName, submission, status.command);
    }

The error belongs to the Context rather than the Stream that caused it, so unrelated work on a second Stream is rejected as well:

    cupvaCmd_t healthyProgram;
    int32_t doNotFail = 0;
    CHECK_ERROR_GOTO(CupvaCmdProgramCreate(&healthyProgram, exec), err, HealthyProgramCreateFailed);
    CHECK_ERROR_GOTO(CupvaCmdProgramGetParameter(&healthyProgram, &shouldFailParameter, "shouldFail"), err,
                     DeAllocateAllResources);
    CHECK_ERROR_GOTO(CupvaParameterSetValueScalar(&shouldFailParameter, &doNotFail, sizeof(int32_t)), err,
                     DeAllocateAllResources);

    cupvaCmd_t const *healthyCmds[2] = {&healthyProgram, &rf};
    cupvaError_t const blockedErr    = CupvaStreamSubmit(otherStream, healthyCmds, NULL, 2, CUPVA_IN_ORDER, -1, -1);
    if (blockedErr == CUPVA_ERROR_NONE)
    {
        CHECK_ERROR_GOTO(CupvaFenceWait(&fence, -1, &waitSuccess), err, DeAllocateAllResources);
        printf("Submission on the second Stream was accepted\n");
    }
    /* CUPVA_INVALID_CONTEXT is expected, since the Context was invalidated earlier. CupvaGetStatus provides the
     * same details as before.
     */
    else if (blockedErr == CUPVA_INVALID_CONTEXT)
    {
        cupvaStatus_t blockedStatus;
        CHECK_ERROR_GOTO(CupvaGetStatus(&blockedStatus), err, DeAllocateAllResources);
        printf("Submission on the second Stream was rejected: %s\n", blockedStatus.errorMsg);
    }
    else
    {
        /* Any other code is a failure of this call alone, and could be retried. */
        char const *blockedMsg = NULL;
        (void)CupvaGetLastError(&blockedMsg);
        printf("Submission on the second Stream failed but the Context is usable: %s\n", blockedMsg);
    }

Host API Thread-Safety#

Each Host API is marked as being thread-safe or not thread-safe. An API is marked as thread-safe if and only if it is guaranteed to conform to its interface specification and avoid unintended behavior when called from multiple threads simultaneously with some reference or pointer arguments being shared. Likewise, different APIs marked as thread-safe may also be called from multiple threads simultaneously with shared reference or pointer arguments.

For the purposes of this definition, the implied ‘this’ pointer for non-static class member functions does count as an argument, but any other implied arguments (for example, the current cuPVA context) do not count as arguments.

All cuPVA host APIs are guaranteed to conform to their interface specification when all reference/pointer arguments are different between simultaneous calls from different threads. Any global state is synchronized between threads internally in such cases. For example, calling cupva::RasterDataFlow::src() on two different objects from two different threads is safe. Calling this method on the same cupva::RasterDataFlow object from two different threads is not safe. Likewise, move and copy assignment may be invoked on unrelated objects simultaneously from different threads.

The user must ensure that multiple threads do not simultaneously use APIs in violation of their concurrency specifications. Failure to follow the specification can lead to data corruption and race conditions.

Host API Design Concepts#

The Host API is designed around three distinct phases of an application: init, steady state and deinit.

  • APIs which can be used at init time may allocate heap and are not intended to be used in performance critical loops.

  • Steady state APIs and objects do not allocate heap and are optimized for use in a frame loop.

  • Deinit APIs are intended to be used for application tear-down only.

To help the user understand which objects allocate heap and are therefore intended to be created only at init time, such objects must be created via a static Create() method, for example Executable::Create(). Objects without such a method may be created and initialized normally via available constructors.

Note

Many objects with Create methods also have default constructors. This is to allow use in STL containers. Such objects still need to be initialized by assigning them the result of a call to Create() before they can be used.

In addition, some methods are designated as init-time only APIs, since they cause heap allocation.

Note

In general, an application using PVA transitions to steady-state after it has called cupva::CmdProgram::compileDataflows() for each CmdProgram it intends to use. After this point, the only changes that can be made to the DMA configuration of a program without recompiling are Dataflows data pointers by using OffsetPointer API.

Dataflows#

The PVA’s DMA engine is a powerful multi-dimensional engine for moving data from generally accessible memories to memories which can be used by the PVA and VPU, and vice-versa. Each cuPVA program is allocated a DMA engine for its use during run-time. A cuPVA CmdProgram needs to request DMA setup for its needed data transfers.

Note

The PVA’s VPU engines cannot access data memory other than VMEM without use of the DMA.

The core software abstraction used to represent data transfers in cuPVA is the Dataflow. Dataflows come in 4 varieties:

The DataFlow host APIs define a contract which must be met by the users’ device code. For example, RasterDataFlow and TensorDataFlow require a certain number of triggers. The VPU code must ensure exactly that number of triggers are provided at run-time. This requirement must be satisfied on every possible branch. On the other hand, GatherScatterDataFlow and SequenceDataFlow allow for a dynamic number of triggers depending on the data presented at runtime to the VPU kernel. These DataFlows must still be opened and closed by the VPU kernel.

Note

Failure to trigger a configured DMA channel to completion can lead to timeouts and have unpredictable effects on run-time for the entire PVA engine.

Dataflow objects are created to be associated with CmdProgram objects. Two APIs exist for this: CmdProgram::addDataFlow and CmdProgram::addDataFlowHead. The difference between these is that CmdProgram::addDataFlowHead allocates a DMA channel and start the DMA channel with this DataFlow. On the other hand, CmdProgram::addDataFlow does not initialize a DMA channel. Such a DataFlow is only executed when it is linked to from a “head.”

NvSci Interop#

The cuPVA Host API contains global functions designed to create cuPVA objects from NvSci objects and vice-versa. Since these APIs introduce an application level dependency on NvSci libraries and headers, they are defined in a separate header, cupva_host_nvsci.hpp. It is expected that users provide the necessary libraries or headers from NvSci to resolve these dependencies at build time.

Since the NvSci APIs interoperate with standard cuPVA objects, support for interop via NvSci can be added onto an existing application via a thin translation layer, with no impact on the underlying application implementation. Therefore, the same application or collection of applications can be used with or without NvSci.

Block Linear Surfaces#

A device pointer can be associated with surface metadata, for example when the device pointer represents a surface imported from NvSci. A surface may have block linear or pitch linear layout. Regardless of the surface layout, cuPVA host APIs always assume a pitch linear layout, with pitch specified by the cuPVA GetSurfaceAttributes API. For example:

// This function always returns a device pointer pointing to a pixel in the surface
// offset by 2 rows, regardless of whether the surface is block linear or pitch linear
const int8_t *offsetTwoRows(const int8_t *src_d)
{
  cupva::mem::SurfaceAttributes attr;
  cupva::mem::GetSurfaceAttributes(src_d, attr);
  uint32_t linePitch = attr.planes[planeIdx].linePitch;
  return src_d + 2 * linePitch;
}

Similarly, the “advance” fields of DataFlows should be configured for pitch linear layout, regardless of the actual layout of the surface.

When configuring surface addresses from VPU code, src and dst APIs should be passed real addresses, rather than pitch linear equivalents. To determine the correct block-linear address for src/dst fields, extra surface metadata is required. This may be written to VMEM by using the VMEM_SURFACE macro as a replacement for VMEM_POINTER. The user can then use cupvaSurfaceAddress2D() device side APIs to calculate the correct address to use.

There are additional alignment constraints when using block linear surfaces with DataFlow APIs. Pointers to block linear surfaces must be aligned to 32 bytes horizontally and 2 rows vertically. Advancement fields (for example, in dim1/dim2/dim3 APIs) must also result in 32Bx2 alignment.

Device API#

Overview#

The cuPVA Device API is used to query core run-time information during VPU execution, and synchronize DataFlows which were configured from Host code. It also provides some helpers to assist users in writing VPU code; for example, configuring AGENs used by the VPU.

Device APIs are generally delivered as source in headers, with only a thin binary runtime. This allows the majority of functions used within an application to be inlined. This is useful in VPU code, where branching to non-local code locations can be costly in some cases.

For full details on available APIs in VPU code, refer to Device API.