Stream

APIs to submit work (cupvaCmdProgram_t) and other commands to the PVA engine.

The Stream APIs are the method by which work is submitted to the PVA engine. Streams also allow submission of generic commands for synchronization with the PVA engine.

Enumerations

cupvaAffinityType_t

Specifies VPU affinity for a Stream.

cupvaEngineType_t

Specifies PVA Engine ID for a Stream.

cupvaL2OperationType_t

L2 operation types for CupvaCmdL2OpsCreate.

cupvaOrderType_t

Specifies scheduling and execution order for the commands submitted to the Stream.

Functions

cupvaError_t CupvaCheckCommandStatus(cupvaCmdStatus_t const status, cupvaError_t *const errorCode)

Check the status of a command.

cupvaError_t CupvaCmdBufferCreate(cupvaCmdBuffer_t *const cmdBuffer, cupvaCmd_t **const cmdList, int32_t const cmdListSize, cupvaCmdStatus_t *const status, cupvaOrderType_t const order)

Create a CmdBuffer object for the given list of commands.

cupvaError_t CupvaCmdBufferDestroy(cupvaCmdBuffer_t const cmdBuffer)

Destroy a CommandBuffer object.

cupvaError_t CupvaCmdDestroy(cupvaCmd_t *const command)

Destroy command.

cupvaError_t CupvaCmdL2OpsCreate(cupvaCmd_t *const cmdL2Ops, cupvaL2OperationType_t const opType, void *const devptr)

Create a command to perform L2 memory coherence operations.

cupvaError_t CupvaCmdPFSDCreate(cupvaCmd_t *const command, uint32_t const idx)

Create a PFSD command.

cupvaError_t CupvaGetPFSDTestCount(uint32_t *const PFSDTestCount)

Get the count of supported PFSD tests.

cupvaError_t CupvaStreamCreate(cupvaStream_t *const stream, cupvaEngineType_t const absEngine, cupvaAffinityType_t const vpuAffinity)

Create a new Stream object.

cupvaError_t CupvaStreamDestroy(cupvaStream_t const stream)

Destroy a stream object.

cupvaError_t CupvaStreamSubmit(cupvaStream_t const stream, cupvaCmd_t const *const *const command, cupvaCmdStatus_t *const status, int32_t const count, cupvaOrderType_t const order, int32_t const executionTimeout, int32_t const submitTimeout)

Submit a batch of Commands to the Stream.

cupvaError_t CupvaStreamSubmitCmdBuffer(cupvaStream_t const stream, cupvaCmdBuffer_t const cmdBuffer, int32_t const executionTimeout, int32_t const submitTimeout)

Submit cupvaCmdBuffer_t to the stream.

Data Structures

cupvaCmd_t

Unit of work which may be submitted to the PVA engine.

Typedefs

cupvaCmdBuffer_t

A container for a set of commands.

cupvaCmdStatus_t

Status output resulting from a submission.

cupvaStream_t

Streams are used to submit PVA commands.

Enumerations

enum cupvaAffinityType_t

Specifies VPU affinity for a Stream.

Values:

enumerator CUPVA_AFFINITY_TYPE_INVALID

Sentinel value

enumerator CUPVA_VPU0

Specifies VPU0 affinity.

enumerator CUPVA_VPU1

Specifies VPU1 affinity.

enumerator CUPVA_VPU_ANY

Specifies VPU ANY affinity. Used to select any available VPU.

enumerator CUPVA_AFFINITY_TYPE_MAX

One greater than max valid value

enum cupvaEngineType_t

Specifies PVA Engine ID for a Stream.

Values:

enumerator CUPVA_ENGINE_TYPE_INVALID

Sentinel value

enumerator CUPVA_PVA0

PVA Engine Id 0

enumerator CUPVA_PVA1

PVA Engine Id 1

enumerator CUPVA_ENGINE_TYPE_MAX

One greater than max valid value

enum cupvaL2OperationType_t

L2 operation types for CupvaCmdL2OpsCreate.

Specifies the operation to be performed on L2SRAM device pointer.

Values:

enumerator CUPVA_L2_OPERATION_FLUSH

flush L2SRAM

enumerator CUPVA_L2_OPERATION_INVALIDATE

invalidate L2SRAM

enumerator CUPVA_L2_OPERATION_FLUSH_AND_INVALIDATE

flush and invalidate L2SRAM

enum cupvaOrderType_t

Specifies scheduling and execution order for the commands submitted to the Stream.

Commands submitted to a Stream can be scheduled and executed by the hardware in two distinct ways. Commands submitted in IN_ORDER mode (default) will begin and complete the execution in the submission order, i.e., the order they appear in the CupvaStreamSubmit() call. Commands submitted in OUT_OF_ORDER mode begin execution in submission order, but may complete out of order. Completion ordering in OUT_OF_ORDER mode can be partially or fully enforced via CmdBarrier command which blocks Stream processing until all preceding (in submission order) commands are completed. This is different from CmdWaitOnFences, which blocks subsequent command processing until specific Fence objects are signalled.

OUT_OF_ORDER mode is useful in Stream instances configured to use VPU_ANY affinity, since multiple VPU engines can be used to process multiple independent commands in parallel.

To maximize the execution performance, it is recommended to terminate the submitted command list with a CmdBarrier instance in the following scenarios:

• Submission has an execution time out specified,

• Submission includes a CmdProgram instance that uses L2SRAM (Orin-specific),

• Stream has CUDA interop enabled.

Values:

enumerator CUPVA_ORDER_TYPE_INVALID

Sentinel value

enumerator CUPVA_IN_ORDER

The execution of the commands in the Stream should be completed in order.

enumerator CUPVA_OUT_OF_ORDER

The execution of the commands in the Stream are allowed to complete out of order.

enumerator CUPVA_ORDER_TYPE_MAX

One greater than max valid value

Functions

cupvaError_t CupvaCheckCommandStatus(

cupvaCmdStatus_t const status,

cupvaError_t *const errorCode,

)

Check the status of a command.

Usage considerations

• Allowed context for the API call

  • Thread-safe: Yes

• API group

  • Init: No

  • Runtime: Yes

  • De-Init: No

Parameters:

• status – [in] The pointer to the command status buffer.

• errorCode – [out] The pointer to return an error code.

Returns:

cupvaError_t The completion status of the operation. Possible values are:

• CUPVA_ERROR_NONE if the operation was successful.

• CUPVA_INVALID_ARGUMENT indicates one of the followings:

  • a NULL status pointer.

  • a NULL errorCode pointer.

• CUPVA_DRIVER_API_ERROR The PVA driver returned an unexpected error.

cupvaError_t CupvaCmdBufferCreate(

cupvaCmdBuffer_t *const cmdBuffer,

cupvaCmd_t **const cmdList,

int32_t const cmdListSize,

cupvaCmdStatus_t *const status,

cupvaOrderType_t const order,

)

Create a CmdBuffer object for the given list of commands.

Usage considerations

• Allowed context for the API call

  • Thread-safe: Yes

• API group

  • Init: Yes

  • Runtime: No

  • De-Init: No

Parameters:

• cmdBuffer – [out] The pointer to return the newly constructed command buffer object.

• cmdList – [in] Array of pointers to commands. Each command pointed to in the array will be consumed and can only be submitted as part of the CmdBuffer. It is still safe to destroy the consumed commands.

• cmdListSize – [in] The number of commands in cmdList.

• status – [in] The pointer to the array of the command status. The array has to be at least as large as the command list. NULL pointer can be passed if command status is not needed.

• order – [in] The command scheduling mode.

Returns:

cupvaError_t The completion status of the operation. Possible values are:

• CUPVA_ERROR_NONE if the operation was successful.

• CUPVA_INVALID_ARGUMENT indicates one of the followings:

  • a NULL cmdBuffer pointer.

  • a NULL cmdList pointer.

• CUPVA_NOT_ALLOWED_IN_OPERATIONAL_STATE if called when NVIDIA DRIVE OS VM state is “Operational”

cupvaError_t CupvaCmdBufferDestroy(cupvaCmdBuffer_t const cmdBuffer)

Destroy a CommandBuffer object.

Usage considerations

• Allowed context for the API call

  • Thread-safe: No

• API group

  • Init: No

  • Runtime: No

  • De-Init: Yes

Parameters:

cmdBuffer – [in] The pointer to a cupvaCmdBuffer_t object.

Returns:

cupvaError_t The completion status of the operation. Possible values are:

• CUPVA_ERROR_NONE if the operation was successful.

• CUPVA_INVALID_ARGUMENT if cmdBuffer was a NULL pointer.

• CUPVA_NOT_ALLOWED_IN_OPERATIONAL_STATE if called when NVIDIA DRIVE OS VM state is “Operational”

cupvaError_t CupvaCmdDestroy(cupvaCmd_t *const command)

Destroy command.

Usage considerations

• Allowed context for the API call

  • Thread-safe: No

• API group

  • Init: No

  • Runtime: No

  • De-Init: Yes

Parameters:

command – [in] The pointer to a cupvaCmd_t object.

Returns:

cupvaError_t The completion status of the operation. Possible values are:

• CUPVA_ERROR_NONE if the operation was successful.

• CUPVA_INVALID_ARGUMENT if command was a NULL pointer.

• CUPVA_DRIVER_API_ERROR The PVA driver returned an unexpected error.

• CUPVA_NOT_ALLOWED_IN_OPERATIONAL_STATE if called to destroy CmdProgram or CmdPFSD when NVIDIA DRIVE OS VM state is “Operational”

cupvaError_t CupvaCmdL2OpsCreate(

cupvaCmd_t *const cmdL2Ops,

cupvaL2OperationType_t const opType,

void *const devptr,

)

Create a command to perform L2 memory coherence operations.

This command can be submitted to a stream to perform operations on L2 memory allocations. It allows control over how L2 memory is managed for coherency between different processing units. Operations include flushing, invalidating, or flushing and invalidating for specified L2SRAM device pointer.

Please refer to cupva::mem::L2OperationType for more details and usage examples.

This API requires driver version >= 2007.

Usage considerations

• Allowed context for the API call

  • Thread-safe: Yes

• API group

  • Init: Yes

  • Runtime: No

  • De-Init: No

Parameters:

• cmdL2Ops – [out] The pointer to a cupvaCmd_t object to fill with the new command.

• opType – [in] The type of L2 operation to perform (FLUSH, INVALIDATE, or FLUSH_AND_INVALIDATE).

• devptr – [in] The device pointer to perform the operation on.

Returns:

cupvaError_t The completion status of the operation. Possible values are:

• CUPVA_ERROR_NONE if the operation was successful.

• CUPVA_INVALID_ARGUMENT if cmdL2Ops was a NULL pointer.

• CUPVA_NOT_ALLOWED_IN_OPERATIONAL_STATE if called when NVIDIA DRIVE OS VM state is “Operational”

cupvaError_t CupvaCmdPFSDCreate(

cupvaCmd_t *const command,

uint32_t const idx,

)

Create a PFSD command.

The PFSD obeys the same scheduling rules as CmdProgram, including respecting barriers, fence waits, fence signals, status buffers etc.

The PFSD application will execute a series of tasks on the hardware and record the result in a CRC checksum. If this checksum does not match the pre-defined expected value, the CupvaCheckCommandStatus() will return CUPVA_PSFD_CHECK_MISMATCH.

CupvaCheckCommandStatus() will return CUPVA_PSFD_TIMEOUT error if the execution of the PFSD task is not completed within the timeout duration. PFSD task execution timeout duration is set internally and different from the the command(s) execution timeout, CupvaStreamSubmit() executionTimeout parameter. User may still set the executionTimeout value while submitting PFSD command(s) to a stream.

Usage considerations

• Allowed context for the API call

  • Thread-safe: Yes

• API group

  • Init: Yes

  • Runtime: No

  • De-Init: No

Parameters:

• command – [in] The pointer to a cupvaCmd_t object.

• idx – [in] Zero-based integer specifying the PFSD index. Refer to platform documentation for supported PFSD indices and details.

Returns:

cupvaError_t The completion status of the operation. Possible values are:

• CUPVA_ERROR_NONE if the operation was successful.

• CUPVA_INVALID_ARGUMENT indicates:

  • command was a NULL pointer.

  • idx is not a supported PFSD test index

• CUPVA_DRIVER_API_ERROR if driver returned error due to one of the following:

  • Internal driver memory allocation failed.

  • PVA engine was in a bad state.

  • The PVA hardware does not support PFSD.

  • The operating system does not support PFSD

• CUPVA_NOT_ALLOWED_IN_OPERATIONAL_STATE if called when NVIDIA DRIVE OS VM state is “Operational”

cupvaError_t CupvaGetPFSDTestCount(uint32_t *const PFSDTestCount)

Get the count of supported PFSD tests.

Usage considerations

• Allowed context for the API call

  • Thread-safe: Yes

• API group

  • Init: Yes

  • Runtime: No

  • De-Init: No

Parameters:

PFSDTestCount – [in] a uint32_t pointer to return the count of supported PFSD tests.

Returns:

cupvaError_t The completion status of the operation. Possible values are:

• CUPVA_ERROR_NONE if the operation was successful.

• CUPVA_INVALID_ARGUMENT if PFSDTestCount was a NULL pointer.

• CUPVA_NOT_ALLOWED_IN_OPERATIONAL_STATE if current context is null and default context does not exist and NVIDIA DRIVE OS VM state is “Operational”

cupvaError_t CupvaStreamCreate(

cupvaStream_t *const stream,

cupvaEngineType_t const absEngine,

cupvaAffinityType_t const vpuAffinity,

)

Create a new Stream object.

Usage considerations

• Allowed context for the API call

  • Thread-safe: No

• API group

  • Init: Yes

  • Runtime: No

  • De-Init: No

Parameters:

• stream – [out] The pointer to return the the newly constructed Stream object.

• absEngine – [in] The PVA engine Id.

• vpuAffinity – [in] The queue submission affinity.

Returns:

cupvaError_t The completion status of the operation. Possible values are:

• CUPVA_ERROR_NONE if the operation was successful.

• CUPVA_INVALID_ARGUMENT indicates one of the followings:

  • stream is a NULL pointer.

  • vpuAffinity was invalid.

• CUPVA_DRIVER_API_ERROR The PVA driver returned an unexpected error.

• CUPVA_NOT_ALLOWED_IN_OPERATIONAL_STATE if called when NVIDIA DRIVE OS VM state is “Operational”

cupvaError_t CupvaStreamDestroy(cupvaStream_t const stream)

Destroy a stream object.

Usage considerations

• Allowed context for the API call

  • Thread-safe: No

• API group

  • Init: No

  • Runtime: No

  • De-Init: Yes

Parameters:

stream – [in] The pointer to an Stream Object.

Returns:

cupvaError_t The completion status of the operation. Possible values are:

• CUPVA_ERROR_NONE if the operation was successful.

• CUPVA_DRIVER_API_ERROR if driver returned error due to one of the following:

  • Releasing symbols previously obtained via CupvaCmdProgramGetParameter

  • Destroying and de-allocating driver resources failed

cupvaError_t CupvaStreamSubmit(

cupvaStream_t const stream,

cupvaCmd_t const *const *const command,

cupvaCmdStatus_t *const status,

int32_t const count,

cupvaOrderType_t const order,

int32_t const executionTimeout,

int32_t const submitTimeout,

)

Submit a batch of Commands to the Stream.

The first PVA task of each instance will likely incur a PVA engine power-up delay (Linux/L4T specific). To avoid CUPVA_COMMAND_SUBMISSION_TIMEOUT error when calling submit with a small submitTimeout, the user needs to submit a single fence request command and wait on it by calling fence.wait() at beginning of PVA application.

Usage considerations

• Allowed context for the API call

  • Thread-safe: Yes

• API group

  • Init: No

  • Runtime: Yes

  • De-Init: No

Parameters:

• stream – [in] The pointer to an Stream Object.

• command – [in] The pointer to the array of the commands.

• status – [in] The pointer to the array of the command status. Size of this array should match the size of the command array. Each index in the command status array will give the status for the corresponding index in the commands array.

• count – [in] The number of commands to be submitted.

• order – [in] The command scheduling mode.

• executionTimeout – [in] The timeout (in usec) for the execution of the commands. If the entire batch doesn’t finish in the specified time, the hardware will abort any currently running command and flush the remaining ones from current batch. The abort status is communicated via command status. A negative timeout value indicates commands are allowed to run indefinitely.

• submitTimeout – [in] The maximum time (in usec) the calling thread will be blocked on CupvaStreamSubmit() call. A negative timeout value indicates calling thread is blocked until the submission succeeds.

Returns:

cupvaError_t The completion status of the operation. Possible values are:

• CUPVA_ERROR_NONE if the operation was successful.

• CUPVA_INVALID_ARGUMENT indicates one of the followings:

  • stream was a NULL pointer.

  • command was a NULL pointer.

  • The number of commands exceeded the maximum.

  • An invalid command program was provided.

• CUPVA_COMMAND_SUBMISSION_TIMEOUT The submitTimeout value was reached before commands could be submitted.

• CUPVA_DRIVER_API_ERROR he PVA driver returned an unexpected error.

cupvaError_t CupvaStreamSubmitCmdBuffer(

cupvaStream_t const stream,

cupvaCmdBuffer_t const cmdBuffer,

int32_t const executionTimeout,

int32_t const submitTimeout,

)

Submit cupvaCmdBuffer_t to the stream.

The first PVA task of each instance will likely incur a PVA engine power-up delay (Linux/L4T specific). To avoid CUPVA_COMMAND_SUBMISSION_TIMEOUT error when calling submit with a small submitTimeout, the user needs to submit a single fence request command and wait on it by calling fence.wait() at beginning of PVA application.

Usage considerations

• Allowed context for the API call

  • Thread-safe: Yes

• API group

  • Init: No

  • Runtime: Yes

  • De-Init: No

Parameters:

• stream – [in] The pointer to an Stream Object.

• cmdBuffer – [in] The pointer to the cupvaCmdBuffer_t to be submitted.

• executionTimeout – [in] The timeout (in usec) for the execution of the command. If the command doesn’t finish in the specified time it gets aborted. The abort is communicated via command status. By default commands are allowed to run indefinitely.

• submitTimeout – [in] The maximum time (in usec) the calling thread will be blocked on CupvaStreamSubmitCmdBuffer() call. By default the calling thread is blocked until the submission succeeds.

Returns:

cupvaError_t The completion status of the operation. Possible values are:

• CUPVA_ERROR_NONE if the operation was successful.

• CUPVA_INVALID_ARGUMENT indicates one of the followings:

  • stream was a NULL pointer.

  • cmdBuffer was a NULL pointer.

  • An invalid command program was provided.

• CUPVA_COMMAND_SUBMISSION_TIMEOUT The submitTimeout value was reached before commands could be submitted.

• CUPVA_DRIVER_API_ERROR he PVA driver returned an unexpected error.

Typedefs

typedef struct cupvaCmdBufferRec *cupvaCmdBuffer_t

A container for a set of commands.

typedef struct cupvaCmdStatusRec *cupvaCmdStatus_t

Status output resulting from a submission.

typedef struct cupvaStreamRec *cupvaStream_t

Streams are used to submit PVA commands.