Stream#

Fully qualified name: cupva::Stream

Defined in src/host/cpp_api/include/cupva_host.hpp

class Stream : public cupva::DynamicStorage<impl::Stream>#

The Stream is used to submit PVA commands.

Stream submit methods supports batches of commands, up to a maximum of MAX_NUM_COMMANDS_PER_SUBMIT. When submitting multiple commands, user should prefer to use batch submission as it can reduce total submission overheads.

The first PVA task of each instance will likely incur a PVA engine power-up delay (Linux/L4T specific). To avoid CommandSubmissionTimeout error when calling submit with a small submitTimeout, the user needs to submit a single fence request command and wait on it by calling fence.wait() at beginning of PVA application.

Public Functions

Stream() noexcept#

Creates a new Stream object.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: No

  • API group

    • Init: Yes

    • Runtime: No

    • De-Init: No

Stream(Stream &&obj) noexcept#

Stream move constructor.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: No

  • API group

    • Init: No

    • Runtime: Yes

    • De-Init: No

Stream &operator=(Stream &&obj) & noexcept#

Stream move assignment.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: No

  • API group

    • Init: No

    • Runtime: Yes

    • De-Init: No

Stream(Stream const&) = delete#
Stream &operator=(Stream const&) & = delete#
void finalize()#

Destroy the resources created by the object.

This method is exposed to allow fine-grained control over error handling.

During destruction of the object, this method will be called but the destructor must not propagate exceptions. To handle exceptions, manually invoke this method prior to object destruction.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: No

  • API group

    • Init: No

    • Runtime: No

    • De-Init: Yes

Throws:

  • cupva::Exception(DriverAPIError) – The PVA driver returned an unexpected error.

  • cupva::Exception(NotAllowedInOperationalState) – if called when NVIDIA DRIVE OS VM state is “Operational”

~Stream() noexcept#

Destroy the Stream object.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: No

  • API group

    • Init: No

    • Runtime: No

    • De-Init: Yes

void submit(
Cmd const *const commands,
CmdStatus *const status = nullptr,
int32_t const count = 1,
OrderType const order = IN_ORDER,
int32_t const executionTimeout = -1,
int32_t const submitTimeout = -1,
)#

Submit a batch of Commands to the Stream.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: Yes

  • API group

    • Init: No

    • Runtime: Yes

    • De-Init: No

Parameters:

  • commands – The pointer to the array of the commands.

  • status – The pointer to the array of the command status. Size of this array should match the size of the command array. Each index in the command status array will give the status for the corresponding index in the commands array.

  • count – The number of commands to be submitted.

  • order – The command scheduling mode.

  • executionTimeout – The timeout (in usec) for the execution of the commands. If the entire batch doesn’t finish in the specified time, the hardware will abort any currently running command and flush the remaining ones from current batch. The abort status is communicated via command status. By default the batch is allowed to run indefinitely. In the presence of system contention, it is not guaranteed that all CmdPrograms in a batch will be executed consecutively. CmdPrograms from other clients may be scheduled between CmdPrograms in the submitted batch. If the user sets an execution timeout, they must ensure that it is sufficient to allow for execution of the batch of submitted CmdPrograms, and must also consider any other workloads submitted by other PVA clients in determining this timeout. The execution timer starts after any CmdWaitOnFences at the start of the submission are signaled. If these fences are never signalled, executionTimeout will never cause abort.

  • submitTimeout – The maximum time (in usec) the calling thread will be blocked on submit() call. By default the calling thread is blocked until the submission succeeds. Submissions may be blocked if the PVA engine has too many tasks pending. It is recommended to set a finite submission timeout value and ensure that submissions cannot be blocked.

Throws:

  • cupva::Exception(InvalidArgument) – The number of commands exceeds the maximum.

  • cupva::Exception(InvalidArgument) – An invalid CmdProgram was provided.

  • cupva::Exception(CommandSubmissionTimeout) – The submitTimeout value was reached before commands could be submitted.

  • cupva::Exception(DriverAPIError) – The PVA driver returned an unexpected error.

void submit(
CmdBuffer const &cmdBuf,
int32_t const executionTimeout = -1,
int32_t const submitTimeout = -1,
)#

Submit a CmdBuffer.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: Yes

  • API group

    • Init: No

    • Runtime: Yes

    • De-Init: No

Parameters:

  • cmdBuf – The reference to the CmdBuffer to be submitted.

  • executionTimeout – The timeout (in usec) for the execution of the command. If the CmdBuffer doesn’t finish in the specified time it gets aborted. The abort is communicated via command status. By default commands are allowed to run indefinitely. The execution timer starts after any CmdWaitOnFences at the start of the CmdBuffer are signaled. If these fences are never signalled, executionTimeout will never cause abort.

  • submitTimeout – The maximum time (in usec) the calling thread will be blocked on submit() call. By default the calling thread is blocked until the submission succeeds. Submissions may be blocked if the PVA engine has too many tasks pending. It is recommended to design applications in a way that ensures that only MAX_NUM_COMMANDS_PER_SUBMIT are active on the PVA at any one time. This will ensure that submit calls are not blocked on PVA completion and can be enforced by setting a finite submission timeout value.

Throws:

  • cupva::Exception(InvalidArgument) – The number of commands exceeds the maximum.

  • cupva::Exception(InvalidArgument) – An invalid CmdProgram was provided.

  • cupva::Exception(CommandSubmissionTimeout) – The submitTimeout value was reached before the command could be submitted.

  • cupva::Exception(DriverAPIError) – The PVA driver returned an unexpected error.

inline void submit(
const BaseCmd &command,
CmdStatus *const status = nullptr,
OrderType const order = IN_ORDER,
int32_t const executionTimeout = -1,
int32_t const submitTimeout = -1,
)#

Submit one command to the Stream.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: Yes

  • API group

    • Init: No

    • Runtime: Yes

    • De-Init: No

Parameters:

  • command – The command to be submitted.

  • status – The pointer to command status. Can be queried by the application to check the completion status.

  • order – The command scheduling mode.

  • executionTimeout – The timeout (in usec) for the execution of the command. If the command doesn’t finish in the specified time it gets aborted. The abort is communicated via command status. By default commands are allowed to run indefinitely.

  • submitTimeout – The maximum time (in usec) the calling thread will be blocked on submit() call. By default the calling thread is blocked until the submission succeeds. Submissions may be blocked if the PVA engine has too many tasks pending. It is recommended to design applications in a way that ensures that only MAX_NUM_COMMANDS_PER_SUBMIT are active on the PVA at any one time. This will ensure that submit calls are not blocked on PVA completion and can be enforced by setting a finite submission timeout value.

Throws:

  • cupva::Exception(InvalidArgument) – The number of commands exceeds the maximum.

  • cupva::Exception(InvalidArgument) – An invalid CmdProgram was provided.

  • cupva::Exception(CommandSubmissionTimeout) – The submitTimeout value was reached before the command could be submitted.

  • cupva::Exception(DriverAPIError) – The PVA driver returned an unexpected error.

void submit(
const std::initializer_list<Cmd> &cmdList,
CmdStatus *const status = nullptr,
OrderType const order = IN_ORDER,
int32_t const executionTimeout = -1,
int32_t const submitTimeout = -1,
)#

Submit a command list to the Stream.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: Yes

  • API group

    • Init: No

    • Runtime: Yes

    • De-Init: No

Parameters:

  • cmdList – The command list to be submitted.

  • status – The pointer to the array of the command status. Size of this array should match the size of the command initializer list. Each index in the command status array will give the status for the corresponding index in the command initializer list.

  • order – The command scheduling mode.

  • executionTimeout – The timeout (in usec) for the execution of the command list. If the entire command list doesn’t finish in the specified time, the hardware will abort any currently running command and flush the remaining ones from current list. The abort status is communicated via command status. By default the commands are allowed to run indefinitely.

  • submitTimeout – The maximum time (in usec) the calling thread will be blocked on submit() call. By default the calling thread is blocked until the submission succeeds.

Throws:

  • cupva::Exception(InvalidArgument) – The number of commands exceeds the maximum.

  • cupva::Exception(InvalidArgument) – An invalid CmdProgram was provided.

  • cupva::Exception(CommandSubmissionTimeout) – The submitTimeout value was reached before commands could be submitted.

  • cupva::Exception(DriverAPIError) – The PVA driver returned an unexpected error.

Public Static Functions

static Stream Create(
EngineType absEngine = PVA0,
AffinityType vpuAffinity = VPU_ANY,
)#

Construct a new Stream object.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: Yes

  • API group

    • Init: Yes

    • Runtime: No

    • De-Init: No

Parameters:

  • absEngine – The PVA engine Id.

  • vpuAffinity – The queue submission affinity. If this is not VPU_ANY, submissions to this Stream will be limited to execution on a single VPU. Generally, the best hardware utilization is achieved by setting affinity to VPU_ANY and allowing the PVA software to dynamically schedule work on available VPUs. In scenarios where contention between different submissions must be avoided, the user should affine all Streams to specific VPUs.

Throws:

  • std::bad_alloc – Allocating the Stream object failed.

  • cupva::Exception(InvalidArgument) – The AffinityType is invalid.

  • cupva::Exception(DriverAPIError) – The PVA driver returned an unexpected error.

  • cupva::Exception(NotAllowedInOperationalState) – if called when NVIDIA DRIVE OS VM state is “Operational”