CUDA Interoperability

APIs for interoperability with CUDA.

Functions

cupvaError_t CupvaCudaStreamSetName(cudaStream_t const cudaStream, char const *const name)

Attach a diagnostic name to a CUDA stream.

bool cupva::cuda::ContextSupportsCuda(cupva::impl::Context const *const ctx=cupva::Context::GetCurrent()) noexcept

Check if CUDA interop is enabled.

Stream cupva::cuda::CreateStream(cudaStream_t const cudaStream, EngineType const absEngine=PVA0, AffinityType const vpuAffinity=VPU_ANY)

Create a PVA Stream object from a CUDA stream handle.

void cupva::cuda::ForceDisableSupport()

Disable CUDA interop support.

void cupva::cuda::SetAffinity(cudaStream_t const cudaStream, AffinityType const vpuAffinity)

Set the VPU affinity for a CUDA stream.

void cupva::cuda::SetName(cudaStream_t const cudaStream, char_t const *const name)

Attach a diagnostic name to a CUDA stream.

void cupva::cuda::Submit(cudaStream_t const cudaStream, Cmd const *const commands, CmdStatus *const status=nullptr, int32_t const count=1, OrderType const order=IN_ORDER, int32_t const executionTimeout=-1, int32_t const submitTimeout=-1)

Submit a batch of Commands to a CUDA Stream .

void cupva::cuda::Submit(cudaStream_t const cudaStream, const BaseCmd &command, CmdStatus *const status=nullptr, OrderType const order=IN_ORDER, int32_t const executionTimeout=-1, int32_t const submitTimeout=-1)

Submit one command to a CUDA Stream .

void cupva::cuda::Submit(cudaStream_t const cudaStream, CmdBuffer const &cmdBuf, int32_t const executionTimeout=-1, int32_t const submitTimeout=-1)

Submit a CmdBuffer to a CUDA Stream .

void cupva::cuda::Submit(cudaStream_t const cudaStream, const std::initializer_list< Cmd > &cmdList, CmdStatus *const status=nullptr, OrderType const order=IN_ORDER, int32_t const executionTimeout=-1, int32_t const submitTimeout=-1)

Submit a command list to a CUDA Stream .

void * cupva::cuda::mem::Import(void *const cudaDevPtr, int64_t const size, cupva::mem::AccessType const access=cupva::mem::AccessType::READ_WRITE)

Create a CUPVA pointer from a CUDA device pointer.

Functions

cupvaError_t CupvaCudaStreamSetName(

cudaStream_t const cudaStream,

char const *const name,

)

Attach a diagnostic name to a CUDA stream.

Equivalent to CupvaStreamSetName, but for a CUDA stream.

Usage considerations

• Allowed context for the API call

  • Interrupt handler: No

  • Signal handler: No

  • Thread-safe: No

  • Re-entrant: No

  • Async/Sync: Sync

• Required privileges: None

• Allowed execution state

  • Initialization: Yes

  • Operational: Yes

  • Reinit/Deinit Preparation: Yes

• Maturity: GA

Note

The same CUDA context requirements as CupvaCudaStreamSetAffinity() apply.

Parameters:

• cudaStream – [in] The CUDA stream handle.

• name – [in] Null terminated name to record.

Returns:

CUPVA_ERROR_NONE on success.

Returns:

CUPVA_INVALID_ARGUMENT if the CUDA stream is invalid or name is NULL.

Returns:

CUPVA_UNSUPPORTED_FEATURE if CUDA interoperability is not enabled in the driver.

Pre:

None

bool cupva::cuda::ContextSupportsCuda(

cupva::impl::Context const *const ctx = cupva::Context::GetCurrent(),

) noexcept

Check if CUDA interop is enabled.

Returns whether the current process has disabled CUDA using ForceDisableSupport().

Usage considerations

• Allowed context for the API call

  • Interrupt handler: No

  • Signal handler: No

  • Thread-safe: No

  • Re-entrant: No

  • Async/Sync: Sync

• Required privileges: None

• Allowed execution state

  • Initialization: Yes

  • Operational: Yes

  • Reinit/Deinit Preparation: Yes

• Maturity: GA

Parameters:

ctx – [in] pointer to a cupva::impl::Context handle. This parameter is deprecated and will be ignored.

Returns:

true if CUDA has not been disabled for the current process, false otherwise.

Pre:

None

Stream cupva::cuda::CreateStream(

cudaStream_t const cudaStream,

EngineType const absEngine = PVA0,

AffinityType const vpuAffinity = VPU_ANY,

)

Create a PVA Stream object from a CUDA stream handle.

Deprecated:

This function is deprecated. Use cupva::cuda::Submit instead.

The Stream and the cudaStream can be used to submit PVA and CUDA commands respectively, and commands will begin execution in the order they are submitted as if they share the same stream. More specifically, PVA commands will execute following the OrderType specified by the Stream::submit call, while CUDA commands will execute per the configuration of the cudaStream. Synchronization is possible using CUDA APIs (for example, cudaStreamSynchronize() cudaStreamWaitEvent() etc.), and/or Fence APIs (CmdWaitOnFences, CmdRequestFences, etc.).

The special concept of “CUDA default stream” (cudaStream value of 0) is not supported and the CUDA stream handle must have been explicitly created as a “non blocking” stream (see cudaStreamCreateWithFlags() and cuStreamCreate() in the CUDA API documentation).

Usage considerations

• Allowed context for the API call

  • Interrupt handler: No

  • Signal handler: No

  • Thread-safe: No

  • Re-entrant: No

  • Async/Sync: Sync

• Required privileges: None

• Allowed execution state

  • Initialization: Yes

  • Operational: No

  • Reinit/Deinit Preparation: Yes

• Maturity: GA

Note

The current PVA context must have been created after a CUDA context was loaded. Any CUDA Runtime API call causes a CUDA context to be loaded, as does a CUDA Driver API call to cuCtxCreate() (see the CUDA documentation for details). The CUDA context must also be current on the host thread that calls this API. If this API is called from a worker thread, a task launched with std::async, or a thread-pool task, first call a CUDA Runtime or Driver API on that thread, for example cudaFree(nullptr), to make the CUDA context current.

Parameters:

• cudaStream – [in] The CUDA stream handle.

• absEngine – [in] The PVA engine ID.

• vpuAffinity – [in] The queue submission affinity.

Throws:

• std::bad_alloc – Allocating the Stream object failed.

• cupva::Exception(InvalidArgument) – The cudaStream value is 0.

• cupva::Exception(InvalidArgument) – The AffinityType is invalid.

• cupva::Exception(DriverAPIError) – The PVA driver returned an unexpected error.

• cupva::Exception(NotAllowedInOperationalState) – if called when NVIDIA DRIVE OS VM state is “Operational”

• cupva::Exception(CUDADisabled) – if called after disabling CUDA interop for this process

Pre:

None

void cupva::cuda::ForceDisableSupport()

Disable CUDA interop support.

Used to disable CUDA on a process-wide basis.

During creation of the first cupva::Context for a process, the CUDA driver library will be loaded.

In certain safety applications, it is desirable for the process to be fully isolated from any dependencies on CUDA. Calling this API will ensure that future calls which require CUDA will fail before loading the CUDA driver or performing any CUDA API calls.

This API will fail with an exception if the process has previously made calls which required the CUDA driver to be loaded. Failure is atomic - if this API succeeds, than user is guaranteed that this process has never, and will never, make successful CUDA interop calls.

To ensure that the calling process has no dependency on CUDA, this API should be called prior to any other CUPVA APIs.

Usage considerations

• Allowed context for the API call

  • Interrupt handler: No

  • Signal handler: No

  • Thread-safe: Yes

  • Re-entrant: No

  • Async/Sync: Sync

• Required privileges: None

• Allowed execution state

  • Initialization: Yes

  • Operational: No

  • Reinit/Deinit Preparation: Yes

• Maturity: GA

Throws:

cupva::Exception(InvalidState) – If this process has previously called CUDA interop APIs

Pre:

None

void cupva::cuda::SetAffinity(

cudaStream_t const cudaStream,

AffinityType const vpuAffinity,

)

Set the VPU affinity for a CUDA stream.

By default, submissions to CUDA streams can be scheduled on any VPU. This function changes that behavior by setting the VPU affinity on a per-stream basis.

Usage considerations

• Allowed context for the API call

  • Interrupt handler: No

  • Signal handler: No

  • Thread-safe: No

  • Re-entrant: No

  • Async/Sync: Sync

• Required privileges: None

• Allowed execution state

  • Initialization: Yes

  • Operational: Yes

  • Reinit/Deinit Preparation: Yes

• Maturity: GA

Note

The current PVA context must have been created after a CUDA context was loaded. Any CUDA Runtime API call causes a CUDA context to be loaded, as does a CUDA Driver API call to cuCtxCreate() (see the CUDA documentation for details). The CUDA context must also be current on the host thread that calls this API. If this API is called from a worker thread, a task launched with std::async, or a thread-pool task, first call a CUDA Runtime or Driver API on that thread, for example cudaFree(nullptr), to make the CUDA context current.

Parameters:

• cudaStream – [in] The CUDA stream handle.

• vpuAffinity – [in] The VPU affinity to set for the stream.

Throws:

• cupva::Exception(InvalidArgument) – The CUDA stream is invalid.

• cupva::Exception(InvalidArgument) – The vpuAffinity is invalid.

• cupva::Exception(UnsupportedFeature) – The CUDA interoperability is not enabled in the driver.

Pre:

None

void cupva::cuda::SetName(

cudaStream_t const cudaStream,

char_t const *const name,

)

Attach a diagnostic name to a CUDA stream.

Equivalent to cupva::Stream::setName(), but for a CUDA stream.

Usage considerations

• Allowed context for the API call

  • Interrupt handler: No

  • Signal handler: No

  • Thread-safe: No

  • Re-entrant: No

  • Async/Sync: Sync

• Required privileges: None

• Allowed execution state

  • Initialization: Yes

  • Operational: Yes

  • Reinit/Deinit Preparation: Yes

• Maturity: GA

Note

The same CUDA context requirements as cupva::cuda::SetAffinity() apply.

Parameters:

• cudaStream – [in] The CUDA stream handle.

• name – [in] Null terminated name to record.

Throws:

• cupva::Exception(InvalidArgument) – The CUDA stream is invalid, or name is nullptr.

• cupva::Exception(UnsupportedFeature) – The CUDA interoperability is not enabled in the driver.

Pre:

None

void cupva::cuda::Submit(

cudaStream_t const cudaStream,

Cmd const *const commands,

CmdStatus *const status = nullptr,

int32_t const count = 1,

OrderType const order = IN_ORDER,

int32_t const executionTimeout = -1,

int32_t const submitTimeout = -1,

)

Submit a batch of Commands to a CUDA Stream.

This function is used to submit a batch of Commands to a CUDA Stream. It requires a PVA driver version >= 2007 and it is not supported in Native mode.

Usage considerations

• Allowed context for the API call

  • Interrupt handler: No

  • Signal handler: No

  • Thread-safe: Yes

  • Re-entrant: No

  • Async/Sync: Sync

• Required privileges: None

• Allowed execution state

  • Initialization: Yes

  • Operational: Yes

  • Reinit/Deinit Preparation: Yes

• Maturity: GA

Note

The current PVA context must have been created after a CUDA context was loaded. Any CUDA Runtime API call causes a CUDA context to be loaded, as does a CUDA Driver API call to cuCtxCreate() (see the CUDA documentation for details). The CUDA context must also be current on the host thread that calls this API. If this API is called from a worker thread, a task launched with std::async, or a thread-pool task, first call a CUDA Runtime or Driver API on that thread, for example cudaFree(nullptr), to make the CUDA context current.

Parameters:

• cudaStream – [in] The CUDA stream handle.

• commands – [in] The commands to submit.

• status – [in] The status of the commands.

• count – [in] The number of commands.

• order – [in] The order of the commands.

• executionTimeout – [in] The timeout for the commands.

• submitTimeout – [in] The timeout for the submission.

Throws:

• cupva::Exception(InvalidArgument) – The CUDA stream is invalid.

• cupva::Exception(InvalidArgument) – The commands are invalid.

• cupva::Exception(InvalidArgument) – The program count is invalid.

• cupva::Exception(UnsupportedFeature) – The CUDA interoperability is not enabled in the driver.

Pre:

None

void cupva::cuda::Submit(

cudaStream_t const cudaStream,

const BaseCmd &command,

CmdStatus *const status = nullptr,

OrderType const order = IN_ORDER,

int32_t const executionTimeout = -1,

int32_t const submitTimeout = -1,

)

Submit one command to a CUDA Stream.

This function is used to submit one command to a CUDA Stream. It requires a PVA driver version >= 2007 and it is not supported in Native mode.

Usage considerations

• Allowed context for the API call

  • Interrupt handler: No

  • Signal handler: No

  • Thread-safe: Yes

  • Re-entrant: No

  • Async/Sync: Sync

• Required privileges: None

• Allowed execution state

  • Initialization: Yes

  • Operational: Yes

  • Reinit/Deinit Preparation: Yes

• Maturity: GA

Note

The current PVA context must have been created after a CUDA context was loaded. Any CUDA Runtime API call causes a CUDA context to be loaded, as does a CUDA Driver API call to cuCtxCreate() (see the CUDA documentation for details). The CUDA context must also be current on the host thread that calls this API. If this API is called from a worker thread, a task launched with std::async, or a thread-pool task, first call a CUDA Runtime or Driver API on that thread, for example cudaFree(nullptr), to make the CUDA context current.

Parameters:

• cudaStream – [in] The CUDA stream handle.

• command – [in] The command to submit.

• status – [in] The status of the command.

• order – [in] The order of the command.

• executionTimeout – [in] The timeout for the command.

• submitTimeout – [in] The timeout for the submission.

Throws:

• cupva::Exception(InvalidArgument) – The CUDA stream is invalid.

• cupva::Exception(UnsupportedFeature) – The CUDA interoperability is not enabled in the driver.

Pre:

None

void cupva::cuda::Submit(

cudaStream_t const cudaStream,

CmdBuffer const &cmdBuf,

int32_t const executionTimeout = -1,

int32_t const submitTimeout = -1,

)

Submit a CmdBuffer to a CUDA Stream.

This function is used to submit a CmdBuffer to a CUDA Stream. It requires a PVA driver version >= 2007 and it is not supported in Native mode.

Usage considerations

• Allowed context for the API call

  • Interrupt handler: No

  • Signal handler: No

  • Thread-safe: Yes

  • Re-entrant: No

  • Async/Sync: Sync

• Required privileges: None

• Allowed execution state

  • Initialization: Yes

  • Operational: Yes

  • Reinit/Deinit Preparation: Yes

• Maturity: GA

Note

The current PVA context must have been created after a CUDA context was loaded. Any CUDA Runtime API call causes a CUDA context to be loaded, as does a CUDA Driver API call to cuCtxCreate() (see the CUDA documentation for details). The CUDA context must also be current on the host thread that calls this API. If this API is called from a worker thread, a task launched with std::async, or a thread-pool task, first call a CUDA Runtime or Driver API on that thread, for example cudaFree(nullptr), to make the CUDA context current.

Parameters:

• cudaStream – [in] The CUDA stream handle.

• cmdBuf – [in] The command buffer to submit.

• executionTimeout – [in] The timeout for the commands.

• submitTimeout – [in] The timeout for the submission.

Throws:

• cupva::Exception(InvalidArgument) – The CUDA stream is invalid.

• cupva::Exception(UnsupportedFeature) – The CUDA interoperability is not enabled in the driver.

Pre:

None

void cupva::cuda::Submit(

cudaStream_t const cudaStream,

const std::initializer_list<Cmd> &cmdList,

CmdStatus *const status = nullptr,

OrderType const order = IN_ORDER,

int32_t const executionTimeout = -1,

int32_t const submitTimeout = -1,

)

Submit a command list to a CUDA Stream.

This function is used to submit a command list to a CUDA Stream. It requires a PVA driver version >= 2007 and it is not supported in Native mode.

Usage considerations

• Allowed context for the API call

  • Interrupt handler: No

  • Signal handler: No

  • Thread-safe: Yes

  • Re-entrant: No

  • Async/Sync: Sync

• Required privileges: None

• Allowed execution state

  • Initialization: Yes

  • Operational: Yes

  • Reinit/Deinit Preparation: Yes

• Maturity: GA

Note

The current PVA context must have been created after a CUDA context was loaded. Any CUDA Runtime API call causes a CUDA context to be loaded, as does a CUDA Driver API call to cuCtxCreate() (see the CUDA documentation for details). The CUDA context must also be current on the host thread that calls this API. If this API is called from a worker thread, a task launched with std::async, or a thread-pool task, first call a CUDA Runtime or Driver API on that thread, for example cudaFree(nullptr), to make the CUDA context current.

Parameters:

• cudaStream – [in] The CUDA stream handle.

• cmdList – [in] The command list to submit.

• status – [in] The status of the command list.

• order – [in] The order of the command list.

• executionTimeout – [in] The timeout for the command list.

• submitTimeout – [in] The timeout for the submission.

Throws:

• cupva::Exception(InvalidArgument) – The CUDA stream is invalid.

• cupva::Exception(InvalidArgument) – The cmdList is invalid.

• cupva::Exception(UnsupportedFeature) – The CUDA interoperability is not enabled in the driver.

Pre:

None

void *cupva::cuda::mem::Import(

void *const cudaDevPtr,

int64_t const size,

cupva::mem::AccessType const access = cupva::mem::AccessType::READ_WRITE,

)

Create a CUPVA pointer from a CUDA device pointer.

Deprecated:

This function is deprecated. CUDA device pointers no longer need to be imported explicitly.

Importing a CUDA device pointer creates a mapping which must be released by calling cupva::mem::Free() when the application has finished using the buffer. The CUDA device pointer must be at least 64B aligned.

Usage considerations

• Allowed context for the API call

  • Interrupt handler: No

  • Signal handler: No

  • Thread-safe: No

  • Re-entrant: No

  • Async/Sync: Sync

• Required privileges: None

• Allowed execution state

  • Initialization: Yes

  • Operational: No

  • Reinit/Deinit Preparation: Yes

• Maturity: GA

Note

The current PVA context must have been created after a CUDA context was loaded. Any CUDA Runtime API call causes a CUDA context to be loaded, as does a CUDA Driver API call to cuCtxCreate() (see the CUDA documentation for details). The CUDA context must also be current on the host thread that calls this API. If this API is called from a worker thread, a task launched with std::async, or a thread-pool task, first call a CUDA Runtime or Driver API on that thread, for example cudaFree(nullptr), to make the CUDA context current.

Parameters:

• cudaDevPtr – [in] The CUDA device pointer returned by a CUDA API (cudaMalloc, for example)

• size – [in] The size of the allocation in bytes

• access – [in] Specifies the access type of the mapped memory.

Throws:

• cupva::Exception(InvalidArgument) – PVA needs 64B aligned addresses and CUDA device pointer is not aligned on 64B.

• cupva::Exception(InternalError) – VA allocation failed.

• cupva::Exception(DriverAPIError) – Getting a device pointer failed due to driver API error.

• cupva::Exception(InvalidArgument) – The size is invalid.

• cupva::Exception(NotAllowedInOperationalState) – if called when NVIDIA DRIVE OS VM state is “Operational”

• cupva::Exception(CUDADisabled) – if called after disabling CUDA interop for this process

Returns:

The CUPVA device pointer pointing to the imported memory.

Pre:

None