CUDA Interoperability#

APIs for interoperability with CUDA.

Functions#

cupvaError_t CupvaCudaContextSupportsCuda(cupvaContext_t const ctx, bool *const cudaFlag)

Check if CUDA interop is enabled.

cupvaError_t CupvaCudaCreateStream(cupvaStream_t *const stream, cudaStream_t const cudaStream, cupvaEngineType_t const absEngine, cupvaAffinityType_t const vpuAffinity)

Create a PVA Stream object from a CUDA stream handle.

cupvaError_t CupvaCudaForceDisableSupport(void)

Disable CUDA interop support.

cupvaError_t CupvaCudaMemImport(void **const devPtr, void *const cudaDevPtr, int64_t const size, cupvaMemAccessType_t const accessType)

Create a CUPVA pointer from a CUDA device pointer.

cupvaError_t CupvaCudaStreamSetAffinity(cudaStream_t const cudaStream, cupvaAffinityType_t const vpuAffinity)

Set the VPU affinity for a CUDA stream.

cupvaError_t CupvaCudaStreamSubmit(cudaStream_t const cudaStream, cupvaCmd_t const *const *const command, cupvaCmdStatus_t *const status, int32_t const count, cupvaOrderType_t const order, int32_t const executionTimeout, int32_t const submitTimeout)

Submit a command list to a CUDA stream.

cupvaError_t CupvaCudaStreamSubmitCmdBuffer(cudaStream_t const cudaStream, cupvaCmdBuffer_t const cmdBuffer, int32_t const executionTimeout, int32_t const submitTimeout)

Submit a command buffer to a CUDA stream.

Functions#

cupvaError_t CupvaCudaContextSupportsCuda(
cupvaContext_t const ctx,
bool *const cudaFlag,
)#

Check if CUDA interop is enabled.

Returns whether the current process has disabled CUDA.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: Yes

  • API group

    • Init: No

    • Runtime: Yes

    • De-Init: No

Parameters:

  • ctx – [in] The pointer to a cupvaContext_t object. This parameter is deprecated and will be ignored.

  • cudaFlag – [out] The pointer to return a flag: true if CUDA has not been disabled for the current process, false otherwise.

Returns:

cupvaError_t The completion status of the operation. Possible values are:

  • CUPVA_ERROR_NONE if the operation was successful.

  • CUPVA_INVALID_ARGUMENT if cudaFlag was a NULL pointer.

cupvaError_t CupvaCudaCreateStream(
cupvaStream_t *const stream,
cudaStream_t const cudaStream,
cupvaEngineType_t const absEngine,
cupvaAffinityType_t const vpuAffinity,
)#

Create a PVA Stream object from a CUDA stream handle.

Deprecated:

This function is deprecated. Use CupvaCudaStreamSubmit instead.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: Yes

  • API group

    • Init: Yes

    • Runtime: No

    • De-Init: No

Parameters:

  • stream – [out] The pointer to return a newly constructed cupvaStream_t object.

  • cudaStream – [in] The CUDA stream handle.

  • absEngine – [in] The PVA engine ID.

  • vpuAffinity – [in] The queue submission affinity.

Returns:

cupvaError_t The completion status of the operation. Possible values are:

  • CUPVA_ERROR_NONE if the operation was successful.

  • CUPVA_INVALID_ARGUMENT indicates one of the following:

    • stream was a NULL pointer.

    • cudaStream was 0.

    • vpuAffinity was invalid.

  • CUPVA_DRIVER_API_ERROR if the PVA driver returned an unexpected error.

  • CUPVA_NOT_ALLOWED_IN_OPERATIONAL_STATE if called when NVIDIA DRIVE OS VM state is “Operational”

  • CUPVA_CUDA_DISABLED if CUDA interop has been disabled for this process

cupvaError_t CupvaCudaForceDisableSupport(void)#

Disable CUDA interop support.

Used to disable CUDA interop on a process-wide basis.

During creation of the first cupvaContext_t for a process, the CUDA driver library will be loaded.

In certain safety applications, it is desirable for the process to be fully isolated from any dependencies on CUDA. Calling this API will ensure that future calls which require CUDA will fail before loading the CUDA driver or performing any CUDA API calls.

This API will also signal whether the process has previously made calls which required the CUDA driver to be loaded. This is atomic - signalling true is a guaranteed that this process has never and will never, make successful CUDA interop calls.

To ensure that the calling process has no dependency on CUDA, this API should be called prior to any other CUPVA APIs.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: Yes

  • API group

    • Init: Yes

    • Runtime: No

    • De-Init: No

Returns:

cupvaError_t The completion status of the operation. Possible values are:

  • CUPVA_ERROR_NONE if the operation was successful.

  • CUPVA_INVALID_STATE if the process has previously made calls which required the CUDA driver to be loaded

cupvaError_t CupvaCudaMemImport(
void **const devPtr,
void *const cudaDevPtr,
int64_t const size,
cupvaMemAccessType_t const accessType,
)#

Create a CUPVA pointer from a CUDA device pointer.

Deprecated:

This function is deprecated. CUDA device pointers no longer need to be imported explicitly.

Importing a CUDA device pointer creates a mapping which must be released by calling CupvaMemFree() when the application has finished using the buffer. The CUDA device pointer must be at least 64B aligned.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: Yes

  • API group

    • Init: Yes

    • Runtime: No

    • De-Init: No

Parameters:

  • devPtr – [out] The double pointer to return the CUPVA device pointer pointing to the imported memory.

  • cudaDevPtr – [in] The CUDA device pointer returned by a CUDA API (cudaMalloc, for example).

  • size – [in] The size of the allocation in bytes.

  • accessType – [in] The access type of the mapped memory.

Returns:

cupvaError_t The completion status of the operation. Possible values are:

  • CUPVA_ERROR_NONE if the operation was successful.

  • CUPVA_INVALID_ARGUMENT indicates one of the followings:

    • devPtr was a NULL pointer.

    • size was invalid.

    • PVA needs 64B aligned addresses and CUDA device pointer is not aligned on 64B.

  • CUPVA_INTERNAL_ERROR if VA allocation failed.

  • CUPVA_DRIVER_API_ERROR if Getting a device pointer failed due to driver API error.

  • CUPVA_NOT_ALLOWED_IN_OPERATIONAL_STATE if called when NVIDIA DRIVE OS VM state is “Operational”

  • CUPVA_CUDA_DISABLED if CUDA interop has been disabled for this process

cupvaError_t CupvaCudaStreamSetAffinity(
cudaStream_t const cudaStream,
cupvaAffinityType_t const vpuAffinity,
)#

Set the VPU affinity for a CUDA stream.

By default, submissions to CUDA streams will permitted for scheduling on any VPU. This function can be used to change this behavior by setting the VPU affinity on a per-stream basis.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: No

  • API group

    • Init: No

    • Runtime: Yes

    • De-Init: No

Parameters:

  • cudaStream – [in] The CUDA stream handle.

  • vpuAffinity – [in] The VPU affinity to set for the stream.

Returns:

cupvaError_t The completion status of the operation.

cupvaError_t CupvaCudaStreamSubmit(
cudaStream_t const cudaStream,
cupvaCmd_t const *const *const command,
cupvaCmdStatus_t *const status,
int32_t const count,
cupvaOrderType_t const order,
int32_t const executionTimeout,
int32_t const submitTimeout,
)#

Submit a command list to a CUDA stream.

This API requires a PVA driver version >= 2007 and it is not supported in Native mode.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: Yes

  • API group

    • Init: No

    • Runtime: Yes

    • De-Init: No

Parameters:

  • cudaStream – [in] The CUDA stream handle.

  • command – [in] The command list to be submitted.

  • status – [out] The pointer to the array of the command status.

  • count – [in] The number of commands in the command list.

  • order – [in] The order type of the command list.

  • executionTimeout – [in] The timeout for the execution of the command list.

  • submitTimeout – [in] The timeout for the submission of the command list.

Returns:

cupvaError_t The completion status of the operation.

cupvaError_t CupvaCudaStreamSubmitCmdBuffer(
cudaStream_t const cudaStream,
cupvaCmdBuffer_t const cmdBuffer,
int32_t const executionTimeout,
int32_t const submitTimeout,
)#

Submit a command buffer to a CUDA stream.

This API requires a PVA driver version >= 2007 and it is not supported in Native mode.

Usage considerations

  • Allowed context for the API call

    • Thread-safe: Yes

  • API group

    • Init: No

    • Runtime: Yes

    • De-Init: No

Parameters:

  • cudaStream – [in] The CUDA stream handle.

  • cmdBuffer – [in] The command buffer to be submitted.

  • executionTimeout – [in] The timeout for the execution of the command buffer.

  • submitTimeout – [in] The timeout for the submission of the command buffer.

Returns:

cupvaError_t The completion status of the operation.