TensorDataFlow API#

TensorDataFlow Device APIs.

Macros#

TDF_DOUBLE

Calculate the minimum buffer length (in pixels) required to support double layout for TensorDataFlow.

TDF_SINGLE

Calculate the minimum buffer length (in pixels) required to support single layout for TensorDataFlow.

VMEM_TDF_HANDLER

Declare a Tensor dataflow handler.

Functions#

void * cupvaTensorDataFlowAcquire(TDFHandler &handler)

Acquire a tensor tile for reading/writing.

void cupvaTensorDataFlowClose(TDFHandler &handler)

Close a TensorDataFlow when using acquire/release semantics.

uint32_t cupvaTensorDataFlowGetBppLog2(TDFHandler const &handler)

Returns log2(bytes per pixel) for the TDF.

uint32_t cupvaTensorDataFlowGetNiterC(TDFHandler const &handler)

Get the number of transfers on dim C.

uint32_t cupvaTensorDataFlowGetNiterHW(TDFHandler const &handler)

Get the total number of transfers on dim H & W.

uint32_t cupvaTensorDataFlowGetNumTiles(TDFHandler const &handler)

Get the number of tiles for the current transfer.

uint32_t cupvaTensorDataFlowGetTileLinePitch(TDFHandler const &handler)

Get the tile buffer line pitch.

uint32_t cupvaTensorDataFlowGetTilePlaneSize(TDFHandler const &handler)

Get the 2D plane size.

void cupvaTensorDataFlowOpen(TDFHandler &handler, void *vmemBuffer)

Open a TDF.

void cupvaTensorDataFlowRelease(TDFHandler &handler)

Release a tile, indicating reading/writing has completed for the current tile.

bool cupvaTensorDataFlowZHasRemainder(TDFHandler const &handler)

Return a flag to indicate whether the tensor depth is not multiple of tile depth.

Data Structures#

TDFHandler

Structure used for TDF with acquire/release semantics.

TensorDataFlowParams

Structure used for passing tensor data parameters from host.

Functions#

inline void *cupvaTensorDataFlowAcquire(TDFHandler &handler)#

Acquire a tensor tile for reading/writing.

Call this API when you are ready to read/write to/from a tensor tile. Only one tensor tile may be acquired at a time. No diagnostic is given for failure to ensure that only a single tile is acquired at a time.

Usage considerations

  • Allowed context for the API call

    • Interrupt handler: No

    • Signal handler: No

    • Thread-safe: No

    • Re-entrant: No

    • Async/Sync: Sync

  • Required privileges: None

  • Allowed execution state

    • Initialization: Yes

    • Operational: Yes

    • Reinit/Deinit Preparation: Yes

  • Maturity: GA

Parameters:

handler – The reference to the TDFHandler object

Returns:

Pointer to top-left of the tile. A tile is not necessarily contiguous in VMEM, as a circular buffer may be in use.

Pre:

None

inline void cupvaTensorDataFlowClose(TDFHandler &handler)#

Close a TensorDataFlow when using acquire/release semantics.

Call this API when all tiles in the TDF have been acquired and released. After this call (assuming all tiles were properly sequenced) the TDF will no longer be active and a Dataflow on a different DataFlowPhase may be triggered.

Usage considerations

  • Allowed context for the API call

    • Interrupt handler: No

    • Signal handler: No

    • Thread-safe: No

    • Re-entrant: No

    • Async/Sync: Sync

  • Required privileges: None

  • Allowed execution state

    • Initialization: Yes

    • Operational: Yes

    • Reinit/Deinit Preparation: Yes

  • Maturity: GA

Parameters:

handler – The reference to the TDFHandler object

Pre:

None

inline uint32_t cupvaTensorDataFlowGetBppLog2(
TDFHandler const &handler,
)#

Returns log2(bytes per pixel) for the TDF.

The DMA engine may be configured to transfer 1, 2 or 4 bytes per pixel. This is configured on the host side and may be queried here.

Usage considerations

  • Allowed context for the API call

    • Interrupt handler: No

    • Signal handler: No

    • Thread-safe: No

    • Re-entrant: No

    • Async/Sync: Sync

  • Required privileges: None

  • Allowed execution state

    • Initialization: Yes

    • Operational: Yes

    • Reinit/Deinit Preparation: Yes

  • Maturity: GA

Parameters:

handler – The reference to the TDFHandler object defined in the VMEM

Returns:

log2(bpp) configured for this TDF

Pre:

None

inline uint32_t cupvaTensorDataFlowGetNiterC(
TDFHandler const &handler,
)#

Get the number of transfers on dim C.

This helper function can be used to query the number of transfers on dim C.

Usage considerations

  • Allowed context for the API call

    • Interrupt handler: No

    • Signal handler: No

    • Thread-safe: No

    • Re-entrant: No

    • Async/Sync: Sync

  • Required privileges: None

  • Allowed execution state

    • Initialization: Yes

    • Operational: Yes

    • Reinit/Deinit Preparation: Yes

  • Maturity: GA

Parameters:

handler – The reference to the TDFHandler object.

Returns:

The number of transfers on dim C.

Pre:

None

inline uint32_t cupvaTensorDataFlowGetNiterHW(
TDFHandler const &handler,
)#

Get the total number of transfers on dim H & W.

This helper function can be used to query the total number of transfers on dim H & W.

Usage considerations

  • Allowed context for the API call

    • Interrupt handler: No

    • Signal handler: No

    • Thread-safe: No

    • Re-entrant: No

    • Async/Sync: Sync

  • Required privileges: None

  • Allowed execution state

    • Initialization: Yes

    • Operational: Yes

    • Reinit/Deinit Preparation: Yes

  • Maturity: GA

Parameters:

handler – The reference to the TDFHandler object.

Returns:

The total number of transfers on dim H & W.

Pre:

None

inline uint32_t cupvaTensorDataFlowGetNumTiles(
TDFHandler const &handler,
)#

Get the number of tiles for the current transfer.

This helper function can be used to query the actual number of tiles for the current transfer.

Usage considerations

  • Allowed context for the API call

    • Interrupt handler: No

    • Signal handler: No

    • Thread-safe: No

    • Re-entrant: No

    • Async/Sync: Sync

  • Required privileges: None

  • Allowed execution state

    • Initialization: Yes

    • Operational: Yes

    • Reinit/Deinit Preparation: Yes

  • Maturity: GA

Parameters:

handler – The reference to the TDFHandler object.

Returns:

The actual number of tiles for the current transfer.

Pre:

None

inline uint32_t cupvaTensorDataFlowGetTileLinePitch(
TDFHandler const &handler,
)#

Get the tile buffer line pitch.

This helper function can be used to query the tile line pitch.

Usage considerations

  • Allowed context for the API call

    • Interrupt handler: No

    • Signal handler: No

    • Thread-safe: No

    • Re-entrant: No

    • Async/Sync: Sync

  • Required privileges: None

  • Allowed execution state

    • Initialization: Yes

    • Operational: Yes

    • Reinit/Deinit Preparation: Yes

  • Maturity: GA

Parameters:

handler – The reference to the TDFHandler object

Returns:

The tile line pitch

Pre:

None

inline uint32_t cupvaTensorDataFlowGetTilePlaneSize(
TDFHandler const &handler,
)#

Get the 2D plane size.

This helper function can be used to query the tile plane size: tile_width * tile_height.

Usage considerations

  • Allowed context for the API call

    • Interrupt handler: No

    • Signal handler: No

    • Thread-safe: No

    • Re-entrant: No

    • Async/Sync: Sync

  • Required privileges: None

  • Allowed execution state

    • Initialization: Yes

    • Operational: Yes

    • Reinit/Deinit Preparation: Yes

  • Maturity: GA

Parameters:

handler – The reference to the TDFHandler object

Returns:

The 2D plane size

Pre:

None

inline void cupvaTensorDataFlowOpen(
TDFHandler &handler,
void *vmemBuffer,
)#

Open a TDF.

Using acquire/release semantics with TDF is a simpler way to access TDF tiles, where the same semantics work regardless of layout.

With the different semantics, user does not have a guarantee for when the underlying DMA engine is active. From DataFlowPhase perspective, an TDF using acquire/release semantics must be assumed to be active immediately after the call to open until the call to close.

Pointers passed to this function must be valid and must not be NULL. Failure to do so will result in undefined behavior.

Usage considerations

  • Allowed context for the API call

    • Interrupt handler: No

    • Signal handler: No

    • Thread-safe: No

    • Re-entrant: No

    • Async/Sync: Sync

  • Required privileges: None

  • Allowed execution state

    • Initialization: Yes

    • Operational: Yes

    • Reinit/Deinit Preparation: Yes

  • Maturity: GA

Parameters:

  • handler – The reference to the TDFHandler object

  • vmemBuffer – Buffer to bind the handler to. Should match the buffer passed to TensorDataFlow::vmemBuffer

Pre:

None

inline void cupvaTensorDataFlowRelease(TDFHandler &handler)#

Release a tile, indicating reading/writing has completed for the current tile.

Call this API when you are finished reading/writing to a tile.

Usage considerations

  • Allowed context for the API call

    • Interrupt handler: No

    • Signal handler: No

    • Thread-safe: No

    • Re-entrant: No

    • Async/Sync: Sync

  • Required privileges: None

  • Allowed execution state

    • Initialization: Yes

    • Operational: Yes

    • Reinit/Deinit Preparation: Yes

  • Maturity: GA

Parameters:

handler – The reference to the TDFHandler object

Pre:

None

inline bool cupvaTensorDataFlowZHasRemainder(
TDFHandler const &handler,
)#

Return a flag to indicate whether the tensor depth is not multiple of tile depth.

Usage considerations

  • Allowed context for the API call

    • Interrupt handler: No

    • Signal handler: No

    • Thread-safe: No

    • Re-entrant: No

    • Async/Sync: Sync

  • Required privileges: None

  • Allowed execution state

    • Initialization: Yes

    • Operational: Yes

    • Reinit/Deinit Preparation: Yes

  • Maturity: GA

Parameters:

handler – The reference to the TDFHandler object

Returns:

A boolean to indicate whether the tensor depth is not multiple of tile depth.

Pre:

None