6. Type System#

All values and operations in Tile IR are statically typed. This section defines Tile IR’s types, as well as their equivalence, layouts, and other type system details that may be relevant for DSL and compiler authors.

Most Tile IR data values are tensor valued. There are two concrete tensor families: tiles, which are pure tensor values, and views, which are structured references to tensors in memory. Tokens are non-data values used to order memory operations. Element types participate in tensor types but do not describe SSA values on their own; in particular, a pointer is an element type normally carried by a rank-0 tile.

6.1. Element Types#

Element types are the native data types supported by Tile IR. By themselves they do not describe a value. As Tile IR operates over tensors, these types describe the hardware accelerated, primitive values that can be contained by a tensor. They specify how a sequence of bits are to be interpreted. Each element type has a size associated with it that represents the number of bits required to represent it.

Note

Note that this is different from a potential storage size, which is specified by the data layout of the tensor which contains these values.

Element types come in two flavors, general purpose fundamental types that come without restriction, and specialized alternative types which each come with a set of restrictions.

6.1.1. Fundamental Types#

Tile IR supports a set of general purpose integer and floating-point types that are supported by all operations and have no restrictions. These can be contained in arbitrary rank, and shape tensors, and 0-rank values of this type can be treated as scalars.

Fundamental Types#

Type

Sizes

Description

i1, i8, i16, i32, i64

1, 8, 16, 32, 64

signless integer type of specified size

f16, f32, f64

16, 32, 64

IEEE floating-point type of specified size

Primary elemental types are supported in all arithmetic operations.

Warning

Integer types are signless, i.e., the type does not encode whether the represented value is to be interpreted as a signed or unsigned value. For operations where this distinction is semantically meaningful signedness is controlled via flags on each arithmetic operation.

6.1.2. Alternative Types#

Tile IR also supports a set of non-standard but hardware accelerated floating-point types. Due to the nature of these types and hardware they each come with a set of restrictions.

Alternative Types#

Type

Size

Description

tf32

32

floating-point format with 8 bits for exponent and 10 bits for mantissa. Storage size is 4 bytes with 4-byte alignment

bf16

16

floating-point format with 8 bits for exponent and 7 bits for mantissa

f8E4M3FN

8

floating-point format with 4 bits for exponent and 3 bits for mantissa

f8E5M2

8

floating-point format with 5 bits for exponent and 2 bits for mantissa

f8E8M0FNU

8

unsigned floating-point format with 8 bits for exponent and no explicit mantissa bits

fnv8E5M3FNU*

8

unsigned floating-point format with 5 bits for exponent and 3 bits for mantissa

f4E2M1FN

4

floating-point format with 2 bits for exponent and 1 bits for mantissa

* (Developer Preview version only)

Tensors of these types may be created, manipulated and loaded and stored from global memory, but certain computations on them are restricted.

6.1.3. Floating-Point Conversion Semantics#

When converting values to a floating-point type (via cuda_tile.ftof or cuda_tile.itof), the behavior for out-of-finite-range values and special values depends on the target type.

f16, f32, f64 (IEEE types) and bf16, tf32 (IEEE-like types): the closest representable value is selected according to the specified rounding mode. This may produce Inf when the source value exceeds the target’s finite range. Inf and NaN values are preserved.

f8E4M3FN, f8E5M2 (signed 8-bit float types): use saturation-to-finite (satfinite) semantics, meaning the closest representable finite value is selected according to the specified rounding mode. Inf is never produced, even if the source value was Inf. This is the case even for f8E5M2, which otherwise supports infinities. NaN values are preserved.

f8E8M0FNU, fnv8E5M3FNU (unsigned 8-bit float types): use satfinite semantics. Positive values above the finite range and positive infinity produce positive MAX_NORM. NaN values are preserved. For any supported conversion, a negative numeric source produces an undefined value; for floating-point sources, this includes negative zero and negative infinity.

Because f8E8M0FNU has no zero representation, converting positive zero produces MIN_POS (\(2^{-127}\)). Direct conversion from an integer type to f8E8M0FNU is not supported; the integer must first be converted to a wider floating-point type and then converted to f8E8M0FNU with cuda_tile.ftof.

f4E2M1FN (4-bit float types): use saturation-to-finite (satfinite) semantics and also do not support NaN values. Inf and NaN values are not preserved.

Table Floating-Point Conversion: Special Value and Saturation Behavior enumerates the behavior for the different supported floating-point types when converting various “corner case” values.

Floating-Point Conversion: Special Value and Saturation Behavior#

Target Type

Out-of-Range Finite Source

Infinite Source

Source is NaN

f16, f32, f64 bf16, tf32

Nearest representable value (may produce Inf)

±Inf

NaN

f8E5M2, f8E4M3FN

Nearest representable finite value (±MAX_NORM)

±Inf → ±MAX_NORM

NaN

f8E8M0FNU, fnv8E5M3FNU

Positive source → +MAX_NORM

+Inf → +MAX_NORM

NaN

f4E2M1FN

Nearest representable finite value (±MAX_NORM)

±Inf → ±MAX_NORM

+MAX_NORM

Note

The f4E2M1FN type does not support NaN; NaN inputs are converted to positive MAX_NORM.

Note

The flush_to_zero flags of various operations treat input subnormals as zero and also flush output subnormals to zero. When an operation specifies both a rounding mode (including the approx and full variants of transcendentals such as cuda_tile.exp and cuda_tile.tanh) and FTZ (flush-to-zero) handling of subnormals, the rounding step is applied first and FTZ is applied to the rounded result.

6.2. Sub-Byte Types#

Tile IR supports element types whose storage size is smaller than a byte. f4E2M1FN is currently the only sub-byte element type that may be used as the pointee type of a pointer or as the element type of a view. i4 is supported in tiles and by cuda_tile.pack and cuda_tile.unpack, but is not a valid pointer or view element type. Because memory is byte-addressable, sub-byte element values must be packed together into bytes for storage and accessed in groups whose total size is an integral number of bytes.

6.2.1. Load and Store Requirements#

An f4E2M1FN pointer contains an integral byte address. Each addressed byte stores two consecutive packed f4 elements. Consequently, tile<ptr<f4E2M1FN>> is valid only at rank 0. This scalar pointer form is used as the base pointer of an f4 tensor view. Direct cuda_tile.load_ptr_tko and cuda_tile.store_ptr_tko operations do not support f4 because pointer values cannot independently address consecutive sub-byte elements.

Packed f4 memory can instead be accessed with pointer-based operations by treating its representation as i8. A rank-1 tile<Nxi8> loaded with cuda_tile.load_ptr_tko can be converted with cuda_tile.unpack to a rank-1 f4 tile containing 2 * N elements. Conversely, cuda_tile.pack converts the f4 tile to tile<Nxi8> before it is written with cuda_tile.store_ptr_tko.

cuda_tile.offset measures its offset in pointee elements. An f4 offset must be even, and advances the byte address by half the element offset. Executing cuda_tile.offset with an odd f4 element offset is undefined behavior.

An f4 view load or store is structurally valid only when its underlying cuda_tile.tensor_view has exactly one statically encoded stride equal to 1. Its dimension is the packing dimension. A dynamic stride does not identify a packing dimension, even when its runtime value is 1, and multiple static unit strides are ambiguous. A program containing a view load or store without a unique statically identified packing dimension is invalid and must be rejected by the compiler, whether or not the operation executes. Merely constructing the tensor view or a tile view derived from it is permitted.

An f4 gather/scatter load or store is likewise structurally invalid when the packing dimension is the sparse dimension and must be rejected by the compiler.

The following constraints are execution-time preconditions on view construction required for any memory access using the view. A view can be constructed that violates one or more conditions; undefined behavior is only triggered if and when a load or store operation executes using the view.

The constraints are:

  • The accessed tile must not contain a pair in the packing dimension for which exactly one element is in bounds of the tensor view.

  • Each applicable value below must be even:

    • every tensor-view element stride other than the packing-dimension stride;

    • the accessed tile extent mapped to the packing dimension;

    • for a cuda_tile.strided_view, the traversal stride mapped to the packing dimension; and

    • for a cuda_tile.gather_scatter_view, the scalar index selecting the origin in the packing dimension.

Values required to be even can be dynamic. No static proof or guarantee of evenness is required; the compiler always assumes evenness for all relevant values. A statically-visible violation does not make the operation structurally invalid and no error will diagnose the violation; it continues to be undefined behavior when the operation executes, as for dynamic values.

The following table summarizes the f4 memory-access requirements and when they are enforced.

f4 Memory Access Requirements Summary#

Form or operation

Requirement

Effect

tile<ptr<f4E2M1FN>>

The pointer tile must have rank 0.

A nonzero-rank pointer tile type is invalid.

Direct pointer load or store

Direct f4 access is unsupported. Access the packed representation as i8, using cuda_tile.unpack after a load or cuda_tile.pack before a store.

A direct f4 pointer load or store is invalid.

cuda_tile.offset on an f4 pointer

The element offset must be even.

An odd offset causes undefined behavior when the offset operation executes.

Tensor-view or tile-view construction

Beyond ordinary view well-formedness, none of the access-specific requirements apply.

Construction is permitted and does not by itself cause undefined behavior.

f4 view load or store

The view must have exactly one statically encoded unit stride, and a gather/scatter view must not use that packing dimension as its sparse dimension.

The operation is structurally invalid and must be rejected, whether or not it executes.

Execution of an f4 view load or store

The access must satisfy the complete-pair and evenness constraints above.

A violation causes undefined behavior only when the load or store executes; no static proof is required.

Examples#

Consider the following underlying tensor_view, which identifies dimension one as its packing dimension:

tensor_view<?x?xf4E2M1FN, strides=[?, 1]>

The following cases illustrate the requirements:

  • Replacing the strides with [?, ?] or [1, 1] would still permit construction of the view, but would make an f4 view load or store structurally invalid because the packing dimension would be absent or ambiguous, respectively.

  • A partition_view<tile=(4x8), ...> has an even accessed-tile extent in packing dimension 1 and is therefore valid.

  • A first stride of 10 satisfies the non-packing stride requirement. A first stride of 9 causes undefined behavior when an f4 view load or store executes.

  • A strided_view<tile=(4x8), traversal_strides=[4, 2], ...> has an even traversal stride in the packing dimension. A traversal stride of 1 causes undefined behavior when the load or store executes.

  • For a gather/scatter view whose sparse dimension is not dimension 1, a scalar origin index of 2 in dimension 1 satisfies the evenness requirement. An origin index of 1 causes undefined behavior when the load or store executes.

The second tensor extent may be odd. For an extent of 9, the packed pairs at the upper boundary are:

elements:    [0 1] [2 3] [4 5] [6 7] [8 OOB]
pair status:  full  full  full  full   split

A packing-dimension access beginning at 0 with extent 8 contains only complete pairs and is defined. An access beginning at 8, or containing element 8, contains the split pair [8 OOB] and has undefined behavior. Both accesses must also satisfy the other ordinary and f4-specific requirements above.

6.2.2. Packing Order#

Sub-byte elements are packed densely with little-endian nibble order: the element at the lower index occupies the lower bits of the byte. For 4-bit elements, elements i and i+1 (where i is a multiple of 2) are packed into a single byte such that element i occupies bits 3...0 and element i+1 occupies bits 7...4.

For example, a tensor_view<2xf4E2M1FN, strides=[1]> holding the values [0.5, 1.5] is stored as the single byte:

Bit position:    7   6   5   4   3   2   1   0
               +---+---+---+---+---+---+---+---+
               | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 1 |
               +---+---+---+---+---+---+---+---+
               |     1.5       |     0.5       |
               | upper nibble  | lower nibble  |

where 0.5 has the f4E2M1FN bit pattern 0001 and 1.5 has the bit pattern 0011.

The same packing order applies when a sub-byte tile is materialized from a byte tile via cuda_tile.unpack, or written back via cuda_tile.pack: the lower-indexed element of each pair occupies the lower bits of the corresponding byte.

6.3. Pointers#

Pointers, or values which contain memory addresses, are typed as pointers to a specific pointee type. A pointer points to a location in memory; the data at that location will be interpreted as being of the pointee element type when loaded.

Pointer arithmetic is measured in pointee elements: an offset of n advances by n consecutive elements in the pointee type’s memory representation. For types whose elements occupy an integral number of bytes, the byte displacement is n times the element’s storage size in bytes. For f4E2M1FN, two consecutive elements share one byte, so only even offsets are defined and an offset of n advances the byte address by n / 2. This preserves element-based pointer arithmetic while ensuring that the resulting address is an integral byte address.

Pointer types are parameterized by element types, i.e., nested pointer types are not supported. For details about converting between different pointer types, or integers, see cuda_tile.bitcast.

6.4. Token Type#

A token is an opaque value that represents an ordering dependency. Token values carry no user data and cannot be inspected. Token-ordered memory operations consume and produce tokens, allowing a program to establish an order that is independent of textual operation order. See Memory Model.

6.5. Tensor Types#

A tensor is a multi-dimensional, rectangular array described by a shape and element type. The shape is a vector that describes the number of elements across each axis of the tensor. The length of said vector describes the rank of the tensor, i.e., the number of its dimensions. All tensors in Tile IR have a statically known rank. Tile IR has two kinds of tensor types tiles and views.

6.5.1. Tile Type#

A tile is a tensor with static shape, i.e., the extent across each dimension is known at compile time. See Syntax for the tile<MxNxKxE> assembly syntax.

Note

In Tile IR, all data values to be operated on are expressed as a tile. In particular, even scalar values are represented as a tile of rank zero.

Note

A tile of pointers is typically used to load from or store to a batch of locations. The tile of pointers defines the shape of the values that are loaded or stored, with one pointer element mapping to one scalar value loaded or stored. It does not imply any structure on the locations themselves. For example, two consecutive pointers in the tile may not point to consecutive locations in memory. The same location may even be present multiple times within a single tile of pointers. See Memory Model for a discussion of implications.

6.5.2. Tensor View#

It is common that data in global memory follows a strided structure. For example, the widely adopted row-major or column-major layouts are strided. It is beneficial for the compiler to be aware of the strided layout of data in memory. Tile IR features a tensor view type to describe such structure in global memory.

Conceptually, a tensor view type describes an abstracted tensor of pointers. Like a regular tensor, it is described by a shape and the type of elements it points to. It in addition has a vector of striding factors that describe the relative position of locations the elements of the tensor view point to. If an element is \(d\) elements apart in dimension \(i\) of the tensor view, the corresponding locations in memory will be \(d * stride_i\) elements away. This information can be used by the compiler to reason about access patterns and layouts of data in memory.

Values of type tensor view are typically never materialized in memory. Rather, they are stored as a compact description of a base-location, shape and striding factors. From this information, a tensor of pointers corresponding to the full view value can be computed using the following formula

\[elem_{[i_0, ..., i_n]} = baseptr + \sum_{m=0}^{n} i_m * s_m\]

where the \(s_m\) are the striding factors of the tensor view and \(baseptr\) is the start address in global memory.

A tensor view supports dynamic extents in its shape and stride vectors; these are bound at runtime when the view is constructed using a cuda_tile.make_tensor_view operation. A tensor view cannot be directly used to access memory — it first needs to be divided into tiles of static size. See Subview Types for options to do so.

6.5.3. Subview Types#

A tensor view is often too large to be loaded as a single tile for processing. Instead, it must first be subdivided into tiles. In Tile IR this is expressed using subview types.

Subviews describe a mapping from an index space to a space of statically-sized tiles loaded from a tensor view. They define the necessary index computations performed by a cuda_tile.load_view_tko and cuda_tile.store_view_tko when accessing elements from a tensor view.

Tile IR provides partition, strided, and gather/scatter subviews. They differ in how an index-space coordinate selects a tile from the underlying tensor view. The type reference below gives their complete parameters and indexing rules.

Partition View#

partition_view is a subview type that represents a view partitioned into a grid of non-overlapping tiles. The index space in this case is the position of the tile in the grid. Partition views are particularly useful in patterns like matrix multiplication, where a large tensor in global memory is traversed as non-overlapping tiles to form the final result. The partition view structure is created using the cuda_tile.make_partition_view constructor.

The formal index-space shape, element-address calculation, bounds behavior, parameters, and examples are defined by cuda_tile.partition_view in the type reference.

6.6. Type Equivalence#

Tile IR does not provide means to name types. Equivalence of types hence is a purely structural property: Two types are considered equal if they are structurally identical.

Note that some types form a natural subtype relationship. Types with dynamic shapes and strides like view cover the values that an identical type with all dynamic shapes and strides substituted with static values would cover. However, we consider these types distinct in Tile IR.

6.7. Type Reference#

6.7.1. cuda_tile.gather_scatter_view#

Gather/scatter view type

13.3

Parameters#

  • tile_shape - tile shape 13.3

  • tensor_view - tensor view 13.3

  • sparse_dim - sparse dimension 13.3

  • padding_value - padding value 13.3

Description#

!cuda_tile.gather_scatter_view represents a view into a tensor_view where one dimension is accessed using a sparse gather/scatter pattern while the remaining dimensions are accessed contiguously.

!cuda_tile.gather_scatter_view has the following specification:

  • Index space rank: as many dimensions as the underlying tensor_view.

  • Tile sizes: as specified by tile_shape.

It consists of:

  • tile_shape: an integer array that describes the shape of the tiles in the view.

  • tensor_view: the type of the tensor_view into which the view is looking.

  • sparse_dim: a non-negative integer that specifies which dimension to gather/scatter over. Must be strictly less than the rank of the tensor_view.

  • padding_value: an optional enum, specifying the value that should be used for out-of-bounds accesses (loads) into the tensor_view.

Supported padding values include:

  • zero: zero

  • neg_zero: negative zero

  • nan: NaN

  • pos_inf: positive infinity

  • neg_inf: negative infinity

Note

Only power-of-two tile dimensions are supported.

When loading or storing via a gather_scatter_view, the index at position sparse_dim must be a 1D tile whose size equals tile_shape[sparse_dim]. Each element of this tile is an independent index into the underlying tensor_view along that dimension. The loaded or stored tile will be the concatenation of the elements in the rows selected by the sparse dimension index. All other indices must be scalar tiles of the same element type, each choosing the offset in the underlying tensor_view along that dimension and selecting a contiguous block of tile_shape[dim] elements.

The gather/scatter view’s index-space shape equals the underlying tensor_view shape. The index-space shape and the offsets selected by the sparse indices, measured in elements from the tensor_view base pointer, are:

\[\begin{split}\begin{aligned} S_{\mathrm{ind}}[d] &= S_{\mathrm{view}}[d], \\ \mathrm{Offsets}[i] &= \mathrm{Coords}[s][i] \cdot \mathrm{Stride}[s] + \sum_{\substack{0 \le d < \mathrm{rank} \\ d \ne s}} \mathrm{Coords}[d] \cdot \mathrm{Stride}[d]. \end{aligned}\end{split}\]

where \(\mathrm{Coords}[s]\) is the 1D sparse index tensor and every other component of \(\mathrm{Coords}\) is a scalar. Capitalized variables are tensors, lowercase variables are scalars, and brackets index a tensor:

\[\begin{split}\begin{aligned} \mathrm{rank} &:\quad \text{rank of the tile view}, \\ d &:\quad \text{tile-view dimension, }0 \le d \le \mathrm{rank}-1, \\ s &:\quad \text{sparse dimension}, \\ i &:\quad \text{index within the sparse index tensor, }0 \le i < S_{\mathrm{tile}}[s], \\ S_{\mathrm{view}} &:\quad \text{shape of the underlying tensor view}, \\ \mathrm{Stride} &:\quad \text{element strides of the underlying tensor view}, \\ S_{\mathrm{tile}} &:\quad \text{shape of the tile loaded or stored}, \\ \mathrm{Coords} &:\quad \text{indices accessing the tile view}, \\ S_{\mathrm{ind}} &:\quad \text{shape of the index space}, \\ \mathrm{Offsets} &:\quad \text{offsets selected by the sparse index tensor}. \end{aligned}\end{split}\]

Each \(\mathrm{Offsets}[i]\) identifies the origin of a tensor slice with extent one along dimension \(s\) and extent \(S_{\mathrm{tile}}[d]\) along every other dimension. A load concatenates these slices in increasing \(i\) order along dimension \(s\) to form the result tile. A store applies the same mapping in reverse, writing slice \(i\) of the source tile at \(\mathrm{Offsets}[i]\).

In the examples below, !gsv_1d, !gsv_2d, and !gsv_2d_padded have index space shapes of 8, 8x8, and 128x256, respectively.

Indices on non-sparse dimensions must be within the index space, although the selected tile may only partially overlap the underlying tensor_view. Individual values in the sparse-dimension index tile may be outside the tensor view. Operations handle out-of-bounds elements as follows:

  • Load operations: If padding_value is set, out-of-bounds elements yield the padding value. Otherwise, they yield unspecified values.

  • Store operations: Out-of-bounds elements are masked.

Note

Sparse-dimension indices need not be unique. Repeated sparse index values in a load cause the same row to be loaded multiple times. Repeated sparse index values in a store produce undefined values in the repeated row.

Examples:

// (1) A 1D gather/scatter view over an 8xf32 tensor_view with a tile
// size of 4 and sparse_dim=0. Since the view is 1D, the only index is
// the gather index: a size-4 1D tile of element indices. The table below
// visualizes a load with gather_indices=[6, 1, 4, 3]. Gathered elements
// are indicated with their index in the single loaded tile. ( ) marks
// elements that are not gathered.
//
//               8
// ←─────────────────────────────→
// ( ) (1) ( ) (3) (2) ( ) (0) ( )
//
// In pseudocode, we can say that loading with gather_indices=[6, 1, 4, 3]
// is equivalent to the following:
//   result = [tensor_view[6], tensor_view[1], tensor_view[4], tensor_view[3]]
//
!gsv_1d= !cuda_tile.gather_scatter_view<
  tile=(4),
  tensor_view<8xf32, strides=[1]>,
  sparse_dim=0
>

// (2) A 2D gather/scatter view over an 8x8xf32 tensor_view with tile
// size 4x4 and sparse_dim=0. The first index (at sparse_dim) is a size-4
// 1-D tile of row indices; the second index is a scalar selecting a
// block of 4 contiguous columns. The table below visualizes a load
// with gather_indices=[5, 1, 7, 3] and col_idx=0. Each number
// indicates the result row, and ( ) marks elements not gathered.
//
//                      8
//       ←─────────────────────────────→
//     ↑ ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )    row 0: not gathered
//     │ (1) (1) (1) (1) ( ) ( ) ( ) ( )    row 1: → result row 1
//     │ ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )    row 2: not gathered
//   8 │ (3) (3) (3) (3) ( ) ( ) ( ) ( )    row 3: → result row 3
//     │ ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )    row 4: not gathered
//     │ (0) (0) (0) (0) ( ) ( ) ( ) ( )    row 5: → result row 0
//     │ ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )    row 6: not gathered
//     ↓ (2) (2) (2) (2) ( ) ( ) ( ) ( )    row 7: → result row 2
//
!gsv_2d= !cuda_tile.gather_scatter_view<
  tile=(4x4),
  tensor_view<8x8xf32, strides=[8, 1]>,
  sparse_dim=0
>

// (3) A larger gather/scatter view with zero-padding for out-of-bounds
// accesses. If the index for dimension 1 is in the range [241, 255],
// the out-of-bounds elements will be filled with zero. If no padding value
// were set, the values would be unspecified. Out of bounds sparse dimension
// indices, in dimension 0, will yield a row of zero (the padding value).
// Likewise, the values of the entire row are unspecified if no padding
// value is set.
//
!gsv_2d_padded= !cuda_tile.gather_scatter_view<
  tile=(8x16),
  padding_value = zero,
  tensor_view<128x256xf32, strides=[256, 1]>,
  sparse_dim=0
>

// (4) Gather along the second dimension (sparse_dim=1). Here the
// first index is a scalar selecting which block of 8 rows, and
// the second index is a 1D tile of 16 column indices.
//
!gsv_2d_col= !cuda_tile.gather_scatter_view<
  tile=(8x16),
  tensor_view<128x256xf32, strides=[256, 1]>,
  sparse_dim=1
>

// (5) Scatter (store) using the same view type as example (2).
// Given a 4x4 tile to store and scatter_indices=[5, 1, 7, 3]
// with col_idx=0, row 0 of the tile is written to row 5 of the
// tensor_view, row 1 to row 1, row 2 to row 7, and row 3 to
// row 3. Columns 0-3 are written contiguously. The table below
// shows which tensor_view rows are written [*] and which are
// untouched ( ).
//
//                      8
//       ←─────────────────────────────→
//     ↑ ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )    row 0: untouched
//     │ [1] [1] [1] [1] ( ) ( ) ( ) ( )    row 1: ← tile row 1
//     │ ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )    row 2: untouched
//   8 │ [3] [3] [3] [3] ( ) ( ) ( ) ( )    row 3: ← tile row 3
//     │ ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )    row 4: untouched
//     │ [0] [0] [0] [0] ( ) ( ) ( ) ( )    row 5: ← tile row 0
//     │ ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )    row 6: untouched
//     ↓ [2] [2] [2] [2] ( ) ( ) ( ) ( )    row 7: ← tile row 2
//
// In pseudocode:
//   tensor_view[5, 0:4] = tile[0, :]
//   tensor_view[1, 0:4] = tile[1, :]
//   tensor_view[7, 0:4] = tile[2, :]
//   tensor_view[3, 0:4] = tile[3, :]
//
!gsv_2d_scatter= !cuda_tile.gather_scatter_view<
  tile=(4x4),
  tensor_view<8x8xf32, strides=[8, 1]>,
  sparse_dim=0
>

6.7.2. cuda_tile.partition_view#

Partition view type

13.1

Parameters#

  • tile_shape - tile shape 13.1

  • tensor_view - tensor view 13.1

  • dim_map - dimension mapping 13.1

  • padding_value - padding value 13.1

Description#

!cuda_tile.partition_view represents a view into a tensor_view where tiles are laid out in a grid pattern across the original tensor_view. The grid is aligned with the start of each dimension and there are no gaps or overlaps between tiles.

!cuda_tile.partition_view has the following specification:

  • Index space rank: as many dimensions as the underlying tensor_view.

  • Tile sizes: as specified by tile_shape.

It consists of:

  • tile_shape: a dense integer array that describes the shape of the tiles in the view.

  • tensor_view: the type of the tensor_view into which the view is looking.

  • dim_map: an integer array that specifies for each tile dimension the corresponding dimension in the underlying tensor_view.

  • padding_value: an optional enum, specifying the value that should be used for out-of-bounds accesses (loads) into the tensor_view.

Supported padding values include:

  • zero: zero

  • neg_zero: negative zero

  • nan: NaN

  • pos_inf: positive infinity

  • neg_inf: negative infinity

Note

Only power-of-two tile dimensions are supported.

The partition view index space is determined by tile_shape, the tensor_view shape, and dim_map. Its shape and the offset of the selected tile’s origin from the tensor_view base pointer, measured in elements, are:

\[\begin{split}\begin{aligned} S_{\mathrm{ind}}[d] &= \operatorname{ceildiv}\left( S_{\mathrm{view}}[\mathrm{DimMap}[d]], S_{\mathrm{tile}}[d] \right), \\ \operatorname{offset}(\mathrm{Coords}) &= \sum_{d=0}^{\mathrm{rank}-1} \mathrm{Coords}[d] \cdot S_{\mathrm{tile}}[d] \cdot \mathrm{Stride}[\mathrm{DimMap}[d]]. \end{aligned}\end{split}\]

where capitalized variables are tensors, lowercase variables are scalars, and brackets index a tensor:

\[\begin{split}\begin{aligned} \mathrm{rank} &:\quad \text{rank of the tile view}, \\ d &:\quad \text{tile-view dimension, }0 \le d \le \mathrm{rank}-1, \\ S_{\mathrm{view}} &:\quad \text{shape of the underlying tensor view}, \\ \mathrm{Stride} &:\quad \text{element strides of the underlying tensor view}, \\ S_{\mathrm{tile}} &:\quad \text{shape of the tile loaded or stored}, \\ \mathrm{DimMap} &:\quad \text{mapping from tile dimensions to tensor-view dimensions}, \\ \mathrm{Coords} &:\quad \text{indices accessing the tile view}, \\ S_{\mathrm{ind}} &:\quad \text{shape of the index space}, \\ \operatorname{offset} &:\quad \text{offset of the selected tile's origin}. \end{aligned}\end{split}\]

In the examples below, !pv_2d has an index space shape of 16x8, whereas !pv_2d_transposed has an index space shape of 4x32.

Partition view indices must lie within the partition view’s index space; otherwise, the behavior is undefined. For example, index (0, 8) is invalid for !pv_2d below because the maximum index in dimension 1 is 7.

An in-bounds view index may select a tile that only partially overlaps the underlying tensor_view. A tile cannot be fully outside the tensor view without its view index also being outside the index space. Operations handle partially out-of-bounds tiles as follows:

  • Load operations: If padding_value is set, out-of-bounds tile elements yield the padding value. Otherwise, they yield unspecified values.

  • Store operations: Out-of-bounds tile elements are masked.

Examples:

// (1) A view into a 16xf32 tensor_view with a tile size of 2. The table
// below visualizes for each element of the tensor_view the corresponding
// tile, as indicated by its index.
//
//                               16
// ←─────────────────────────────────────────────────────────────→
// (0) (0) (1) (1) (2) (2) (3) (3) (4) (4) (5) (5) (6) (6) (7) (7)
//
!pv_1d= !cuda_tile.partition_view<
  tile=(2),
  tensor_view<16xf32, strides=[1]>
>

// (2) A view into a 64x16xf32 tensor_view with a tile size of 4x2. By
// convention, in the below table, the Y axis corresponds to the first
// tensor_view dimension and the X axis corresponds to the second one.
//
//                                   16
//       ←────────────────────────────────────────────────────────── ...
//     ↑ (0,0) (0,0) (0,1) (0,1) (0,2) (0,2) (0,3) (0,3) (0,4) (0,4) ...
//     │ (0,0) (0,0) (0,1) (0,1) (0,2) (0,2) (0,3) (0,3) (0,4) (0,4) ...
//     │ (0,0) (0,0) (0,1) (0,1) (0,2) (0,2) (0,3) (0,3) (0,4) (0,4) ...
//     │ (0,0) (0,0) (0,1) (0,1) (0,2) (0,2) (0,3) (0,3) (0,4) (0,4) ...
//  64 │ (1,0) (1,0) (1,1) (1,1) (1,2) (1,2) (1,3) (1,3) (1,4) (1,4) ...
//     │ (1,0) (1,0) (1,1) (1,1) (1,2) (1,2) (1,3) (1,3) (1,4) (1,4) ...
//     │ (1,0) (1,0) (1,1) (1,1) (1,2) (1,2) (1,3) (1,3) (1,4) (1,4) ...
//     │ (1,0) (1,0) (1,1) (1,1) (1,2) (1,2) (1,3) (1,3) (1,4) (1,4) ...
//     │ (2,0) (2,0) (2,1) (2,1) (2,2) (2,2) (2,3) (2,3) (2,4) (2,4) ...
//    ...
//
!pv_2d= !cuda_tile.partition_view<
  tile=(4x2),
  tensor_view<64x16xf32, strides=[16, 1]>
>

// (3) A view into a 64x16xf32 tensor_view with a tile size of 4x2. The
// first tile dimension is mapped to the second tensor_view dimension. The
// second tile dimension is mapped to the first tensor_view dimension.
//
//                                   16
//       ←────────────────────────────────────────────────────────── ...
//     ↑ (0,0) (0,0) (0,0) (0,0) (1,0) (1,0) (1,0) (1,0) (2,0) (2,0) ...
//     │ (0,0) (0,0) (0,0) (0,0) (1,0) (1,0) (1,0) (1,0) (2,0) (2,0) ...
//     │ (0,1) (0,1) (0,1) (0,1) (1,1) (1,1) (1,1) (1,1) (2,1) (2,1) ...
//  64 │ (0,1) (0,1) (0,1) (0,1) (1,1) (1,1) (1,1) (1,1) (2,1) (2,1) ...
//     │ (0,2) (0,2) (0,2) (0,2) (1,2) (1,2) (1,2) (1,2) (2,2) (2,2) ...
//     │ (0,2) (0,2) (0,2) (0,2) (1,2) (1,2) (1,2) (1,2) (2,2) (2,2) ...
//    ...
//
!pv_2d_transposed= !cuda_tile.partition_view<
  tile=(4x2),
  tensor_view<64x16xf32, strides=[16, 1]>,
  dim_map=[1, 0]
>

// Note: A load from partition_view with non-default dim_map is
// semantically identical to a load with default dim_map followed by a
// permutation.
//
// %0 = load_view_tko ... %view[%a, %b]
//     : partition_view<tile=(4x2), ..., dim_map=[1, 0]> -> tile<4x2xf32>
//
// Is identical to:
//
// %0 = load_view_tko ... %view[%b, %a]
//     : partition_view<tile=(2x4), ..., dim_map=[0, 1]> -> tile<2x4xf32>
// %1 = permute %0 [1, 0] : tile<2x4xf32> -> tile<4x2xf32>

// (4) A view into a 8x2xf32 tensor_view with a tile size of 1x4 and NaN
// padding. The right half of the below table consists of padded NaN
// values.
//
//            2
//       ←─────────→
//     ↑ (0,0) (0,0) (0,0) (0,0)
//     │ (1,0) (1,0) (1,0) (1,0)
//   8 │ (2,0) (2,0) (2,0) (2,0)
//     │ (3,0) (3,0) (3,0) (3,0)
//     │ (4,0) (4,0) (4,0) (4,0)
//    ...
//
!pv_2d_padded= !cuda_tile.partition_view<
  tile=(1x4),
  padding_value = nan,
  tensor_view<8x2xf32, strides=[2,1]>,
>

6.7.3. cuda_tile.ptr#

Pointer type

13.1

Parameters#

  • pointeeType - f16 or bf16 or f32 or tf32 or f64 or f8E4M3FN or f8E5M2 or f8E8M0FNU or f4E2M1FN or fnv8E5M3FNU or i1 or i8 or i16 or i32 or i64 13.1

  • ptrAttr - ptr_attr 13.4

Description#

An element pointer type $pointerType represents a single location in global device memory. Pointer types are typed, i.e., they carry the type they point to. Any numeric type can be used as pointee type.

An optional #cuda_tile.ptr_attr<...> suffix records the pointer classification; #cuda_tile.ptr_attr<none> denotes an ordinary local pointer.

6.7.4. cuda_tile.strided_view#

Strided view type

13.3

Parameters#

  • tile_shape - tile shape 13.3

  • traversal_strides - traversal strides 13.3

  • tensor_view - tensor view 13.3

  • dim_map - dimension mapping 13.3

  • padding_value - padding value 13.3

Description#

!cuda_tile.strided_view represents a view into a tensor_view where tiles are laid out in a grid pattern across the original tensor_view. The grid is aligned with the start of each dimension, but the grid pattern’s striding factor is parametric, allowing interleaved or overlapping tiles.

!cuda_tile.strided_view has the following specification:

  • Index space rank: as many dimensions as the underlying tensor_view.

  • Tile sizes: as specified by tile_shape.

It consists of:

  • tile_shape: an integer array that describes the shape of the tiles in the view.

  • traversal_strides: an integer array that describes the traversal strides when traversing the underlying tensor_view. For example, if the shape of tiles is 2x2 and the traversal strides are [3,3], the tensor_view will be traversed with a gap of one element between each tile. Alternatively, if the traversal strides are [1,1], the tensor_view will be traversed with an overlapping sliding window, advancing by one element for each tile.

  • tensor_view: the type of the tensor_view into which the view is looking.

  • dim_map: an integer array that specifies for each tile dimension the corresponding dimension in the underlying tensor_view.

  • padding_value: an optional enum, specifying the value that should be used for out-of-bounds accesses (loads) into the tensor_view.

Supported padding values include:

  • zero: zero

  • neg_zero: negative zero

  • nan: NaN

  • pos_inf: positive infinity

  • neg_inf: negative infinity

Note

Only power-of-two tile dimensions are supported. In contrast, traversal strides can be any strictly positive value.

The strided view index space is determined by traversal_strides, the tensor_view shape, and dim_map. Partial tiles at the edges of the tensor view are included in the index space. Its shape and the offset of the selected tile’s origin from the tensor_view base pointer, measured in elements, are:

\[\begin{split}\begin{aligned} S_{\mathrm{ind}}[d] &= \operatorname{ceildiv}\left( S_{\mathrm{view}}[\mathrm{DimMap}[d]], \mathrm{Traversal}[d] \right), \\ \operatorname{offset}(\mathrm{Coords}) &= \sum_{d=0}^{\mathrm{rank}-1} \mathrm{Coords}[d] \cdot \mathrm{Traversal}[d] \cdot \mathrm{Stride}[\mathrm{DimMap}[d]]. \end{aligned}\end{split}\]

where capitalized variables are tensors, lowercase variables are scalars, and brackets index a tensor:

\[\begin{split}\begin{aligned} \mathrm{rank} &:\quad \text{rank of the tile view}, \\ d &:\quad \text{tile-view dimension, }0 \le d \le \mathrm{rank}-1, \\ S_{\mathrm{view}} &:\quad \text{shape of the underlying tensor view}, \\ \mathrm{Stride} &:\quad \text{element strides of the underlying tensor view}, \\ \mathrm{Traversal} &:\quad \text{traversal strides of the strided view}, \\ \mathrm{DimMap} &:\quad \text{mapping from tile dimensions to tensor-view dimensions}, \\ \mathrm{Coords} &:\quad \text{indices accessing the tile view}, \\ S_{\mathrm{ind}} &:\quad \text{shape of the index space}, \\ \operatorname{offset} &:\quad \text{offset of the selected tile's origin}. \end{aligned}\end{split}\]

In the examples below, !sv_1d_tra2 and !sv_1d_tra3 have index space shapes of 8 and 6, respectively. !sv_2d has an index space shape of 16x6, and !sv_2d_transposed has an index space shape of 4x22.

Strided view indices must lie within the strided view’s index space; otherwise, the behavior is undefined. For example, index (0, 6) is invalid for !sv_2d below because the maximum index in dimension 1 is 5.

An in-bounds view index may select a tile that only partially overlaps the underlying tensor_view. A tile cannot be fully outside the tensor view without its view index also being outside the index space. Operations handle partially out-of-bounds tiles as follows:

  • Load operations: If padding_value is set, out-of-bounds tile elements yield the padding value. Otherwise, they yield unspecified values.

  • Store operations: Out-of-bounds tile elements are masked.

Examples:

// (1) A view into a 16xf32 tensor_view with a tile size of 2 and a
// traversal stride of 2. This behavior is identical to PartitionView. The
// table below visualizes for each element of the tensor_view the
// corresponding tile, as indicated by its index.
//
//                               16
// ←─────────────────────────────────────────────────────────────→
// (0) (0) (1) (1) (2) (2) (3) (3) (4) (4) (5) (5) (6) (6) (7) (7)
//
!sv_1d_tra2= !cuda_tile.strided_view<
  tile=(2),
  traversal_strides=[2],
  tensor_view<16xf32, strides=[1]>
>

// (2) A view into a 16xf32 tensor_view with a tile size of 2 and a
// traversal stride of 3. This time, one out of three elements are
// skipped, as the stride moves three elements while the tile only
// covers two. Notice that tile 5 is partially out of bounds.
//
//                               16
// ←─────────────────────────────────────────────────────────────→
// (0) (0) ( ) (1) (1) ( ) (2) (2) ( ) (3) (3) ( ) (4) (4) ( ) (5) (5)
//
!sv_1d_tra3= !cuda_tile.strided_view<
  tile=(2),
  traversal_strides=[3],
  tensor_view<16xf32, strides=[1]>
>

// (3) A view into a 8xf32 tensor_view with a tile size of 2 and a
// traversal stride of 1. This time, tiles are overlapping, and the
// same element will be present in multiple tiles. For example,
// element 1 of the tensor view is present both as the second element of
// tile 0, and as the first element tile 1. Tile 7 is partially out of bounds.
// (Note the space is still 1-dimensional; the second row indicates overlap,
// not a second dimension.)
//
//               8
// ←─────────────────────────────→
// (0) (0) (2) (2) (4) (4) (6) (6)
//     (1) (1) (3) (3) (5) (5) (7) (7)
//
!sv_1d_tra1= !cuda_tile.strided_view<
  tile=(2),
  traversal_strides=[1],
  tensor_view<8xf32, strides=[1]>
>

// (4) A view into a 64x16xf32 tensor_view with a tile size of 4x2 and
// traversal strides of [4,3]. Tiles are adjacent in the first dimension,
// but have a gap of one element in the second dimension. By convention,
// in the below table, the Y axis corresponds to the first tensor_view
// dimension and the X axis corresponds to the second one.
//
//                                   16
//       ←────────────────────────────────────────────────────────── ...
//     ↑ (0,0) (0,0) (   ) (0,1) (0,1) (   ) (0,2) (0,2) (   ) (0,3) ...
//     │ (0,0) (0,0) (   ) (0,1) (0,1) (   ) (0,2) (0,2) (   ) (0,3) ...
//     │ (0,0) (0,0) (   ) (0,1) (0,1) (   ) (0,2) (0,2) (   ) (0,3) ...
//     │ (0,0) (0,0) (   ) (0,1) (0,1) (   ) (0,2) (0,2) (   ) (0,3) ...
//  64 │ (1,0) (1,0) (   ) (1,1) (1,1) (   ) (1,2) (1,2) (   ) (1,3) ...
//     │ (1,0) (1,0) (   ) (1,1) (1,1) (   ) (1,2) (1,2) (   ) (1,3) ...
//     │ (1,0) (1,0) (   ) (1,1) (1,1) (   ) (1,2) (1,2) (   ) (1,3) ...
//     │ (1,0) (1,0) (   ) (1,1) (1,1) (   ) (1,2) (1,2) (   ) (1,3) ...
//     │ (2,0) (2,0) (   ) (2,1) (2,1) (   ) (2,2) (2,2) (   ) (2,3) ...
//    ...
//
!sv_2d= !cuda_tile.strided_view<
  tile=(4x2),
  traversal_strides=[4,3]
  tensor_view<64x16xf32, strides=[16, 1]>
>

// (5) A view into a 64x16xf32 tensor_view with a tile size of 4x2 and
// traversal strides of [4,3], as above, but with dim_map=[1, 0].
// (The first tile dimension is mapped to the second tensor_view dimension
// and the second tile dimension is mapped to the first tensor_view dimension.)
// Tiles are adjacent in the second dimension, but have a gap of one element in
// the first dimension.
//
//                                   16
//       ←────────────────────────────────────────────────────────── ...
//     ↑ (0,0) (0,0) (0,0) (0,0) (1,0) (1,0) (1,0) (1,0) (2,0) (2,0) ...
//     │ (0,0) (0,0) (0,0) (0,0) (1,0) (1,0) (1,0) (1,0) (2,0) (2,0) ...
//     | (   ) (   ) (   ) (   ) (   ) (   ) (   ) (   ) (   ) (   ) ...
//     │ (0,1) (0,1) (0,1) (0,1) (1,1) (1,1) (1,1) (1,1) (2,1) (2,1) ...
//  64 │ (0,1) (0,1) (0,1) (0,1) (1,1) (1,1) (1,1) (1,1) (2,1) (2,1) ...
//     | (   ) (   ) (   ) (   ) (   ) (   ) (   ) (   ) (   ) (   ) ...
//     │ (0,2) (0,2) (0,2) (0,2) (1,2) (1,2) (1,2) (1,2) (2,2) (2,2) ...
//     │ (0,2) (0,2) (0,2) (0,2) (1,2) (1,2) (1,2) (1,2) (2,2) (2,2) ...
//     | (   ) (   ) (   ) (   ) (   ) (   ) (   ) (   ) (   ) (   ) ...
//    ...
//
!sv_2d_transposed= !cuda_tile.strided_view<
  tile=(4x2),
  traversal_strides=[4,3],
  tensor_view<64x16xf32, strides=[16, 1]>,
  dim_map=[1, 0]
>

// Note: A load from a strided_view with non-default dim_map is
// semantically identical to a load with default dim_map followed by a
// permutation.
//
// %0 = load_view_tko ... %view[%a, %b]
//     : strided_view<tile=(4x2), traversal_strides=[4,2], ...,
//                    dim_map=[1, 0]> -> tile<4x2xf32>
//
// Is identical to:
//
// %0 = load_view_tko ... %view[%b, %a]
//     : strided_view<tile=(2x4), traversal_strides=[2,4], ...,
//                    dim_map=[0, 1]> -> tile<2x4xf32>
// %1 = permute %0 [1, 0] : tile<2x4xf32> -> tile<4x2xf32>

// (6) A view into a 8x2xf32 tensor_view with a tile size of 1x4 and NaN
// padding. The right half of the below table consists of padded NaN
// values.
//
//            2
//       ←─────────→
//     ↑ (0,0) (0,0) (0,0) (0,0)
//     │ (1,0) (1,0) (1,0) (1,0)
//   8 │ (2,0) (2,0) (2,0) (2,0)
//     │ (3,0) (3,0) (3,0) (3,0)
//     │ (4,0) (4,0) (4,0) (4,0)
//    ...
//
!sv_2d_padded= !cuda_tile.strided_view<
  tile=(1x4),
  traversal_strides=[1,4],
  padding_value = nan,
  tensor_view<8x2xf32, strides=[2,1]>,
>

6.7.5. cuda_tile.tensor_view#

Tensor view type

13.1

Parameters#

  • elementType - f16 or bf16 or f32 or tf32 or f64 or f8E4M3FN or f8E5M2 or f8E8M0FNU or f4E2M1FN or fnv8E5M3FNU or i1 or i8 or i16 or i32 or i64 13.1

  • shape - shape of the tensor view 13.1

  • strides - strides of the tensor view 13.1

  • ptrAttr 13.4

Description#

!cuda_tile.tensor_view represents a reference to a tensor in global memory.

It consists of:

  • elementType: the type of the elements in the tensor_view.

  • shape: an integer array that specifies the size of each dimension. Sizes must be strictly positive.

  • strides: an integer array that describes the stride of each dimension. The stride is the number of elements to offset in memory when increasing the corresponding index by one. Strides must be strictly positive.

The shape and the stride can be dynamic on a per-dimension basis. In those cases, their values are printed as ?.

Note

Only power-of-two tile dimensions are supported.

Note

4-bit element types are packed densely in global memory. Tensor views with 4-bit element types must have at least one dimension with a unit stride and such dimensions must have an even number of elements to ensure byte-alignment.

Elements i and i+1 (where i is a multiple of 2) are packed into a byte as follows: element i is stored in bits 3...0 and element i+1 is stored in bits 7...4. This corresponds to a little-endian nibble order.

Example: A tensor_view<2xf4E2M1FN, strides=[1]> with values [0.5, 1.5] is stored as:

Bit position:    7     6     5     4     3     2     1     0
              ┌─────┬─────┬─────┬─────┬─────┬─────┬─────┬─────┐
              │  0  │  0  │  1  │  1  │  0  │  0  │  0  │  1  │
              └─────┴─────┴─────┴─────┴─────┴─────┴─────┴─────┘
              ├──── upper nibble ────┤ ├──── lower nibble ────┤
              │          1.5         │ │         0.5          │
              └──────────────────────┘ └──────────────────────┘
Bit pattern of 0.5 in f4E2M1FN: 0001
Bit pattern of 1.5 in f4E2M1FN: 0011

Examples:

// A 512x1024 global memory tensor in row-major (lexicographic) order.
!cuda_tile.tensor_view<512x1024xf16, strides=[1024, 1]>

// A 512x1024 global memory tensor in column-major (colexicographic) order.
!cuda_tile.tensor_view<512x1024xf16, strides=[1, 512]>

// A 512x1024 global memory tensor that enumerates the same memory location
// multiple times.
!cuda_tile.tensor_view<512x1024xf16, strides=[1, 1]>

// A 32x16x32 global memory tensor that is neither row-major nor
// column-major.
!cuda_tile.tensor_view<32x16x32xf16, strides=[512, 1, 16]>

// A ?x? global memory tensor with a unit stride at the last dimension.
!cuda_tile.tensor_view<?x?xf16, strides=[?, 1]>

// A ?x16 global memory tensor with a unit stride at the first dimension.
!cuda_tile.tensor_view<?x16xf32, strides=[1, ?]>

6.7.6. cuda_tile.tile#

Tile type

13.1

Parameters#

  • shape - shape of the tile 13.1

  • elementType - f16 or bf16 or f32 or tf32 or f64 or f8E4M3FN or f8E5M2 or f8E8M0FNU or f4E2M1FN or fnv8E5M3FNU or i1 or i8 or i16 or i32 or i64 or Pointer type or i4 13.1

Description#

A tile is a value type that has a shape and an element type. The shape of the tile must be fully static. All elements of the tile have the same element type. Any numeric type or pointer type can be used as an element type.

Only power-of-two shape dimensions are supported.

Examples:

!cuda_tile.tile<8x4xf32>

!cuda_tile.tile<4x!cuda_tile.ptr<i8>>

6.7.7. cuda_tile.token#

Cuda tile token type

13.1

Parameters#

No parameters.

Description#

Tokens are not runtime values. Their purpose is to explicitly represent ordering constraints between token-ordered operations executed within a tile.