Kernel discovery#

get_operators() is the entry point for finding Operators that support a given operation, operands, and target. It returns Operators drawn from the Manifest of registered kernels.

Operators can be additionally discovered or filtered by target compute capability, which is described by TargetSm.

For performance ranking and the heuristic= parameter, see Heuristics.

Return Operators that match the given arguments, metadata filter, and target.

Parameters:

args (RuntimeArguments | None) – Runtime arguments describing the operator invocation (e.g. GemmArguments). When None, no argument-based filtering is applied.
metadata_filter (Callable[[OperatorMetadata], bool] | None) – An optional Callable that takes OperatorMetadata as input and returns a boolean indicating if it should be considered for inclusion in results. The result is an intersection of operators filtered by the callable and by other parameters passed to this method.
target_sm (TargetSm | str | None) – Compute capability to target (e.g. "100a" or a TargetSm instance). Filters Operators that cannot run on this target.
providers (list[Provider] | None) – Optional list of Providers to restrict discovery to (e.g. [ops.CuTeDSLProvider]).
heuristic (Heuristic | None) – Optional performance heuristic used to order the matching Operators best-first. When None (the default), the unranked filtered list is returned (identical to discovery without a heuristic). When a heuristic is requested, it may also narrow the list to the Operators it can recommend.
limit (int | None) – If set, return at most limit Operators. With a heuristic, that is the top limit ranked Operators; without one, the first limit in discovery order. Forwarded to the heuristic when ranking.

Returns:

Operators matching all filters. When a heuristic is provided, the list is ranked best-first by the heuristic, and may potentially prune some suboptimal but otherwise valid candidates.

Return type:

list[Operator]

Raises:

ValueError – If limit is set and is not a positive integer.
ImportError – If the requested heuristic’s required packages are not installed.
TypeError – If the heuristic rejects the provided args type (e.g. nvmatmul with non-GemmArguments).
RuntimeError – If the heuristic returns an Operator that wasn’t in the candidate list, or duplicates one beyond its original count.
Exception – Any other error raised by the heuristic while ranking.

class cutlass.operators.manifest.Manifest#

Bases: object

A Manifest holds a collection of generated Operators.

static get_operators( args: RuntimeArguments = None, metadata_filter: Callable[[OperatorMetadata], bool] | None = None, target_sm: TargetSm | str | None = None, providers: list[Provider | type[Provider]] = None, ) → list[Operator]#

Get the operators that match the given arguments, metadata filter, compute capability and available providers.

Parameters:

args (RuntimeArguments) – The arguments of the operator
metadata_filter (Callable[[OperatorMetadata], bool]) – A boolean function that takes in OperatorMetadata and returns whether an Operator with this metadata should be included
target_sm (TargetSm | str | None) – Optional target compute capability for which operators should be fetched
providers (list[Provider]) – Optionally limit the search to the given providers. If None, all available providers are used.

Returns:

The operators that match the given arguments, metadata filter, and compute capability

Return type:

list[Operator]

add_operators(operators: list[Operator]) → None#

Add the given operators to the Manifest’s set of discovered operators.

Parameters:: operators (list[Operator]) – The operators to add to the Manifest

filter_operators( args: RuntimeArguments = None, metadata_filter: Callable[[OperatorMetadata], bool] | None = None, target_sm: TargetSm | str | None = None, providers: list[Provider] = None, ) → list[Operator]#

Filter the operators that have been added to this Manifest.

This is similar to get_operators, but only considers operators previously added to this Manifest using add_operators().

property operators: list[Operator]#: Get the operators that have been added to this Manifest.

class cutlass.operators.arch.TargetSm( sm_str: str | None = None, *, cc: int | None = None, portability: ArchPortability | None = None, )#

Bases: object

Target compute capability & portability to compile for.

classmethod ensure( value: TargetSm | str | None, ) → TargetSm | None#: Coerce a string to TargetSm, or return as-is if already one.

static get_supported_targets( design: DesignMetadata, operands: OperandsMetadata, ) → list[TargetSm]#

Get the supported targets for a given design and operands.

Raises:: ValueError – If design does not have a mma_instruction_type field.

static intersect_supported_targets( target_groups: list[list[TargetSm]], ) → list[TargetSm]#

Intersect component target groups by portability (logical AND across groups).

Generalises the supported-target rule to any number of component groups (MMA, tile scheduler, …): an arch is kept only when every group holds some target that is_portable_to it.

Parameters:: target_groups (list[list[TargetSm]]) – One target list per component.
Returns:: Targets supported by all groups, in first-seen order.
Return type:: list[TargetSm]

is_portable_to( other: TargetSm | str, ) → bool#: Check if this target can compile/run on architecture described by other TargetSm.

supports_operators_from( operator_targets: Iterable[TargetSm | str], ) → bool#: Check if this target can compile/run operators designed for any of the given targets.

property major: int#: Get the major version of the target SM.

property minor: int#: Get the minor version of the target SM.

class cutlass.operators.arch.ArchPortability(value)#

Bases: Enum

Portability of a compiled Operator to other architectures/compute capabilities.

See:: https://docs.nvidia.com/cuda/cuda-programming-guide/05-appendices/compute-capabilities.html#feature-set-compiler-targets

Portable = 1#: Portable to future architectures (e.g. sm_100 without “a” or “f”).

FamilyPortable = 2#: Portable to future architectures within the same family (e.g. sm_100f). Only applicable to Blackwell and newer architectures (cc >= 100).

ArchConditional = 3#: Not portable to any other architecture (e.g. sm_100a). Only applicable to Hopper and newer architectures (cc >= 90).