core.inference.moe

Submodules

• core.inference.moe.metadata
• core.inference.moe.flashinfer_mxfp8
• core.inference.moe.fused_moe
• core.inference.moe.permute
• core.inference.moe.activations
• core.inference.moe.vllm_fused_moe
• core.inference.moe.batch_invariant

Package Contents

Classes

InferenceGroupedGemmBackend

Backend for grouped GEMM operations during inference.

API

class core.inference.moe.InferenceGroupedGemmBackend(*args, **kwds)

Bases: enum.Enum

Backend for grouped GEMM operations during inference.

The string values match the inference_grouped_gemm_backend config field.

Initialization

FLASHINFER

‘flashinfer’

TORCH

‘torch’

VLLM

‘vllm’

classmethod from_config(

value: str | core.inference.moe.InferenceGroupedGemmBackend,

) → core.inference.moe.InferenceGroupedGemmBackend

Convert a config value to a grouped-GEMM backend.

Parameters:

value – Backend name or an already-converted enum member.

Returns:

The corresponding grouped-GEMM backend.

Raises:

ValueError – If value is not a supported backend.