For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo function

heuristic_and_outliers_dispatch

def heuristic_and_outliers_dispatch[c_type: DType, a_type: DType, b_type: DType, scales_dtype: DType, //, SF_VECTOR_SIZE: Int, transpose_b: Bool = True, elementwise_lambda_fn: Optional[def[dtype: DType, width: SIMDLength, *, alignment: Int = Int(1)](IndexList[Int(2)], SIMD[dtype, width]) capturing thin -> None] = None, elementwise_compute_lambda_fn: Optional[def[dtype: DType, width: SIMDLength, *, alignment: Int = Int(1)](IndexList[Int(2)], SIMD[dtype, width]) capturing thin -> SIMD[dtype, width]] = None, pdl_level: PDLLevel = PDLLevel()](c: TileTensor[c_type, Storage=c.Storage, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[a_type, Storage=a.Storage, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b: TileTensor[b_type, Storage=b.Storage, address_space=b.address_space, linear_idx_type=b.linear_idx_type], a_scales: TileTensor[scales_dtype, Storage=a_scales.Storage, address_space=a_scales.address_space, linear_idx_type=a_scales.linear_idx_type], b_scales: TileTensor[scales_dtype, Storage=b_scales.Storage, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type], tensor_sf: Float32, ctx: DeviceContext) -> Int

Dispatches an SM100 block-scaled matmul by selecting a tuning config from per-format outlier tables for specific M ranges, falling back to a small-BN config for GEMVs (m == 1) and a heuristic config table for the remaining cases. Returns DISPATCH_HIT when a matching config is found and launched, or DISPATCH_MISS when no config matches.

Parameters:

c_type (DType): Element type of the output tensor c (inferred).
a_type (DType): Element type of the LHS input tensor a (inferred).
b_type (DType): Element type of the RHS input tensor b (inferred).
scales_dtype (DType): Element type of the per-block scale tensors a_scales and b_scales (inferred).
SF_VECTOR_SIZE (Int): Number of elements each scale factor covers. Must match the format: 16 for NVFP4, 32 for MXFP4, or 32 for MXFP8.
transpose_b (Bool): Whether b is stored transposed. Must be True (defaults to True).
elementwise_lambda_fn (Optional[def[dtype: DType, width: SIMDLength, *, alignment: Int = Int(1)](IndexList[Int(2)], SIMD[dtype, width]) capturing thin -> None]): Optional epilogue applied to the matmul result c in a separate kernel after the matmul completes (defaults to None).
elementwise_compute_lambda_fn (Optional[def[dtype: DType, width: SIMDLength, *, alignment: Int = Int(1)](IndexList[Int(2)], SIMD[dtype, width]) capturing thin -> SIMD[dtype, width]]): Optional compute function fused into the matmul kernel epilogue (defaults to None).
pdl_level (PDLLevel): Programmatic Dependent Launch scheduling level for overlapping this kernel with prior GPU work (defaults to PDLLevel()).

Args:

c (TileTensor[c_type, Storage=c.Storage, address_space=c.address_space, linear_idx_type=c.linear_idx_type]): Output TileTensor accumulating the matmul result.
a (TileTensor[a_type, Storage=a.Storage, address_space=a.address_space, linear_idx_type=a.linear_idx_type]): LHS input TileTensor.
b (TileTensor[b_type, Storage=b.Storage, address_space=b.address_space, linear_idx_type=b.linear_idx_type]): RHS input TileTensor (must be transposed).
a_scales (TileTensor[scales_dtype, Storage=a_scales.Storage, address_space=a_scales.address_space, linear_idx_type=a_scales.linear_idx_type]): Per-block scales for a.
b_scales (TileTensor[scales_dtype, Storage=b_scales.Storage, address_space=b_scales.address_space, linear_idx_type=b_scales.linear_idx_type]): Per-block scales for b.
tensor_sf (Float32): Global tensor scaling factor applied as alpha.
ctx (DeviceContext): Device context used to launch the kernel.

Returns:

Int: DISPATCH_HIT if a config was selected and the kernel launched, otherwise DISPATCH_MISS.