For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo function

enqueue_matmul2d_fp8

def enqueue_matmul2d_fp8[c_type: DType = DType.float32, elementwise_lambda_fn: Optional[def[dtype: DType, width: SIMDLength, *, alignment: Int = Int(1)](IndexList[Int(2)], SIMD[dtype, width]) capturing thin -> None] = None, block_m: Int = Int(64), block_n: Int = Int(64), block_k: Int = Int(16), sg_m: Int = Int(32), sg_n: Int = Int(32)](c: TileTensor[c_type, Storage=c.Storage, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[DType.bfloat16, Storage=a.Storage, address_space=a.address_space, linear_idx_type=a.linear_idx_type], weight: TileTensor[DType.float8_e4m3fn, Storage=weight.Storage, address_space=weight.address_space, linear_idx_type=weight.linear_idx_type], ctx: DeviceContext)

Enqueue the tiled W8A16 GEMM: out = a @ W_fp8^T (raw, unscaled).

a is the bf16 activation (M, K), weight the FP8-E4M3 weight (N, K) (transpose_b). C is (M, N). bf16 A x fp8 B -> fp32 on the native Apple MMA, direct DRAM B feed (no SMEM). Any M/N/K (bounded MMA + guarded store). The per-tensor scalar weight_scale folds post-matmul (graph lowering).

This is the shared AppleM5MatMul simdgroup-tiled GEMM body specialized on b_type=float8_e4m3fn (the weight-loader seam swap) -- the SAME body the dense bf16 path uses, so FP8 W8A16 inherits its Morton rect tile schedule and epilogue with only the B operand widened. Numerically identical per output to the standalone kernel it replaces (Morton reorders WHICH threadgroup computes a tile, not the per-tile K accumulation). Single-pass launch: fp8 is not wired to split-K (its partial kernels are the bf16 family).

Raises:

If the attached GPU is not Apple M5 (compute_capability == 5).