For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

TiledMatmul

struct TiledMatmul[config: KernelConfig, transpose_b: Bool, b_packed: Bool, elementwise_epilogue_enabled: Bool, kernel_id: InnerKernelID, a_type: DType, a_layout: TensorLayout, a_origin: ImmOrigin, b_type: DType, b_layout: TensorLayout, b_origin: ImmOrigin, c_type: DType, c_layout: TensorLayout, c_origin: MutOrigin, algorithm: InnerMatmulKernel, ElementwiseEpilogueFnType: def(GemmShape, GemmShape, TileTensor[*?, *?, origin_of(origin), Storage=*?, linear_idx_type=*?]) -> None & ImplicitlyCopyable]

Tiled matmul implementation integrating packing, inner loop and tile partitions.

Parameters

• ​config (KernelConfig): Kernel configuration controlling tile dimensions and SIMD width.
• ​transpose_b (Bool): Whether the B operand is transposed.
• ​b_packed (Bool): Whether B was pre-packed offline in the cache-friendly layout.
• ​elementwise_epilogue_enabled (Bool): Whether to apply the elementwise epilogue on the last K tile.
• ​kernel_id (InnerKernelID): Identifier of the inner microkernel to dispatch.
• ​a_type (DType): Element type of the A operand.
• ​a_layout (TensorLayout): Memory layout of the A operand.
• ​a_origin (ImmOrigin): Memory origin provenance of the A operand.
• ​b_type (DType): Element type of the B operand.
• ​b_layout (TensorLayout): Memory layout of the B operand.
• ​b_origin (ImmOrigin): Memory origin provenance of the B operand.
• ​c_type (DType): Element type of the C output.
• ​c_layout (TensorLayout): Memory layout of the C output.
• ​c_origin (MutOrigin): Memory origin provenance of the mutable C output.
• ​algorithm (InnerMatmulKernel): Microkernel implementing the inner accumulate loop.
• ​ElementwiseEpilogueFnType (def(GemmShape, GemmShape, TileTensor[*?, *?, origin_of(origin), Storage=*?, linear_idx_type=*?]) -> None & ImplicitlyCopyable): Type of the elementwise epilogue function applied to each output tile.

Fields

• ​alg (algorithm): Inner microkernel conforming to InnerMatmulKernel.
• ​c (TileTensor[c_type, c_layout, c_origin]): Output TileTensor the result accumulates into.
• ​a (TileTensor[a_type, a_layout, a_origin]): Left operand TileTensor.
• ​b (TileTensor[b_type, b_layout, b_origin]): Right operand TileTensor.
• ​tile_n_k (IndexList[Int(2)]): Dynamic (N, K) tile extents used to partition the problem.
• ​global_tile_offset (GemmShape): (M, N, K) offset of this routine's tile region.
• ​global_tile_shape (GemmShape): (M, N, K) extent of this routine's tile region.
• ​b_tile_generator (BTileGenerator[config, a_type, b_type, c_type, b_layout, transpose_b, b_packed, b_origin]): Generator that packs B sub-tiles for the inner kernel.
• ​elementwise_epilogue_fn (ElementwiseEpilogueFnType): Fused elementwise epilogue applied to each output tile.

Implemented traits

AnyType, Copyable, ImplicitlyCopyable, ImplicitlyDeletable, Movable