For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
Inner_matmul_i8mm
struct Inner_matmul_i8mm
ARM I8MM (8-bit integer matrix multiply) microkernel for CPU matmul.
Implements InnerMatmulKernel using the _neon_matmul intrinsic to
compute 8-bit integer dot products in pairs of two rows. Operates on a
pre-packed A buffer (packed by packA_i8mm) and a packed B tile, and
accumulates int32 or uint32 results from uint8/uint8,
uint8/int8, or int8/int8 operands (not signed-A, unsigned-B)
in LoadStore_i8mm.
Implemented traits
AnyType,
Copyable,
ImplicitlyCopyable,
ImplicitlyDeletable,
InnerMatmulKernel,
Movable
Methods
__inner_matmul__
def __inner_matmul__[kernel_rows: Int, kernel_cols: Int, simd_size: Int](self, c: TileTensor[Storage=c.Storage, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[Storage=a.Storage, address_space=a.address_space, linear_idx_type=a.linear_idx_type], b_packed: TileTensor[Storage=b_packed.Storage, address_space=b_packed.address_space, linear_idx_type=b_packed.linear_idx_type], global_offset: GemmShape, global_bound: GemmShape, tile_n_k: IndexList[Int(2)], skip_boundary_check: Bool)
Utility function on the inner loop. Run the inner kernel on the whole (kernel_rows2, TileN, TileK) tile.
Parameters:
- kernel_rows (
Int): Number of rows in the inner kernel tile. Halved internally for I8MM row pairing unless equal to 1. - kernel_cols (
Int): Number of columns in the inner kernel tile, also the N-dimension step size. - simd_size (
Int): SIMD lane count for the I8MM dot-product intrinsic (must be 4).
Args:
- c (
TileTensor[Storage=c.Storage, address_space=c.address_space, linear_idx_type=c.linear_idx_type]): Output C matrix tile where accumulated results are stored. - a (
TileTensor[Storage=a.Storage, address_space=a.address_space, linear_idx_type=a.linear_idx_type]): Input A matrix tile in pre-packed layout, read non-transposed. - b_packed (
TileTensor[Storage=b_packed.Storage, address_space=b_packed.address_space, linear_idx_type=b_packed.linear_idx_type]): Packed B matrix tile in cache-friendly rank-3 layout. - global_offset (
GemmShape): Global (M, N, K) coordinate offset of this tile within the full matmul problem space. - global_bound (
GemmShape): Global (M, N, K) upper bound of the full matmul problem space, used for boundary checking. - tile_n_k (
IndexList[Int(2)]): Index list with the (N, K) range to process within this tile. - skip_boundary_check (
Bool): Whether to skip boundary checks when loading and storing the C tile.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!