For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

PackMatrixCols

struct PackMatrixCols[original_mut: Bool, //, dtype: DType, simd_size: Int, column_inner_size: Int, use_vnni: Bool, use_i8mm: Bool, packed_origin: MutOrigin, original_origin: Origin[mut=original_mut], packed_layout: TensorLayout, original_layout: TensorLayout]

Pack columns from a matrix into the mlas packed layout and extract inner vectors of columns into the packed inner dimension, e.g. extracts [X, Y] and packs as [Yo][X][Yi].

Parameters

• ​original_mut (Bool): True if the original matrix buffer is mutable (inferred).
• ​dtype (DType): Element type of the matrix being packed.
• ​simd_size (Int): SIMD vector width for dtype.
• ​column_inner_size (Int): Size of the inner dimension along columns in the packed layout; must be a multiple of simd_size.
• ​use_vnni (Bool): True to pack for the VNNI instruction layout.
• ​use_i8mm (Bool): True to pack for the i8mm instruction layout.
• ​packed_origin (MutOrigin): Origin of the packed output TileTensor.
• ​original_origin (Origin[mut=original_mut]): Origin of the original input TileTensor.
• ​packed_layout (TensorLayout): Layout of the packed output TileTensor.
• ​original_layout (TensorLayout): Layout of the original input TileTensor.

Fields

• ​packed_matrix (TileTensor[dtype, packed_layout, packed_origin]):
• ​original_matrix (TileTensor[dtype, original_layout, original_origin]):
• ​global_offset (IndexList[Int(2)]):
• ​pack_tile_dim (IndexList[Int(2)]):
• ​valid_data_dim (IndexList[Int(2)]):

Implemented traits

AnyType, Copyable, ImplicitlyCopyable, ImplicitlyDeletable, Movable

Methods

run

static def run(packed_matrix: TileTensor[dtype, packed_layout, packed_origin], original_matrix: TileTensor[dtype, original_layout, original_origin], global_offset: IndexList[Int(2)], pack_tile_dim: IndexList[Int(2)], valid_data_dim: IndexList[Int(2)])

Interface function to run the packing routine. Args: packed_matrix: Pre-allocated buffer space for packed data. original_matrix: Data buffer containing the original matrix to pack. global_offset: Offset to use when indexing the original matrix. pack_tile_dim: 2D dimension tuple describing the size of the packed tile. valid_data_dim: 2D dimension tuple describing the amount of valid data on the global buffer starting from the offset.