For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo function

matmul_qint4_pack_b

def matmul_qint4_pack_b[group_size: Int](b_tt: TileTensor[DType.uint8, Storage=b_tt.Storage, linear_idx_type=b_tt.linear_idx_type], b_rot_tt: TileTensor[DType.uint8, Storage=b_rot_tt.Storage, linear_idx_type=b_rot_tt.linear_idx_type])

Repacks block-wise quantized int4 weights into the tiled layout expected by the matmul_qint4 kernels.

Parameters:

group_size (Int): Number of elements per quantization group.

Args:

b_tt (TileTensor[DType.uint8, Storage=b_tt.Storage, linear_idx_type=b_tt.linear_idx_type]): Source tensor holding packed uint8 weights with float16 scales.
b_rot_tt (TileTensor[DType.uint8, Storage=b_rot_tt.Storage, linear_idx_type=b_rot_tt.linear_idx_type]): Destination tensor for the repacked weights.

Raises:

If N is not a multiple of 32.