For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo function

enqueue_apple_int8_quantize_activation

def enqueue_apple_int8_quantize_activation[in_type: DType = DType.bfloat16](q: TileTensor[DType.int8, Storage=q.Storage, address_space=q.address_space, linear_idx_type=q.linear_idx_type], a: TileTensor[in_type, Storage=a.Storage, address_space=a.address_space, linear_idx_type=a.linear_idx_type], a_scale: TileTensor[DType.float32, Storage=a_scale.Storage, address_space=a_scale.address_space, linear_idx_type=a_scale.linear_idx_type], ctx: DeviceContext)

Quantize a (M, K) to int8 q + per-row fp32 a_scale (M,).

One threadgroup per row (M threadgroups, 64 threads each). Symmetric absmax/127, no zero-point. Raises if the GPU is not Apple M5.

Parameters:

in_type (DType): Element type of the input activation tensor (defaults to bfloat16).

Args:

q (TileTensor[DType.int8, Storage=q.Storage, address_space=q.address_space, linear_idx_type=q.linear_idx_type]): Output int8 TileTensor of shape (M, K), written mutable.
a (TileTensor[in_type, Storage=a.Storage, address_space=a.address_space, linear_idx_type=a.linear_idx_type]): Input activation TileTensor of shape (M, K) in in_type.
a_scale (TileTensor[DType.float32, Storage=a_scale.Storage, address_space=a_scale.address_space, linear_idx_type=a_scale.linear_idx_type]): Output per-row fp32 scale TileTensor of shape (M,), written mutable.
ctx (DeviceContext): Device context used to enqueue the kernel.