For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
AppleInt8ActQuant
struct AppleInt8ActQuant[in_type: DType = DType.bfloat16, *, THREADS: Int = Int(64)]
Per-row symmetric-absmax int8 quantization of the activation.
One threadgroup per row: the THREADS threads cooperatively reduce
absmax over the row's K columns (strided), then quantize
q = roundeven(x * 127 / absmax) and write the row's fp32 scale = absmax / 127. Mirrors _quantize_a_block's scheme (symmetric absmax/127,
no zero-point) but as a GPU row kernel. No SMEM reduction primitive is
assumed; the absmax reduction is a two-pass strided scan (the row fits in
L1/L2 and is re-read cheaply -- K <= 12288 for FLUX).
Parameters
- in_type (
DType): Element type of the input activation tensor (defaults tobfloat16). - THREADS (
Int): Number of threads per threadgroup cooperating on the per-row reduction (defaults to 64).
Implemented traits
Methods
run
static def run[q_layout: TensorLayout, a_layout: TensorLayout, s_layout: TensorLayout](q: TileTensor[DType.int8, q_layout, MutAnyOrigin], a: TileTensor[in_type, a_layout, ImmutAnyOrigin], a_scale: TileTensor[DType.float32, s_layout, MutAnyOrigin], K: Int)
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!