For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
Struct_matmul_int8_w8a8_apple
struct Struct_matmul_int8_w8a8_apple
Apple M5 int8 W8A8 matmul (no bias): out = dequant(quant(a) @ b^T).
A single FUSED graph op wrapping int8_matmul.mojo (online activation quant
- int8 widening-MMA GEMM + dequant, all internal). Inputs (matching
nn/kernels.py::_apple_int8_w8a8_matmulwithbias=None):abf16[M, K];bint8[N, K](transpose_b);b_scalefp32 per-channel[N](the Python squeezes the rowwise[N, 1]). Outputc_type[M, N]. The bias variant is the siblingmo.matmul.int8.w8a8.apple.biasop (custom-op input arity is fixed per registration, so the optional bias is a separate op name -- the same idiom asmo.fused_qkv_matmul.ragged.paged{,.bias}).
Implemented traits
Methods
execute
static def execute[c_type: DType, //, target: StringSlice[ImmStaticOrigin]](c: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=c.static_spec], a: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=a.static_spec], b: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=b.static_spec], b_scale: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=b_scale.static_spec], context: DeviceContext)
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!