For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
Struct_matmul_int8_w8a8_apple_bias
struct Struct_matmul_int8_w8a8_apple_bias
Apple M5 int8 W8A8 matmul WITH per-output-channel bias.
The bias sibling of mo.matmul.int8.w8a8.apple: identical fused body plus a
bias input in c_type [N] added after dequant. Selected by
nn/kernels.py::_apple_int8_w8a8_matmul (which appends .bias to the op
name when a bias is provided, mirroring the FP8 fused-QKV op).
Implemented traits
Methods
execute
static def execute[c_type: DType, //, target: StringSlice[ImmStaticOrigin]](c: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=c.static_spec], a: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=a.static_spec], b: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=b.static_spec], b_scale: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=b_scale.static_spec], bias: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=bias.static_spec], context: DeviceContext)
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!