For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo struct

Int8DequantWriter

struct Int8DequantWriter[c_origin: MutOrigin, s_origin: ImmOrigin, //, c_type: DType, c_layout: TensorLayout, as_layout: TensorLayout, bs_layout: TensorLayout, bias_layout: TensorLayout, *, has_bias: Bool]

Owns the C-output write of the W8A8 epilogue (the write-side counterpart to the FP4 Fp4WeightLoader).

Encapsulates the int32-accumulator -> dequant -> c_type store: given a 16x16 accumulator fragment and its tile origin, it applies the per-row activation scale x per-column weight scale (+ optional per-column bias) and stores each in-bounds element through the C TileTensor. All addressing is TileTensor indexing / store -- no raw pointer arithmetic and no origin rebase. The struct is generic over the view origins (c_origin mutable for the C write, s_origin immutable and shared by the scale/bias reads), inferred by @fieldwise_init from the constructor args, so the fields hold the caller's real origins directly (a plain fieldwise constructor, no origin-erasing helper needed). Per the KB TileIO pattern (new-primitives/amd-tile-io-expert-objects), every register->DRAM transition has an owner; this is that owner for the int8 GEMM.

The _apple_frag_layout maps a lane's 8-element fragment to rows {rb, rb+8} x cols {cb, cb+1, cb+2, cb+3}: frag[0:4] = row rb, frag[4:8] = row rb+8 (matching MmaOpApple._store_fragment).

Parameters

c_origin (MutOrigin): Mutable MutOrigin of the C output write view (inferred).
s_origin (ImmOrigin): Immutable ImmutOrigin shared by the scale and bias read views (inferred).
c_type (DType): Output element type (bf16 / fp16 / fp32).
c_layout (TensorLayout): TensorLayout of the C output TileTensor.
as_layout (TensorLayout): TensorLayout of the per-row activation scale TileTensor.
bs_layout (TensorLayout): TensorLayout of the per-column weight scale TileTensor.
bias_layout (TensorLayout): TensorLayout of the per-column bias TileTensor.
has_bias (Bool): If True, add bias[col] to each output element after dequant.

Fields

c (TileTensor[c_type, c_layout, c_origin]):
a_scale (TileTensor[DType.float32, as_layout, s_origin]):
b_scale (TileTensor[DType.float32, bs_layout, s_origin]):
bias (TileTensor[c_type, bias_layout, s_origin]):
M (Int):
N (Int):

Implemented traits

AnyType, Copyable, ImplicitlyCopyable, ImplicitlyDeletable, Movable

Methods

`write_frag`

def write_frag(self, frag: SIMD[DType.int32, SIMDLength(8)], tile_r0: Int, tile_c0: Int, rb: Int, cb: Int)

Dequant + store one 16x16 accumulator fragment (this lane's 8 elems).

Bounds-guards rows < M and cols < N for the partial last tile; width-4 store when the 4 cols are in-bounds, scalar on the N-edge.

Args:

frag (SIMD[DType.int32, SIMDLength(8)]): This lane's 8 int32 accumulator elements from the 16x16 MMA fragment (frag[0:4] = row rb, frag[4:8] = row rb+8).
tile_r0 (Int): Absolute row origin of the 16x16 accumulator tile in the C output.
tile_c0 (Int): Absolute column origin of the 16x16 accumulator tile in the C output.
rb (Int): In-fragment row base offset for this lane; selects rows rb and rb+8 within the tile.
cb (Int): In-fragment column base offset for this lane; selects the 4-column group {cb, cb+1, cb+2, cb+3} within the tile.

`write_frag_full`

def write_frag_full(self, frag: SIMD[DType.int32, SIMDLength(8)], tile_r0: Int, tile_c0: Int, rb: Int, cb: Int)

Interior full-tile store: dequant + width-4 store, no bounds check.

The run caller invokes this only for not is_edge tiles, where every row and 4-col group is provably in-bounds -- so the per-fragment guards of write_frag are dropped. Bit-identical to write_frag here.

Parameters​

Fields​

Implemented traits​

Methods​

write_frag​

write_frag_full​