For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
Fp8WeightLoader
struct Fp8WeightLoader[w_layout: TensorLayout]
Owner of the FP8 weight -> f32 register transition for the W8A16 GEMV.
The FP8 sibling of Fp4WeightLoader (matmul2d_fp4.mojo), specialized for
the unpacked E4M3 GEMV: there is no nibble unpack and no per-block scale, so a
column chunk is a single coalesced width-W float8_e4m3fn load widened to
f32. Holds the weight [N, K] TileTensor view and does ALL addressing via
TileTensor width-loads -- no raw pointer arithmetic.
Parameters
- w_layout (
TensorLayout): Layout of the FP8 weight[N, K]view.
Fields
- weight (
TileTensor[DType.float8_e4m3fn, w_layout, ImmUntrackedOrigin]):
Implemented traits
AnyType,
Copyable,
ImplicitlyCopyable,
ImplicitlyDeletable,
Movable
Methods
from_kernel_args
static def from_kernel_args(weight: TileTensor[DType.float8_e4m3fn, w_layout, ImmutAnyOrigin]) -> Self
Build the loader from the kernel's AnyOrigin weight arg.
Rebases the view onto ImmUntrackedOrigin (the field-origin rule; the
arg outlives the K-loop), preserving layout/shape/stride.
load_col_chunk
def load_col_chunk[width: Int](self, n: Int, k0: Int) -> SIMD[DType.float32, width]
This column's width FP8 weights at [n, k0 : k0+width), E4M3 -> f32.
One coalesced width-width float8_e4m3fn load (adjacent lanes read
adjacent runs) widened to f32 by the hardware E4M3 conversion (exact on
M5). alignment=1 makes no alignment claim (never miscompiles; the FP4
GEMV's packed-byte load uses the same conservative claim). Callers pass
k0 + width <= K (the aligned interior uses width = TILE_K; the K tail
uses width = 1).
Parameters:
- width (
Int): SIMD width of the load (TILE_Kinterior,1for the tail).
Returns:
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!