For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo module

fp4_dequant

Portable NVFP4 -> bf16 weight-only dequant for the Apple M5 matmul (W4A16).

SM100/Apple M5 (compute_capability == 5), but the math here is hardware-neutral (plain LUT + scale, no PTX / MFMA intrinsics). FLUX.2-dev-NVFP4 stores Linear weights as packed E2M1 values with per-16-element FP8-E4M3 block scales; the goal is to keep them packed in DRAM and dequant to bf16 in-register at the matmul loader seam, feeding the EXISTING bf16 Apple MMA (the M5 MMA has no FP4 input).

Layout contract (Apple = PLAIN rank-2, NOT the SM100 5D TCGEN interleave):

packed weight: uint8 [N, K // 2], two E2M1 nibbles per byte, LOW nibble first (element 2*j is byte & 0xF, element 2*j+1 is byte >> 4). This matches cast_uint_to_fp4e2m1 in linalg/fp4_utils.mojo.
block scales: float8_e4m3fn [N, K // NVFP4_SF_VECTOR_SIZE] (block size 16 along K), applied as abs(scale).
per-element dequant: w_bf16 = E2M1_TO_FLOAT32[nibble] * |scale_block|.

The global / tensor scalar scale (alpha) folds in once at the matmul epilogue, not here -- this routine produces the per-element dequantized weight only.

Functions

dequant_fp4_nibble: Dequantizes one E2M1 nibble of a packed byte to out_type.
enqueue_fp4_materialize: Enqueues fp4_materialize_kernel: packed FP4 + scales -> dense [N, K].
fp4_materialize_kernel: Materializes the packed-FP4 weight into a dense [N, K] out_type buffer.

Functions​

Functions