For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

quantize_mxfp4_amd

def quantize_mxfp4_amd[out_dtype: DType = DType.uint8, scales_dtype: DType = DType.float8_e8m0fnu, in_dtype: DType = DType.bfloat16, //, *, num_max_threads: Int = Int(512)](ctx: DeviceContext, output_tile: TileTensor[out_dtype, Storage=output_tile.Storage, linear_idx_type=output_tile.linear_idx_type], scales_tile: TileTensor[scales_dtype, Storage=scales_tile.Storage, linear_idx_type=scales_tile.linear_idx_type], input_tile: TileTensor[in_dtype, Storage=input_tile.Storage, linear_idx_type=input_tile.linear_idx_type])

Quantize BF16 activations to MXFP4 on AMD CDNA4 (MI355X).

Produces packed uint8 output and 2D E8M0 block scales compatible with dequant_mxfp4() and V_MFMA_SCALE_F32_16X16X128_F8F6F4.

NOTE: The 2D scales layout is a stand-in. The optimized CDNA4 layout will likely be 6D (32x32 tiles) or 7D (16x16 tiles), mirroring how SM100 uses a 5D interleaved layout for its tensor core scale feed.

Parameters:

• ​out_dtype (DType): Element type of the quantized output tensor. Must be uint8 for packed MXFP4 (inferred).
• ​scales_dtype (DType): Element type of the block scale-factor tensor. Must be float8_e8m0fnu (inferred).
• ​in_dtype (DType): Element type of the input activation tensor. Must be bfloat16 (inferred).
• ​num_max_threads (Int): Maximum number of threads per block for the launch grid (defaults to 512).