For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

quantize_dynamic_scaled_fp4fp8

def quantize_dynamic_scaled_fp4fp8[out_dtype: DType, scales_dtype: DType, in_dtype: DType, //, *, SF_VECTOR_SIZE: Int = Int(16), num_max_threads: Int = Int(512)](ctx: DeviceContext, output_tile: TileTensor[out_dtype, Storage=output_tile.Storage, linear_idx_type=output_tile.linear_idx_type], scales_tile: TileTensor[scales_dtype, Storage=scales_tile.Storage, linear_idx_type=scales_tile.linear_idx_type], input_tile: TileTensor[in_dtype, Storage=input_tile.Storage, linear_idx_type=input_tile.linear_idx_type], num_cols: Int, num_cols_padded: Int, tensor_sf: Float32 = 1)

Launches the SM100 kernel that quantizes BF16 input to NVFP4, MXFP4, or MXFP8 with per-block scale factors.

Each thread processes ELEMENTS_PER_THREAD elements, cooperatively reduces the per-group maximum across warp lanes, derives the scale factor, and writes the quantized output and scales tensors.

Parameters:

• ​out_dtype (DType): Element type of the quantized output tensor. Must be uint8 for NVFP4 or MXFP4, or float8_e4m3fn for MXFP8 (inferred).
• ​scales_dtype (DType): Element type of the block scale-factor tensor. Must be float8_e4m3fn for NVFP4 or float8_e8m0fnu for MXFP4 and MXFP8 (inferred).
• ​in_dtype (DType): Element type of the input activation tensor. Must be bfloat16 (inferred).
• ​SF_VECTOR_SIZE (Int): Number of input elements covered by each block scale factor: 16 for NVFP4 or MXFP4, 32 for MXFP8 (defaults to 16).
• ​num_max_threads (Int): Maximum number of threads per block for the launch grid (defaults to 512).