For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo function

quantize_dynamic_block_scaled

def quantize_dynamic_block_scaled[out_dtype: DType, scales_dtype: DType, in_dtype: DType, //, *, SF_VECTOR_SIZE: Int, target: StringSlice[ImmStaticOrigin] = StringSlice("cpu")](output_device: TileTensor[out_dtype, Storage=output_device.Storage, linear_idx_type=output_device.linear_idx_type], scales_device: TileTensor[scales_dtype, Storage=scales_device.Storage, linear_idx_type=scales_device.linear_idx_type], input_device: TileTensor[in_dtype, Storage=input_device.Storage, linear_idx_type=input_device.linear_idx_type], tensor_sf: Float32, ctx: DeviceContext)

Dispatches dynamic block-scaled quantization to the appropriate hardware-specific kernel.

Routes to the SM100 NVFP4 TMA-async path, the SM100 quantize_dynamic_scaled_fp4fp8 path, or the AMD CDNA4 MXFP4 path based on the target device and scale-factor format.

Parameters:

out_dtype (DType): Element type of the quantized output tensor (inferred). uint8 for NVFP4/MXFP4 or float8_e4m3fn for MXFP8.
scales_dtype (DType): Element type of the block scale-factor tensor (inferred). float8_e4m3fn for NVFP4 or float8_e8m0fnu for MXFP8.
in_dtype (DType): Element type of the input activation tensor (inferred). Must be bfloat16.
SF_VECTOR_SIZE (Int): Number of elements covered by each block scale factor: 16 for NVFP4 or 32 for MXFP8.
target (StringSlice[ImmStaticOrigin]): Trace target string identifying the device in profiling output (defaults to "cpu").