For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

quantize_dynamic_scaled_fp4_async

def quantize_dynamic_scaled_fp4_async[input_dtype: DType, output_dtype: DType, scales_dtype: DType, //, SF_VECTOR_SIZE: Int](ctx: DeviceContext, output_tensor_tile: TileTensor[output_dtype, Storage=output_tensor_tile.Storage, linear_idx_type=output_tensor_tile.linear_idx_type], scales_tensor_tile: TileTensor[scales_dtype, Storage=scales_tensor_tile.Storage, linear_idx_type=scales_tensor_tile.linear_idx_type], input_tensor_tile: TileTensor[input_dtype, Storage=input_tensor_tile.Storage, linear_idx_type=input_tensor_tile.linear_idx_type], tensor_sf: Float32 = 1)

Launches the TMA-based NVFP4 quantization kernel on SM100 hardware.

Sets up TMA tile descriptors for input, output, and scales, then enqueues the quantize_dynamic_scaled_async_fp4_kernel with warp-specialized PDL.

Parameters:

• ​input_dtype (DType): Element type of the input activation tensor. Must be bfloat16 (inferred).
• ​output_dtype (DType): Element type of the quantized output tensor. Must be uint8 for NVFP4 (inferred).
• ​scales_dtype (DType): Element type of the block scale-factor tensor. Must be float8_e4m3fn for NVFP4 (inferred).
• ​SF_VECTOR_SIZE (Int): Number of input elements covered by each block scale factor. Must be 16 for NVFP4.