For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo function

grouped_quantize_dynamic_scaled_fp4_async

def grouped_quantize_dynamic_scaled_fp4_async[input_dtype: DType, output_dtype: DType, scales_dtype: DType, //](output_tensor: TileTensor[output_dtype, Storage=output_tensor.Storage, linear_idx_type=output_tensor.linear_idx_type], scales_tensor: TileTensor[scales_dtype, Storage=scales_tensor.Storage, linear_idx_type=scales_tensor.linear_idx_type], input_tensor: TileTensor[input_dtype, Storage=input_tensor.Storage, linear_idx_type=input_tensor.linear_idx_type], row_offsets: TileTensor[DType.uint32, Storage=row_offsets.Storage, linear_idx_type=row_offsets.linear_idx_type], scales_offsets: TileTensor[DType.uint32, Storage=scales_offsets.Storage, linear_idx_type=scales_offsets.linear_idx_type], expert_ids: TileTensor[DType.int32, Storage=expert_ids.Storage, linear_idx_type=expert_ids.linear_idx_type], sf_tensor: TileTensor[DType.float32, Storage=sf_tensor.Storage, linear_idx_type=sf_tensor.linear_idx_type], ctx: DeviceContext)

Launches the grouped per-expert quantization kernel for NVFP4/MXFP4/MXFP8 on SM100 hardware.

Sets up the TMA scale-factor descriptor and enqueues the grouped_quantize_dynamic_scaled_fp4_async_kernel over a grid spanning the scale-factor tile dimensions.

Parameters:

input_dtype (DType): Element type of the input activation tensor (inferred).
output_dtype (DType): Element type of the quantized output tensor (inferred).
scales_dtype (DType): Element type of the block scale-factor tensor (inferred).