For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo module

quantization

Registers quantization and dequantization graph ops backed by the quantization and linalg kernels.

`comptime` values

`logger`

comptime logger = Logger(stdout, prefix=String(""), source_location=False)

Structs

GGMLQ40Dequantize: Registers the ggml_q4_0_dequantize graph op with the graph compiler.
GGMLQ4KDequantize: Registers the ggml_q4_k_dequantize graph op with the graph compiler.
GGMLQ6KDequantize: Registers the ggml_q6_k_dequantize graph op with the graph compiler.
QMatmulGPU_b4_g128: Registers the qmatmul_b4_g128 graph op with the graph compiler.
QMatmulGPU_b4_g32: Registers the qmatmul_b4_g32 graph op with the graph compiler.
QMatmulGPURepackGGUF: Registers the GGUF_gpu_repack_q4_0 graph op with the graph compiler.
QMatmulGPURepackGPTQ_b4_g128: Registers the GPTQ_gpu_repack_b4_g128 graph op with the graph compiler.
QMatmulGPURepackGPTQ_b4_g128_desc_act: Registers the GPTQ_gpu_repack_b4_g128_desc_act graph op with the graph compiler.
QuantizeDynamicScaledFloat8: Registers the mo.quantize_dynamic_scaled_float8 graph op with the graph compiler.
QuantizeStaticScaledFloat8: Registers the mo.quantize_static_scaled_float8 graph op with the graph compiler.
QuantizeTensorDynamicScaledFloat8: Registers the mo.quantize_tensor_dynamic_scaled_float8 graph op with the graph compiler.
ResizeBicubic: Registers the mo.resize.bicubic graph op with the graph compiler.
ResizeLinear: Registers the mo.resize.linear graph op with the graph compiler.
ResizeNearest: Registers the mo.resize.nearest graph op with the graph compiler.
RMSNormFusedQuantizeDynamicScaledFP8: Registers the mo.composite.rms_norm_fused_quantize_dynamic_scaled_fp8 graph op with the graph compiler.
Struct_dequant_mxfp4: Registers the mo.dequant.mxfp4 graph op with the graph compiler.
Struct_grouped_quantize_dynamic_block_scaled: Registers the mo.grouped.quantize.dynamic.block.scaled graph op with the graph compiler.
Struct_interleave_block_scales: Registers the mo.interleave.block.scales graph op with the graph compiler.
Struct_mxfp4_preshuffle_b_5d: Run the AMD CDNA4 MXFP4 B 5D preshuffle as a custom op.
Struct_mxfp4_preshuffle_scale_4d_per_expert: Per-step A-scale preshuffle for the AMD CDNA4 preb grouped matmul.
Struct_quantize_dynamic_block_scaled: Registers the mo.quantize.dynamic.block.scaled graph op with the graph compiler.
Struct_quantize_dynamic_block_scaled_mxfp4: Registers the mo.quantize.dynamic.block.scaled.mxfp4 graph op with the graph compiler.
Struct_unfused_qkv_matmul_ragged_paged_gguf_quantized: Registers the mo.unfused_qkv_matmul.ragged.paged.gguf_quantized graph op with the graph compiler.
VroomQ40Matmul: Registers the vroom_q4_0_matmul graph op with the graph compiler.
VroomQ40RepackWeights: Registers the vroom_q4_0_repack_weights graph op with the graph compiler.
VroomQ4KMatmul: Registers the vroom_q4_k_matmul graph op with the graph compiler.
VroomQ4KRepackWeights: Registers the vroom_q4_k_repack_weights graph op with the graph compiler.
VroomQ6KMatmul: Registers the vroom_q6_k_matmul graph op with the graph compiler.
VroomQ6KRepackWeights: Registers the vroom_q6_k_repack_weights graph op with the graph compiler.

Functions

composite_rms_norm_fused_quantize_dynamic_scaled_fp8_shape: Computes the output shapes for the fused RMS norm and dynamic scaled FP8 quantization op.
ggml_q4_0_dequantize_shape: Computes the output shape for the ggml_q4_0_dequantize graph op.
ggml_q4_k_dequantize_shape: Computes the output shape for the ggml_q4_k_dequantize graph op.
ggml_q6_k_dequantize_shape: Computes the output shape for the ggml_q6_k_dequantize graph op.
GGUF_gpu_repack_q4_0_shape: Computes the output shape for the GGUF_gpu_repack_q4_0 graph op.
GPTQ_gpu_repack_b4_g128_desc_act_shape: Computes the output shape for the GPTQ_gpu_repack_b4_g128_desc_act graph op.
GPTQ_gpu_repack_b4_g128_shape: Computes the output shape for the GPTQ_gpu_repack_b4_g128 graph op.
qmatmul_b4_g128_shape: Computes the output shape for the qmatmul_b4_g128 graph op.
qmatmul_b4_g32_shape: Computes the output shape for the qmatmul_b4_g32 graph op.
resize_bicubic_shape: Computes the output shape for the mo.resize.bicubic graph op.
resize_linear_shape: Computes the output shape for the mo.resize.linear graph op.
resize_nearest_shape: Computes the output shape for the mo.resize.nearest graph op.
vroom_q4_0_matmul_shape: Computes the output shape for the vroom_q4_0_matmul graph op.
vroom_q4_0_repack_weights_shape: Computes the output shape for the vroom_q4_0_repack_weights graph op.
vroom_q4_k_matmul_shape: Computes the output shape for the vroom_q4_k_matmul graph op.
vroom_q4_k_repack_weights_shape: Computes the output shape for the vroom_q4_k_repack_weights graph op.
vroom_q6_k_matmul_shape: Computes the output shape for the vroom_q6_k_matmul graph op.
vroom_q6_k_repack_weights_shape: Computes the output shape for the vroom_q6_k_repack_weights graph op.

comptime values​

logger​

Structs​

Functions​

`comptime` values

`logger`

Structs

Functions