For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

MXFormat

struct MXFormat

Names the element encoding of an OCP microscaling (MX) operand.

An MX operand is a block of elements sharing one float8_e8m0fnu scale; this type names the encoding of the elements, which is all that varies between MXFP4, MXFP6 and MXFP8.

Deliberately not tied to any vendor: AMD CDNA4 selects these through its f8f6f4 MFMA selector and NVIDIA through its own block-scaled MMA, so the numeric values here are ours and each backend maps them to its own encoding (see CDNA4F8F6F4MatrixFormat.__init__ for the CDNA4 mapping). Kernels that are not hardware-specific -- EP dispatch packing, quantization, layout math -- should carry this type rather than a vendor selector.

Prefer deriving it from a dtype with from_dtype, so the format travels with the data instead of being inferred at each call site.

Implemented traits

AnyType, Copyable, Deinitable, Equatable, ImplicitlyCopyable, Movable, RegisterPassable, TrivialRegisterPassable

comptime members

FP4_E2M1

comptime FP4_E2M1 = MXFormat(Int(4))

FP6_E2M3

comptime FP6_E2M3 = MXFormat(Int(2))

FP6_E3M2

comptime FP6_E3M2 = MXFormat(Int(3))

FP8_E4M3

comptime FP8_E4M3 = MXFormat(Int(0))

FP8_E5M2

comptime FP8_E5M2 = MXFormat(Int(1))

Methods

__init__

def __init__(value: Int) -> Self

__eq__

def __eq__(self, other: Self) -> Bool

Returns:

Bool

from_dtype

static def from_dtype[dtype: DType]() -> Self

Returns the MX format a quantized operand dtype denotes.

Parameters:

• ​dtype (DType): The storage dtype of the quantized elements.

Returns:

Self: The corresponding MX element format.

from_name

static def from_name[name: StringSpan[ImmStaticOrigin]]() -> Self

Returns the MX format a graph-parameter name denotes.

Parameters:

• ​name (StringSpan[ImmStaticOrigin]): One of mxfp4, mxfp6, mxfp6_e3m2, mxfp8, mxfp8_e5m2. The bare mxfp6 and mxfp8 select the encoding each format defaults to, E2M3 and E4M3.

Returns:

Self: The corresponding MX element format.

bits_per_element

def bits_per_element(self) -> Int

Returns how many bits one element of this format occupies.

This is the payload width, not the fragment width: 32 FP6 elements occupy 24 bytes but travel in a wider fragment on some hardware.

Returns:

Int: The element width in bits.

is_fp6

def is_fp6(self) -> Bool

Returns whether this is one of the two six-bit encodings.

Returns:

Bool: True for FP6_E2M3 and FP6_E3M2.

fp6_format

def fp6_format(self) -> FP6Format

Returns the FP6 element encoding, for the FP6 packing routines.

Both FP6 encodings occupy six bits and pack four codes into three bytes, so nothing downstream can tell them apart from the bytes alone -- this format is the only record of which one they hold.

Returns:

FP6Format: The matching FP6Format.