For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo struct

MLA_Decode_Pack

struct MLA_Decode_Pack[ValidLengthType: OptionalPointer, MaskType: MHAMask, SplitAccumType: OptionalPointer]

Bundles the mask, valid-length, and split-K accumulator pointers passed to decode kernels.

Parameters

• ​ValidLengthType (OptionalPointer): OptionalPointer type wrapping the per-batch valid-sequence-length tensor (may be Null when unused).
• ​MaskType (MHAMask): MHAMask type applied to the attention scores.
• ​SplitAccumType (OptionalPointer): OptionalPointer type wrapping the split-K LSE accumulator buffer (may be Null when split-K is unused).

Fields

• ​mask (MaskType):
• ​valid_length (ValidLengthType):
• ​lse_accum_split_ptr (SplitAccumType):
• ​num_partitions (Int):

Implemented traits

AnyType, Copyable, DevicePassable, ImplicitlyCopyable, ImplicitlyDeletable, Movable, RegisterPassable, TrivialRegisterPassable

comptime members

device_type

comptime device_type = MLA_Decode_Pack[ValidLengthType, MaskType, SplitAccumType]

Methods

__init__

def __init__(mask: MaskType, valid_length: ValidLengthType, lse_accum_split_ptr: SplitAccumType, num_partitions: Int) -> Self

get_type_name

static def get_type_name() -> String

Returns:

String

get_device_type_name

static def get_device_type_name() -> String

Returns:

String

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit