IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /max/get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo struct

AMDStructuredConfig

struct AMDStructuredConfig[config: MHAConfig[config.dtype], group: Int, token_gen: Bool = False, mla_mode: Bool = False]

Implemented traits​

AnyType, Copyable, ImplicitlyCopyable, ImplicitlyDeletable, Movable

comptime members​

depth_padded​

comptime depth_padded = False

double_buffer​

comptime double_buffer = not token_gen

double_buffer_k_only​

comptime double_buffer_k_only = token_gen and (config.block_n() <= Int(64))

full_kv​

comptime full_kv = True

shared_kv​

comptime shared_kv = token_gen and (config > Int(256))

Methods​

heads_per_tile​

static def heads_per_tile() -> Int

Returns:

Int

q_head_idx​

static def q_head_idx() -> Int

Returns:

Int

q_tile_idx​

static def q_tile_idx() -> Int

Returns:

Int

kv_head_idx​

static def kv_head_idx() -> Int

Returns:

Int

get_mma_shape​

static def get_mma_shape() -> IndexList[Int(3)]

Returns:

IndexList[Int(3)]

get_q_offset​

static def get_q_offset[q_depth: Int]() -> UInt32

Returns:

UInt32

get_output_offset​

static def get_output_offset[output_depth: Int]() -> UInt32

Returns:

UInt32

Models

Cookbook

Puzzles

Blog

Community

Contact