For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
AMDStructuredConfig
struct AMDStructuredConfig[config: MHAConfig[config.dtype], group: Int, token_gen: Bool = False, mla_mode: Bool = False]
Holds the tile layout and indexing helpers for GFX950 structured attention.
Wraps an MHAConfig with group-query and MLA metadata, deriving the
comptime shared-memory decisions (shared_kv, full_kv,
depth_padded, double_buffer) and exposing query/key-value head and
tile index helpers used by the attention kernels.
Parameters
- config (
MHAConfig[config.dtype]): The base multi-head attention configuration. - group (
Int): Number of query heads sharing one key-value head. - token_gen (
Bool): Selects decode (True) versus prefill (False) tiling. - mla_mode (
Bool): Enables multi-latent attention tiling whenTrue.
Implemented traits
AnyType,
Copyable,
ImplicitlyCopyable,
ImplicitlyDeletable,
Movable
comptime members
depth_padded
comptime depth_padded = False
double_buffer
comptime double_buffer = not token_gen
double_buffer_k_only
comptime double_buffer_k_only = token_gen and (config.block_n() <= Int(64))
full_kv
comptime full_kv = True
shared_kv
comptime shared_kv = token_gen and (config > Int(256))
Methods
heads_per_tile
q_head_idx
q_tile_idx
kv_head_idx
get_mma_shape
get_q_offset
get_output_offset
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!