For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
MHADecodeDispatchMetadata
struct MHADecodeDispatchMetadata
Runtime metadata required to dispatch an MHA decode kernel launch.
Bundles the batch size, maximum query sequence length, split-K partition
count, and maximum cache length so that callers can construct the correct
grid shape for the decode kernel without recomputing partition counts.
Use from_runtime_values() to construct this from raw scalars.
Fields
- batch_size (
Int): - q_max_seq_len (
Int): - num_partitions (
Int): - max_cache_valid_length (
Int):
Implemented traits
AnyType,
Copyable,
ImplicitlyCopyable,
ImplicitlyDeletable,
Movable,
RegisterPassable,
TrivialRegisterPassable
Methods
from_runtime_values
static def from_runtime_values[num_heads: Int, group: Int](batch_size: Int, q_max_seq_len: Int, max_cache_valid_length: Int, ctx: DeviceContext) -> Self
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!