For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
KVCacheStaticParams
struct KVCacheStaticParams
Compile-time shape parameters shared across all layers of a KV cache.
Groups the attention-head count and per-head size that are fixed for the entire model lifetime, along with the Multi-head Latent Attention flag that changes the KV layout from two caches (K + V) to one (K only).
Fields
- num_heads (
Int): - head_size (
Int): - is_mla (
Bool):
Implemented traits
AnyType,
Copyable,
Equatable,
ImplicitlyCopyable,
ImplicitlyDeletable,
Movable,
RegisterPassable,
TrivialRegisterPassable
Methods
__init__
def __init__(num_heads: Int, head_size: Int, is_mla: Bool = False) -> Self
Initialize KVCacheStaticParams. Args: num_heads (Int): Number of attention heads. head_size (Int): Size of each attention head. is_mla (Bool, optional): Whether to use Multi-Linear Attention (MLA) mode. If true, we only store k cache. If False, we store k and v cache. Defaults to False.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!