Mojo struct
KVCacheStaticParams
struct KVCacheStaticParams
Fields
- num_heads (
UInt): - head_size (
UInt): - is_mla (
Bool):
Implemented traits
AnyType,
Copyable,
Equatable,
ImplicitlyCopyable,
ImplicitlyDestructible,
Movable,
RegisterPassable,
TrivialRegisterPassable
Methods
__init__
__init__(num_heads: UInt, head_size: UInt, is_mla: Bool = False) -> Self
Initialize KVCacheStaticParams. Args: num_heads (UInt): Number of attention heads. head_size (UInt): Size of each attention head. is_mla (Bool, optional): Whether to use Multi-Linear Attention (MLA) mode. If true, we only store k cache. If False, we store k and v cache. Defaults to False.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!