Mojo struct
KVCacheStaticParams
@register_passable(trivial)
struct KVCacheStaticParams
Fields
- num_heads (
UInt): - head_size (
UInt): - is_mla (
Bool):
Implemented traits
AnyType,
Copyable,
EqualityComparable,
ImplicitlyCopyable,
Movable,
UnknownDestructibility
Aliases
__copyinit__is_trivial
alias __copyinit__is_trivial = True
__del__is_trivial
alias __del__is_trivial = True
__moveinit__is_trivial
alias __moveinit__is_trivial = True
Methods
__init__
__init__(num_heads: UInt, head_size: UInt, is_mla: Bool = False) -> Self
Initialize KVCacheStaticParams. Args: num_heads (UInt): Number of attention heads. head_size (UInt): Size of each attention head. is_mla (Bool, optional): Whether to use Multi-Linear Attention (MLA) mode. If true, we only store k cache. If False, we store k and v cache. Defaults to False.
__eq__
__ne__
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!