For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
DecodeKVProducer
struct DecodeKVProducer[dtype: DType, config: MLA_SM100_Decode_Config, num_producer: Int = Int(1)]
Fields
- pipe (
DecodeKVProducer[dtype, config, num_producer].KVPipeType): - smem (
UnsafePointer[Scalar[dtype], MutAnyOrigin, address_space=AddressSpace.SHARED]):
Implemented traits
AnyType,
Copyable,
ImplicitlyCopyable,
ImplicitlyDeletable,
Movable,
RegisterPassable,
TrivialRegisterPassable
comptime members
kv_stage_bytes
comptime kv_stage_bytes = (Int((mul config.BN_QK, config.q_depth)) * size_of[dtype]())
kv_stage_elems
comptime kv_stage_elems = (config * config)
KVPipeType
comptime KVPipeType = KVPipelineGeneric[config.num_kv_stages, Int(1), num_producer, Int(2)]
Methods
__init__
def __init__(pipe: KVPipelineGeneric[config.num_kv_stages, Int(1), num_producer, Int(2)], smem: UnsafePointer[Scalar[dtype], MutAnyOrigin, address_space=AddressSpace.SHARED]) -> Self
init
def init(self)
stage_base_ptr
def stage_base_ptr[*, qk_stage: Int = Int(0)](self) -> UnsafePointer[Scalar[dtype], MutAnyOrigin, address_space=AddressSpace.SHARED]
Returns:
UnsafePointer[Scalar[dtype], MutAnyOrigin, address_space=AddressSpace.SHARED]
stage_index
producer_mbar
def producer_mbar[*, qk_stage: Int = Int(0)](self) -> MBarType
Returns:
MBarType
acquire
def acquire[*, qk_stage: Int = Int(0)](self)
commit_step
def commit_step(mut self)
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!