Mojo module

buffers

Aliases

KBuffer

alias KBuffer[out_type: DType, in_type: DType, shape: IndexList[3], group_size: Int, transpose_b: Bool, //, tensor_core_mma: TiledTensorCore[out_type, in_type, shape, group_size, transpose_b], swizzle: OptionalReg[Swizzle], BN: Int, WN: Int, BK: Int, depth: Int, num_threads: Int, num_stages: Int = 1, token_gen: Bool = False] = KVBufferImpl[KBufferConfig[BN, BK, WN], tensor_core_mma, swizzle, BN, WN, BK, depth, num_threads, num_stages, token_gen]

Parameters

• ​out_type (DType):
• ​in_type (DType):
• ​shape (IndexList):
• ​group_size (Int):
• ​transpose_b (Bool):
• ​tensor_core_mma (TiledTensorCore):
• ​swizzle (OptionalReg):
• ​BN (Int):
• ​WN (Int):
• ​BK (Int):
• ​depth (Int):
• ​num_threads (Int):
• ​num_stages (Int):
• ​token_gen (Bool):

VBuffer

alias VBuffer[out_type: DType, in_type: DType, shape: IndexList[3], group_size: Int, transpose_b: Bool, //, tensor_core_mma: TiledTensorCore[out_type, in_type, shape, group_size, transpose_b], swizzle: OptionalReg[Swizzle], BN: Int, WN: Int, BK: Int, depth: Int, num_threads: Int, num_stages: Int = 1, token_gen: Bool = False] = KVBufferImpl[VBufferConfig[BN, BK, WN, depth], tensor_core_mma, swizzle, BN, WN, BK, depth, num_threads, num_stages, token_gen]

Parameters

• ​out_type (DType):
• ​in_type (DType):
• ​shape (IndexList):
• ​group_size (Int):
• ​transpose_b (Bool):
• ​tensor_core_mma (TiledTensorCore):
• ​swizzle (OptionalReg):
• ​BN (Int):
• ​WN (Int):
• ​BK (Int):
• ​depth (Int):
• ​num_threads (Int):
• ​num_stages (Int):
• ​token_gen (Bool):

Structs

• ​KBufferConfig:
• ​KVBufferImpl:
• ​OutputRegisterBuffer:
• ​PRegisterBuffer:
• ​QRegisterBuffer:
• ​VBufferConfig:
• ​VBufferTransposeLoads:

Traits

• ​KVBuffer:
• ​KVBufferConfig:
• ​RegisterBuffer:
• ​RegisterMMABuffer:

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit