Mojo module

buffers

comptime values

KBuffer

comptime KBuffer[kv_tile_layout: TensorLayout, tensor_core_mma: TiledTensorCore[out_type, in_type, shape, group_size, transpose_b], swizzle: Optional[Swizzle], BN: Int, WN: Int, BK: Int, depth: Int, num_threads: Int, num_stages: Int = 1, token_gen: Bool = False] = KVBufferImpl[KBufferConfig[BN, BK, WN], tensor_core_mma, swizzle, BN, WN, BK, depth, num_threads, num_stages, token_gen]

Parameters

• ​kv_tile_layout (TensorLayout):
• ​tensor_core_mma (TiledTensorCore):
• ​swizzle (Optional):
• ​BN (Int):
• ​WN (Int):
• ​BK (Int):
• ​depth (Int):
• ​num_threads (Int):
• ​num_stages (Int):
• ​token_gen (Bool):

KBufferLDS

comptime KBufferLDS[kv_tile_layout: TensorLayout, tensor_core_mma: TiledTensorCore[out_type, in_type, shape, group_size, transpose_b], swizzle: Optional[Swizzle], BN: Int, WN: Int, BK: Int, depth: Int, num_threads: Int, token_gen: Bool = False] = KVBufferLDS[KBufferConfig[BN, BK, WN], tensor_core_mma, swizzle, BN, WN, BK, depth, num_threads, token_gen]

Parameters

• ​kv_tile_layout (TensorLayout):
• ​tensor_core_mma (TiledTensorCore):
• ​swizzle (Optional):
• ​BN (Int):
• ​WN (Int):
• ​BK (Int):
• ​depth (Int):
• ​num_threads (Int):
• ​token_gen (Bool):

VBuffer

comptime VBuffer[kv_tile_layout: TensorLayout, tensor_core_mma: TiledTensorCore[out_type, in_type, shape, group_size, transpose_b], swizzle: Optional[Swizzle], BN: Int, WN: Int, BK: Int, depth: Int, num_threads: Int, num_stages: Int = 1, token_gen: Bool = False] = KVBufferImpl[VBufferConfig[BN, BK, WN, depth], tensor_core_mma, swizzle, BN, WN, BK, depth, num_threads, num_stages, token_gen]

Parameters

• ​kv_tile_layout (TensorLayout):
• ​tensor_core_mma (TiledTensorCore):
• ​swizzle (Optional):
• ​BN (Int):
• ​WN (Int):
• ​BK (Int):
• ​depth (Int):
• ​num_threads (Int):
• ​num_stages (Int):
• ​token_gen (Bool):

Structs

• ​KBufferConfig:
• ​KVBufferImpl:
• ​KVBufferLDS: KV buffer with direct DRAM→LDS DMA (no register staging).
• ​OutputRegisterBuffer:
• ​PRegisterBuffer:
• ​QRegisterBuffer:
• ​VBufferConfig:
• ​VBufferTransposeLoads:

Traits

• ​KVBuffer:
• ​KVBufferConfig:
• ​RegisterBuffer:
• ​RegisterMMABuffer:

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit