Skip to main content

Mojo struct

OffsetPosition

struct OffsetPosition[config: MLA_SM100_Decode_Config, KVLUTType: MHAOperand, ragged: Bool, is_cache_length_accurate: Bool, ValidLengthType: OptionalPointer, decoding_warp_split_k: Bool = False, sparse: Bool = False, has_extra_kv: Bool = False, has_variable_topk: Bool = False]

Fields​

  • ​seq_len (Int):
  • ​max_seq_len (Int):
  • ​num_keys (Int):
  • ​q_row_offset (Int):
  • ​out_row_offset (Int):
  • ​split_idx (Int):
  • ​batch_idx (Int):
  • ​kv_start_row (Int):
  • ​num_keys_this_split (Int):
  • ​q_token_idx (Int):

Implemented traits​

AnyType, Copyable, ImplicitlyCopyable, ImplicitlyDestructible, Movable, RegisterPassable, TrivialRegisterPassable

Methods​

__init__​

__init__(k: KVLUTType, valid_length: UnsafePointer[Scalar[ValidLengthType.dtype], ImmutAnyOrigin], max_seq_len: Int, num_partitions: Int, batch_size: Int, sparse_indices_stride: Int = 0, sparse_topk_lengths: OptionalReg[UnsafePointer[Int32, MutAnyOrigin]] = None, sparse_extra_indices_stride: Int = 0, sparse_extra_topk_lengths: OptionalReg[UnsafePointer[Int32, MutAnyOrigin]] = None) -> Self

cache_len​

cache_len(self) -> Int

Returns:

Int

start_pos​

start_pos(self, cache_start_pos: UInt32) -> UInt32

Returns:

UInt32

q_row_offset_at​

q_row_offset_at(self, q_local: Int) -> Int

Returns:

Int

out_row_offset_at​

out_row_offset_at(self, q_local: Int) -> Int

Returns:

Int

q_token_idx_at​

q_token_idx_at(self, q_local: Int) -> Int

Returns:

Int

Models

Cookbook

Puzzles

Blog

Community

Contact