For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
Struct_rope_split_store_ragged_paged
struct Struct_rope_split_store_ragged_paged[interleaved: Bool]
Registers the mo.rope_split_store.ragged.paged graph op with the graph compiler.
Parameters
- interleaved (
Bool): Whether RoPE pairs adjacent real and imaginary components (real, imag, real, imag, ...) as in GGUF, rather than splitting them into halves (real, ..., real, imag, ..., imag) as in safetensors.
Implemented traits
Methods
execute
static def execute[out_dtype: DType, qkv_dtype: DType, freq_dtype: DType, cache_dtype: DType, //, target: StringSlice[ImmStaticOrigin]](output: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=output.static_spec], qkv: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=qkv.static_spec], input_row_offsets: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=input_row_offsets.static_spec], freqs_cis: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=freqs_cis.static_spec], kv_blocks: ManagedTensorSlice[IOSpec[_, _].MutableInput, static_spec=kv_blocks.static_spec], cache_lengths: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=cache_lengths.static_spec], kv_lookup_table: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=kv_lookup_table.static_spec], max_prompt_length: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=max_prompt_length.static_spec], max_cache_length: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=max_cache_length.static_spec], layer_idx: UInt32, ctx: DeviceContext)