Mojo function
generic_fused_qk_rope_bshd_continuous_batch_ragged
generic_fused_qk_rope_bshd_continuous_batch_ragged[dtype: DType, freq_dtype: DType, //, *, interleaved: Bool, has_position_ids: Bool, target: StringSlice[StaticConstantOrigin], mrope_section: Optional[IntTuple] = Optional(None)](q_proj: NDBuffer[dtype, 3, origin, shape, strides], input_row_offsets: NDBuffer[uint32, 1, origin, shape, strides], kv_collection: ContinuousBatchingKVCacheCollection[dtype_, kv_params_], freqs_cis: NDBuffer[freq_dtype, 2, origin, shape, strides], position_ids: NDBuffer[uint32, 2, origin, shape, strides], layer_idx: SIMD[uint32, 1], output: NDBuffer[dtype, 3, origin, shape, strides], context: DeviceContextPtr)
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!