For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo function

mla_prefill_branch_sparse_bf16

def mla_prefill_branch_sparse_bf16[collection_t: KVCollectionT, //, kv_input_fn: def[width: Int](IndexList[Int(2)]) capturing thin -> SIMD[DType.bfloat16, width], indices_stride: Int, target: StringSlice[ImmStaticOrigin] = StringSlice("cpu")](output: TileTensor[DType.bfloat16, Storage=output.Storage, linear_idx_type=output.linear_idx_type], q: TileTensor[DType.bfloat16, Storage=q.Storage, linear_idx_type=q.linear_idx_type], input_row_offsets: TileTensor[DType.uint32, Storage=input_row_offsets.Storage, linear_idx_type=input_row_offsets.linear_idx_type], freqs_cis: TileTensor[Storage=freqs_cis.Storage, linear_idx_type=freqs_cis.linear_idx_type], kv_norm_gamma: TileTensor[Storage=kv_norm_gamma.Storage, linear_idx_type=kv_norm_gamma.linear_idx_type], kv_collection: collection_t, layer_idx: UInt32, scale: Float32, epsilon: Float32, w_uk: TileTensor[DType.bfloat16, Storage=w_uk.Storage, linear_idx_type=w_uk.linear_idx_type], w_uv: TileTensor[DType.bfloat16, Storage=w_uv.Storage, linear_idx_type=w_uv.linear_idx_type], ctx: DeviceContext, d_indices: Pointer[Int32, MutAnyOrigin, _safe=False], topk_lengths: Pointer[Int32, MutAnyOrigin, _safe=False], attn_sink_ptr: OptionalReg[Pointer[Float32, MutAnyOrigin, _safe=False]])

Sparse MLA prefill branch (DSv3.2/GLM absorbed shape, BF16 weights).

BF16 analogue of mla_prefill_branch_sparse_fp8: reuses mla_decode_branch_bf16's absorbed-Q construction (q_nope up-proj via w_uk

RoPE/RMSNorm) and the identical w_uv output up-projection, and swaps the attention call to the existing mla_sm100_prefill_sparse kernel over the paged BF16 latent cache. The caller (.sparse op) has already remapped d_indices from logical to physical rows, so they are passed straight through. Only supported for a BF16 KV cache.