For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
PrebuiltSchedule
struct PrebuiltSchedule
The reusable scheduler overlay, already on the device.
Lets a caller build the schedule once (host or device builder) and reuse
its overlay across prefill calls. Carries only the four tensors the fwd +
combine read off the reusable object -- meta (scheduler_metadata),
work_count, qsplit (the packed q-indices), split_counts. The CSR
proper (row_ptr) is NOT stored: reuse rebuilds it fresh each call and
substitutes this overlay, matching the reference (CSR fresh, overlay
reused). Sizes (total_rows, work_capacity) come from the fresh build,
not from here -- work_capacity is len(meta) // 6.
Reuse trusts the overlay: the caller MUST build it from the SAME q2k
selection, cu_seqlens, and shapes as the call it's passed to. The reuse
branch re-checks buffer lengths only, never the selection contents.
Fields
- meta (
DeviceBuffer[DType.int32]): - work_count (
DeviceBuffer[DType.int32]): - qsplit (
DeviceBuffer[DType.int32]): - split_counts (
DeviceBuffer[DType.int32]):
Implemented traits
AnyType,
Copyable,
ImplicitlyCopyable,
ImplicitlyDeletable,
Movable
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!