For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Python class
KVCacheInputs
KVCacheInputs
class max.nn.kv_cache.KVCacheInputs(inputs)
Bases: Generic[_Tensor, _Buffer], KVCacheInputsInterface[_Tensor, _Buffer]
Symbolic graph input types for a leaf KV cache.
This contains the KV cache inputs for all TP shards.
-
Parameters:
-
inputs (Sequence[KVCacheInputsPerDevice[_Tensor, _Buffer]])
flatten()
flatten()
Flattens this (sub)tree into a flattened buffer/tensor list.
-
Return type:
-
list[_Tensor | _Buffer]
inputs
inputs: Sequence[KVCacheInputsPerDevice[_Tensor, _Buffer]]
unflatten()
unflatten(it)
Rebuilds this (sub)tree by consuming values from it.
-
Parameters:
-
Return type:
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!