IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /max/get-started.md). For the complete documentation index, see llms.txt.

Skip to main content

For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Python class

KVCacheInputs

`KVCacheInputs`

class max.nn.kv_cache.KVCacheInputs(inputs)

source

Bases: Generic[_Tensor, _Buffer]

Symbolic graph input types for all devices’ paged KV cache.

Parameters:: inputs (Sequence[KVCacheInputsPerDevice[_Tensor, _Buffer]])

`flatten()`

flatten()

source

Return type:: list[_Tensor | _Buffer]

`inputs`

inputs: Sequence[KVCacheInputsPerDevice[_Tensor, _Buffer]]

source

`unflatten()`

unflatten(it)

source

Parameters:: it (Iterator[Any])
Return type:: KVCacheInputs[TensorValue, BufferValue]

KVCacheInputs