For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Python class
KVConnectorType
KVConnectorType
class max.nn.kv_cache.KVConnectorType(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)
Identifies which off-device backing store the KV cache uses.
Set on KVCacheParams.kv_connector to control whether evicted
cache pages stay on device only, spill to host memory, tier across host
and disk, or route through a distributed block store.
dkv
dkv = 'dkv'
Routes pages through a distributed KV block store.
Requires a block_store_endpoint on the connector config.
local
local = 'local'
Spills evicted pages to host memory.
Requires enable_prefix_caching and host_kvcache_swap_space_gb
to be set on KVCacheParams.
null
null = 'null'
No off-device backing store. Pages live on device only.
tiered
tiered = 'tiered'
Tiers evicted pages across host memory and disk.
Requires enable_prefix_caching, host_kvcache_swap_space_gb,
and a disk_offload_dir on the connector config.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!