For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Python class
ModelConfigWithKVCache
ModelConfigWithKVCache
class max.pipelines.kv_cache.ModelConfigWithKVCache(*args, **kwargs)
Bases: ModelConfig, Protocol
Extension of ModelConfig for models with a KV cache.
Adds get_kv_params so planners that need cache parameters can work
with a typed interface instead of getattr lookups.
get_kv_params()
get_kv_params()
Returns the KV cache parameters for this model.
-
Return type:
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!