Python function
load_multi_kv_managers
load_multi_kv_managers()
max.kv_cache.load_multi_kv_managers(params, max_batch_size, max_seq_len, session, available_cache_memory)
Loads a list of KV cache managers from the given params.
Deprecated
Deprecated since version Use: load_kv_manager(params) instead, which returns a single
PagedKVCacheManager that natively handles multiple caches.
This function is retained for backward compatibility with
speculative decoding.
-
Parameters:
-
- params (MultiKVCacheParams)
- max_batch_size (int | None)
- max_seq_len (int)
- session (InferenceSession)
- available_cache_memory (int | None)
-
Return type:
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!