Python class
DummyKVCache
DummyKVCache
class max.pipelines.kv_cache.DummyKVCache(*args, **kwargs)
Bases: PagedKVCacheManager
No-op KV cache implementation for testing or when cache is disabled.
Initializes the dummy cache with a single replica and no host swapping.
alloc()
alloc(*args, **kwargs)
No-op.
claim()
claim(request_id, replica_idx)
No-op.
contains()
contains(request_id, replica_idx)
Returns True for any request.
get_metrics()
get_metrics(replica_idx)
Returns empty metrics.
-
Parameters:
-
replica_idx (int)
-
Return type:
get_num_disk_pages()
get_num_disk_pages(replica_idx)
Returns 1.
get_num_host_pages()
get_num_host_pages(replica_idx)
Returns 1.
get_num_pages()
get_num_pages(replica_idx)
Returns 1.
get_num_used_disk_pages()
get_num_used_disk_pages(replica_idx)
Returns 1.
get_num_used_host_pages()
get_num_used_host_pages(replica_idx)
Returns 1.
get_num_used_pages()
get_num_used_pages(replica_idx)
Returns 1.
get_pct_used_blocks_after_allocation()
get_pct_used_blocks_after_allocation(*args, **kwargs)
Returns a fixed low percentage (0.01).
release()
release(request_id, replica_idx)
No-op.
reset_metrics()
reset_metrics()
No-op.
-
Return type:
-
None
step()
step(*args, **kwargs)
No-op.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!