Python class
KVTransferEngineMetadata
KVTransferEngineMetadata
class max.kv_cache.KVTransferEngineMetadata(*, name, total_num_pages, bytes_per_page, memory_type, hostname, agents_meta)
Bases: Struct
Metadata associated with a transfer engine.
This is safe to send between threads/processes.
-
Parameters:
agents_meta
[replica][tp_shard].
-
Type:
-
Metadata for each replica’s agents
bytes_per_page
bytes_per_page: int
Bytes per page for each tensor.
hostname
hostname: str
Hostname of the machine that the transfer engine is running on.
memory_type
memory_type: MemoryType
Memory type of the transfer engine.
name
name: str
Base name of the transfer engine.
total_num_pages
total_num_pages: int
Total number of pages in each tensor.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!