Python function

attention_dispatch_metadata

attention_dispatch_metadata()

max.nn.kv_cache.attention_dispatch_metadata(kv_collection, *, device_idx=None)

source

Extracts the AttentionDispatchMetadata from a KV collection.

Parameters:

• kv_collection (PagedCacheValues) – The paged KV cache values to extract metadata from.
• device_idx (int | None) – Optional device index included in the error message when dispatch_metadata is None.

Returns:

The AttentionDispatchMetadata stored on the collection.

Raises:

ValueError – If kv_collection.dispatch_metadata is None.

Return type:

AttentionDispatchMetadata[TensorValue]

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit