For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo function

print_kv_cache_paged_generic_kernel_api

def print_kv_cache_paged_generic_kernel_api[dtype: DType, //, target: StringSlice[ImmStaticOrigin], kv_params: KVCacheStaticParams, page_size: Int](valid_lengths: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=valid_lengths.static_spec], kv_collection: PagedKVCacheCollection[dtype, kv_params, page_size, scale_dtype_=kv_collection.scale_dtype_, quantization_granularity_=kv_collection.quantization_granularity_], layer_idx: UInt32, is_print_compact: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=is_print_compact.static_spec], context: DeviceContext)

Prints the contents of a paged KV cache for debugging.

Parameters:

• ​dtype (DType): Element type of the KV cache blocks tensor (inferred).
• ​target (StringSlice[ImmStaticOrigin]): Target device identifier for kernel dispatch.
• ​kv_params (KVCacheStaticParams): Static KV cache parameters carrying num_heads and head_size.
• ​page_size (Int): Number of tokens stored per page in the paged KV cache.

Args:

• ​valid_lengths (ManagedTensorSlice[IOSpec[_, _].Input, static_spec=valid_lengths.static_spec]): One-dimensional tensor of shape [batch] giving the valid length of each sequence; only positions within these lengths are printed.
• ​kv_collection (PagedKVCacheCollection[dtype, kv_params, page_size, scale_dtype_=kv_collection.scale_dtype_, quantization_granularity_=kv_collection.quantization_granularity_]): Paged KV cache collection holding the keys and values to print; the cache for this layer is retrieved via layer_idx.
• ​layer_idx (UInt32): Index of the layer whose key and value caches are printed.
• ​is_print_compact (ManagedTensorSlice[IOSpec[_, _].Input, static_spec=is_print_compact.static_spec]): One-element boolean tensor; when element zero is true, abbreviates the output with ellipses (CPU only).
• ​context (DeviceContext): Device context used to copy device buffers to host on GPU targets.