Skip to main content
Log in

Mojo package

kv_cache

Contains implementations for several types of key-value caches.

KV caches are used in transformer models to store key-value tensors output from self-attention layers.

These APIs are used in the higher-level functions in the nn package.

Modules

  • types: This module contains the types for the key-value cache APIs.