module
encodings
Implementations of quantization encodings.
Aliases
-
QK_K = 256
: Size of superblock quantized elements, in bytes. -
K_SCALE_SIZE = 12
: Size of superblock scales and mins, in bytes.
Structs
-
BFloat16Encoding
: The bfloat16 quantization encoding. -
Float32Encoding
: The float32 quantization encoding. -
Q4_0Encoding
: The Q4_0 quantization encoding. -
Q4_KEncoding
: The Q4_K quantization encoding. -
Q6_KEncoding
: The Q6_K quantization encoding.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!
If you'd like to share more information, please report an issue on GitHub
😔 What went wrong?