module

encodings

Implementations of quantization encodings.

Aliases

• ​QK_K = 256: Size of superblock quantized elements, in bytes.
• ​K_SCALE_SIZE = 12: Size of superblock scales and mins, in bytes.

Structs

• ​BFloat16Encoding: The bfloat16 quantization encoding.
• ​Float32Encoding: The float32 quantization encoding.
• ​Q4_0Encoding: The Q4_0 quantization encoding.
• ​Q4_KEncoding: The Q4_K quantization encoding.
• ​Q6_KEncoding: The Q6_K quantization encoding.