struct
Q4_KEncoding
The Q4_K quantization encoding.
Because this holds the quantized data in a special packing format, it currently does not print float values at runtime—it's just a bag of bits in uint8 format.
Implemented traits
AnyType
,
QuantizationEncoding
Methods
quantize
static quantize(_tensor: Tensor[float32]) -> Tensor[uint8]
Quantizes the full-precision tensor tensor
to Q4_K.
The quantize method is not yet implemented. However, since Q4_K quantized ops are supported, Q4_KEncoding is still provided to allow code to be generic over quantization encoding type.
id
static id() -> String
Identifier for the Q4_K quantized encoding.