struct

Q6_KEncoding

The Q6_K quantization encoding.

Because this holds the quantized data in a special packing format, it currently does not print float values at runtime—it's just a bag of bits in uint8 format.

Implemented traits

AnyType, QuantizationEncoding

Methods

quantize

static quantize(_tensor: Tensor[float32]) -> Tensor[uint8]

Quantizes the full-precision tensor tensor to Q6_K.

The quantize method is not yet implemented. However, since Q6_K quantized ops are supported, Q6_KEncoding is still provided to allow code to be generic over quantization encoding type.

id

static id() -> String

Identifier for the Q6_K quantized encoding.