Skip to main content

struct

Q6_KEncoding

The Q6_K quantization encoding.

Implemented traits

AnyType, QuantizationEncoding

Methods

quantize

static quantize(_tensor: Tensor[float32]) -> Tensor[uint8]

Quantizes the full-precision tensor tensor to Q6_K.

The quantize method is not yet implemented. However, since Q6_K quantized ops are supported, Q6_KEncoding is still provided to allow code to be generic over quantization encoding type.

id

static id() -> String

Identifier for the Q6_K quantized encoding.