Skip to main content

struct

Q4_KEncoding

The Q4_K quantization encoding.

Implemented traits

AnyType, QuantizationEncoding

Methods

quantize

static quantize(_tensor: Tensor[float32]) -> Tensor[uint8]

Quantizes the full-precision tensor tensor to Q4_K.

The quantize method is not yet implemented. However, since Q4_K quantized ops are supported, Q4_KEncoding is still provided to allow code to be generic over quantization encoding type.

id

static id() -> String

Identifier for the Q4_K quantized encoding.