For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

LinearLoRA

LinearLoRA

class max.nn.LinearLoRA(in_dim, out_dim, max_num_loras, max_lora_rank, dtype, device, has_lora_bias=False, name=None, quantization_encoding=None)

source

Bases: Module, SupportsLoRA

Applies a linear transformation and LoRA to input:

$y_l = (xA^T) @ B^T$. $y = (xW^T + b) + y_l$

from max.driver import Accelerator, CPU, accelerator_count
from max.dtype import DType
from max.graph import DeviceRef
from max.nn.lora import LinearLoRA

device = Accelerator() if accelerator_count() > 0 else CPU()
device_ref = DeviceRef.from_device(device)

linear_layer = LinearLoRA(
    in_dim=256,
    out_dim=128,
    max_lora_rank=16,
    max_num_loras=100,
    dtype=DType.float32,
    device=device_ref,
    has_lora_bias=True,
    name="lora_linear"
)

Parameters:

• in_dim (int)
• out_dim (int)
• max_num_loras (int)
• max_lora_rank (int)
• dtype (DType)
• device (DeviceRef)
• has_lora_bias (bool)
• name (str | None)
• quantization_encoding (QuantizationEncoding | None)

set_lora_batch_info()

set_lora_batch_info(lora_ids, lora_ranks, lora_grouped_offsets, num_active_loras, lora_end_idx, batch_seq_len)

source

Parameters:

• lora_ids (TensorValue)
• lora_ranks (TensorValue)
• lora_grouped_offsets (TensorValue)
• num_active_loras (TensorValue)
• lora_end_idx (TensorValue)
• batch_seq_len (TensorValue)

Return type:

None