For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Python class

MAXModelConfig

MAXModelConfig

class max.pipelines.MAXModelConfig(*, config_file=None, section_name=None, use_subgraphs=True, data_parallel_degree=1, pool_embeddings=True, max_length=None, model_path='', served_model_name=None, weight_path=<factory>, quantization_encoding=None, huggingface_model_revision='main', huggingface_weight_revision='main', trust_remote_code=False, subfolder=None, device_specs=<factory>, force_download=False, vision_config_overrides=<factory>, rope_type=None, sliding_window=None, enable_echo=False, chat_template=None, kv_cache=<factory>)

source

Bases: MAXModelConfigBase

Configuration for a pipeline model.

Initialize, seeding private attrs and resolving the weight path.

Private attributes (PrivateAttr) aren’t accepted as constructor kwargs by default, so we pop the seeded ones (_huggingface_config, _weights_repo_id) here, then resolve the weight-path identity eagerly.

Parameters:

• config_file (str | None)
• section_name (str | None)
• use_subgraphs (bool)
• data_parallel_degree (int)
• pool_embeddings (bool)
• max_length (int | None)
• model_path (str)
• served_model_name (str | None)
• weight_path (list[Path])
• quantization_encoding (Literal['float32', 'float16', 'bfloat16', 'q4_k', 'q4_0', 'q6_k', 'float8_e4m3fn', 'float4_e2m1fnx2', 'gptq'] | None)
• huggingface_model_revision (str)
• huggingface_weight_revision (str)
• trust_remote_code (bool)
• subfolder (str | None)
• device_specs (list[DeviceSpec])
• force_download (bool)
• vision_config_overrides (dict[str, Any])
• rope_type (Literal['none', 'normal', 'neox', 'longrope', 'yarn'] | None)
• sliding_window (int | None)
• enable_echo (bool)
• chat_template (Path | None)
• kv_cache (KVCacheConfig)

architecture_name

property architecture_name: str | None

source

Returns the architecture class name from the HuggingFace config.

For transformers models, returns architectures[0] from the HuggingFace config.

chat_template

chat_template: Path | None

source

An optional custom chat template to override the one shipped with the model.

data_parallel_degree

data_parallel_degree: int

source

The degree of data parallelism for replicating the model.

default_device_spec

property default_device_spec: DeviceSpec

source

Returns the default device spec for the model.

This is the first device spec in the list, used for device spec checks throughout config validation.

Returns:

The default device spec for the model.

device_specs

device_specs: list[DeviceSpec]

source

The devices to run inference on.

enable_echo

enable_echo: bool

source

Whether the model should be built with echo capabilities.

force_download

force_download: bool

source

Whether to force download a file even if it’s already in the local cache.

from_pipeline_args()

classmethod from_pipeline_args(args)

source

Builds a MAXModelConfig from a PipelineArgs’s flat fields.

Returns a new object on every call – args holds no live handle back to it, so mutating the returned object (e.g. MAXModelConfig.from_pipeline_args(args).foo = x) has no effect on a subsequent call with the same args. Set the corresponding field on args itself instead.

Parameters:

args (PipelineArgs)

Return type:

Self

generation_config

property generation_config: GenerationConfig

source

Returns the Hugging Face GenerationConfig for this model.

Loaded once at construction (see _populate_generation_config()) and stored in a PrivateAttr; this getter returns it, falling back to loading on demand for a never-populated config (e.g. a placeholder). Loading failures yield a default GenerationConfig.

huggingface_config

property huggingface_config: PreTrainedConfig

source

Returns the Hugging Face model config.

Loaded once at construction (see _populate_hf_config()) and stored in a PrivateAttr; this getter returns it. Falls back to loading on demand – without caching – for a never-populated config (e.g. a placeholder with no model_path).

For transformers models this is the AutoConfig subclass; for non-transformers models (e.g. diffusers components) it is the raw config.json wrapped in a PretrainedConfig.

Raises:

FileNotFoundError – If no config.json can be found for the model repo/subfolder.

huggingface_model_repo

property huggingface_model_repo: HuggingFaceRepo

source

Returns the Hugging Face repo handle for the model.

Built once at construction (see _populate_repo_handles()) and stored in a PrivateAttr; this getter returns it. Falls back to building a fresh handle only for a never-populated config and never writes back.

huggingface_model_revision

huggingface_model_revision: str

source

The branch or Git revision of the Hugging Face model repository.

huggingface_weight_repo

property huggingface_weight_repo: HuggingFaceRepo

source

Returns the Hugging Face repo handle for weight files.

Built once at construction (see _populate_repo_handles()) and stored in a PrivateAttr; this getter returns it. Falls back to building a fresh handle only for a never-populated config (e.g. a placeholder with no model_path) and never writes back.

huggingface_weight_repo_id

property huggingface_weight_repo_id: str

source

Returns the Hugging Face repo ID used for weight files.

huggingface_weight_revision

huggingface_weight_revision: str

source

The branch or Git revision of the Hugging Face weights repository.

kv_cache

kv_cache: KVCacheConfig

source

The KV cache configuration.

loader()

loader()

source

Returns a WeightLoader over this config’s weights.

The loader’s namespace is the raw parameter names from the source files (un-prefixed). Pass this directly to a single-model pipeline’s Module tree; for multi-component pipelines, use loader() which exposes the role-prefixed union across configs.

Resolution is lazy: the safetensors mmap stays cold for parameters the Module never asks for. Inherits the HuggingFace download side-effect from resolved_weight_paths() for online repos.

Resolves quantization_encoding/weight_path on demand (see _resolve_component_encoding_and_weights()) rather than assuming resolution already populated them – a no-op when they’re already set.

Returns an empty loader when there are no weight paths – common for components in a diffusion manifest that are config-only (for example, the scheduler).

Returns:

A WeightLoader over this config’s source namespace.

Return type:

WeightLoader

log_model_info()

log_model_info(role)

source

Logs model configuration information for this config.

Parameters:

role (str) – The semantic role of this model (e.g. "main", "draft", "vae").

Return type:

None

max_length

max_length: int | None

source

The maximum sequence length the model can process.

model_config

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'strict': False}

source

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_name

property model_name: str

source

Returns the served model name or model path.

model_path

model_path: str

source

The repository ID of a Hugging Face model to use.

model_post_init()

model_post_init(context, /)

source

This function is meant to behave like a BaseModel method to initialise private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

• self (BaseModel) – The BaseModel instance.
• context (Any) – The context.

Return type:

None

pool_embeddings

pool_embeddings: bool

source

Whether to pool embedding outputs.

quantization_encoding

quantization_encoding: SupportedEncoding | None

source

The weight encoding type.

resolved_weight_paths()

resolved_weight_paths(weight_path=None)

source

Resolve weight paths to absolute local paths, downloading if needed.

For online repos, downloads weight files from HuggingFace Hub. For local repos, constructs absolute paths from the repo root.

Parameters:

weight_path (list[Path] | None) – Weight files to resolve, relative to the repo. Defaults to self.weight_path. Pass an explicit, already-resolved list for a config whose weight_path was never populated during resolution (e.g. a diffusion component resolved on demand at consumption time – see _resolve_component_encoding_and_weights()).

Returns:

Absolute paths to weight files on disk.

Return type:

list[Path]

rope_type

rope_type: RopeType | None

source

The RoPE type to use, forced regardless of model defaults.

sampling_params_defaults

property sampling_params_defaults: SamplingParamsGenerationConfigDefaults

source

Returns sampling defaults derived from the generation config.

served_model_name

served_model_name: str | None

source

An optional override for the client-facing model name.

sliding_window

sliding_window: int | None

source

Override the attention sliding-window size in tokens.

subfolder

subfolder: str | None

source

Subdirectory within the HuggingFace repo to load config and weights from.

trust_remote_code

trust_remote_code: bool

source

Whether to allow custom modeling files from Hugging Face.

use_subgraphs

use_subgraphs: bool

source

Whether to use subgraphs for the model.

validate_and_resolve_with_resolved_quantization_encoding()

validate_and_resolve_with_resolved_quantization_encoding(resolved_encoding, applied_dtype_cast_from, supported_encodings, default_weights_format)

source

Validates model path and weight path against resolved quantization encoding.

Parameters:

• resolved_encoding (SupportedEncoding) – The encoding the model will actually run with, as resolved by _select_quantization_encoding().
• applied_dtype_cast_from (SupportedEncoding | None) – The encoding weights are cast from at load time, or None when no cast applies.
• supported_encodings (set[SupportedEncoding]) – A dictionary of supported encodings and their corresponding KV cache strategies.
• default_weights_format (WeightsFormat) – The default weights format to use if no weights format is provided.

Return type:

None

validate_lora_compatibility()

validate_lora_compatibility()

source

Validates that LoRA configuration is compatible with model settings.

Raises:

ValueError – If LoRA is enabled but incompatible with current model configuration.

Return type:

None

validate_max_length()

classmethod validate_max_length(v)

source

Validate that max_length is non-negative if provided.

Parameters:

v (int | None)

Return type:

int | None

validate_multi_gpu_supported()

validate_multi_gpu_supported(multi_gpu_supported)

source

Validates that the model architecture supports multi-GPU inference.

Parameters:

multi_gpu_supported (bool) – Whether the model architecture supports multi-GPU inference.

Return type:

None

validate_repo_access()

validate_repo_access()

source

Validates that the model’s Hugging Face repo is accessible.

Deferred out of __init__ so a MAXModelConfig can be constructed offline; invoked from PipelineConfig construction. A no-op when weights are given explicitly (weight_path), when no model is specified (a placeholder config), or when model_path is a local path – there is no remote repo to check in those cases. Requiring a model to actually run is enforced later, during architecture resolution.

Raises:

ValueError – If the specified Hugging Face repo is inaccessible.

Return type:

None

vision_config_overrides

vision_config_overrides: dict[str, Any]

source

Model-specific vision configuration overrides.

weight_path

weight_path: list[Path]

source

The path or URL of the model weights to use.

weights_size()

weights_size()

source

Calculates the total size in bytes of all weight files in weight_path.

Attempts to find the weights locally first to avoid network calls, checking in the following order:

1. If repo_type is "local", it checks if the path in weight_path exists directly as a local file path.
2. Otherwise, if repo_type is "online", it first checks the local Hugging Face cache using huggingface_hub.try_to_load_from_cache(). If not found in the cache, it falls back to querying the Hugging Face Hub API via HuggingFaceRepo.size_of().

Returns:

The total size of all weight files in bytes.

Raises:

• FileNotFoundError – If repo_type is "local" and a file specified in weight_path is not found within the local repo directory.
• ValueError – If HuggingFaceRepo.size_of() fails to retrieve the file size from the Hugging Face Hub API (for example, file metadata not available or API error).
• RuntimeError – If the determined repo_type is unexpected.

Return type:

int