IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /max/get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Python class

PipelineConfig

PipelineConfig​

class max.pipelines.PipelineConfig(*, config_file=None, section_name=None, debug_verify_replay=False, models=<factory>, model_override=<factory>, sampling=<factory>, profiling=<factory>, lora=None, speculative=None, runtime=<factory>, task=PipelineTask.UNDEFINED)

source

Bases: ConfigFileModel

Configuration for a pipeline.

Contains settings for model selection, batch sizing, sampling, profiling, LoRA adapters, and speculative decoding. Once initialized, all fields are resolved to their final values from CLI flags, config files, environment variables, or internal defaults.

Parameters:

configure_session()​

configure_session(session)

source

Configures a InferenceSession with standard pipeline settings.

Parameters:

session (InferenceSession)

Return type:

None

debug_verify_replay​

debug_verify_replay: bool

source

Whether to run eager verification before device graph replay.

draft_model​

property draft_model: MAXModelConfig | None

source

The draft model configuration. Alias for models.get("draft").

estimate_signal_buffer_memory()​

estimate_signal_buffer_memory(arch_config=None)

source

Estimates total signal-buffer memory across all devices.

Signal buffers are fixed-size (NUM_BYTES) per-GPU allocations used by P2P collectives. Each independent allocation site contributes one set of ngpus buffers. The base estimate counts the sites visible from PipelineConfig:

  • main model graph (multi-GPU only),
  • BlockOffloadEngine for KV-cache offloading, only when its replicate_kv_across_tp path is active (MLA model with DP=1 and multi-device TP). See block_copy_engine.py / transfer_engine.py.

Returns 0 for single-device pipelines.

Parameters:

arch_config (ArchConfig | None) – Optional architecture config. When provided and it exposes KV params, the BCE term is gated on the actual replicates_kv_across_tp flag rather than only the kv_connector setting. Without it, the BCE term is added whenever a connector is configured (conservative).

Returns:

Estimated total signal-buffer memory in bytes (across all devices).

Return type:

int

from_args()​

classmethod from_args(args)

source

Construct a PipelineConfig from a PipelineArgs.

Parameters:

args (PipelineArgs) – Flat user-facing pipeline arguments.

Returns:

A fully constructed PipelineConfig ready for architecture-driven resolution via resolve().

Return type:

Self

from_flat_kwargs()​

classmethod from_flat_kwargs(**kwargs)

source

Construct a PipelineConfig from a flat CLI kwargs namespace.

Accepts the flat kwargs produced by pipeline_config_options (for example model_path, kv_cache_size, enable_lora) and routes them into the appropriate sub-configs before constructing the instance.

This is the entry point for CLI and legacy callers. Direct construction via PipelineConfig(models=..., runtime=..., ...) with properly typed sub-configs is also supported and requires no routing.

Parameters:

kwargs (Any)

Return type:

Self

graph_quantization_encoding​

property graph_quantization_encoding: QuantizationEncoding | None

source

Converts the CLI encoding to a MAX graph quantization encoding.

Returns:

The graph quantization encoding corresponding to the CLI encoding.

lora​

lora: LoRAConfig | None

source

The LoRA configuration.

model​

property model: MAXModelConfig

source

The main model config. Alias for models["main"].

model_config​

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'strict': False}

source

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_override​

model_override: list[str]

source

Per-component model overrides applied before resolution.

model_post_init()​

model_post_init(context, /)

source

This function is meant to behave like a BaseModel method to initialise private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

  • self (BaseModel) – The BaseModel instance.
  • context (Any) – The context.

Return type:

None

models​

models: _ModelsType

source

The model manifest containing all model configs keyed by role.

needs_bitmask_constraints​

property needs_bitmask_constraints: bool

source

Whether constrained decoding can fire and requires the bitmask path.

True if the user enabled --enable-structured-output (for user-supplied response_format=json_schema) or a tool parser is configured (tool-call grammars work without the flag — they are server-generated and gated on having a parser that can both produce the grammar and parse the resulting output).

Tool-call constrained decoding can be turned off independently via sampling.enable_tool_call_constrained_decode: when that is False the tool parser still parses tool calls out of generated text, but no grammar is generated and the bitmask path is not needed on its account.

Drives whether model / sampler graphs are compiled with a bitmask input and whether the D2H pinned buffer is allocated. Distinct from sampling.enable_structured_output, which is the user-facing flag and only gates honoring user-supplied JSON schemas.

profiling​

profiling: ProfilingConfig

source

The profiling configuration.

resolve()​

resolve(arch, draft_arch=None)

source

Validates the config.

Parameters:

  • arch (Any) – Pre-resolved target architecture from the registry.
  • draft_arch (Any) – Pre-resolved draft architecture (speculative decoding only). Required when draft_model is set.

Return type:

None

runtime​

runtime: PipelineRuntimeConfig

source

The model-agnostic runtime settings for pipeline execution.

sampling​

sampling: SamplingConfig

source

The sampling configuration.

speculative​

speculative: SpeculativeConfig | None

source

The speculative decoding configuration.

task​

task: PipelineTask

source

The pipeline task, used for arch disambiguation during config resolution.

Models

Cookbook

Puzzles

Blog

Community

Contact