For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Python class

PipelineArgs

`PipelineArgs`

class max.pipelines.lib.PipelineArgs(*, models=None, config_file=None, section_name=None, model_override=<factory>, task=PipelineTask.UNDEFINED, debug_verify_replay=False, model_path='', served_model_name=None, weight_path=<factory>, quantization_encoding=None, huggingface_model_revision='main', huggingface_weight_revision='main', trust_remote_code=False, subfolder=None, device_specs=<factory>, force_download=False, vision_config_overrides=<factory>, rope_type=None, sliding_window=None, enable_echo=False, chat_template=None, use_subgraphs=True, data_parallel_degree=1, pool_embeddings=True, max_length=None, kv_cache=<factory>, pipeline_role='prefill_and_decode', max_batch_size=None, max_queue_size_tg=None, min_batch_size_tg=None, ep_size=1, ep_use_allreduce=False, eplb_profile=<factory>, ce_delay_ms=0.0, enable_prioritize_first_decode=False, enable_chunked_prefill=True, enable_in_flight_batching=False, eplb_replicas_per_gpu=0, max_num_steps=1, max_batch_input_tokens=8192, use_experimental_kernels='false', use_vendor_blas='false', use_vendor_ccl='false', custom_architectures=<factory>, execute_empty_batches=False, max_batch_total_tokens=None, device_graph_capture=None, force=False, kvcache_ce_watermark=0.95, decode_stall_timeout_s=None, decode_request_ttl_s=None, enable_overlap_scheduler=False, allow_unsupported_logprobs=False, allow_extra_request_fields=False, prefer_module_v3=False, reasoning_parser=None, tool_parser=None, emit_reasoning_content=False, temperature=None, thinking_temperature=None, max_vision_cache_entries=256, denoising_cache=<factory>, in_dtype=float32, out_dtype=float32, enable_structured_output=False, structured_output_backend='llguidance', enable_variable_logits=False, enable_penalties=False, enable_min_tokens=False, sample_on_host=False, gpu_profiling='off', profiling_enabled=False, profiling_output_path=None, profiling_dynolog_enabled=True, profiling_warmup_steps=0, profiling_active_steps=10, profiling_periodic_flush_seconds=60, lora=None, speculative=None, draft_model=None)

source

Bases: ConfigFileModel

Flat, user-settable input arguments for a pipeline.

PipelineArgs is the user-facing input to the pipeline system. It holds only scalar fields and a small number of cohesive sub-config objects (kv_cache, lora, speculative, draft_model).

Multi-component pipelines (e.g. diffusion) that require a pre-built ModelManifest may pass models=<manifest> to the constructor. That manifest is stored as a private override and used verbatim by PipelineConfig.from_args() instead of constructing one from the flat scalar fields.

Call PipelineConfig.from_args() to obtain a fully-constructed PipelineConfig ready for architecture-driven resolution.

Parameters:

models (ModelManifest | None)
config_file (str | None)
section_name (str | None)
model_override (list[str])
task (PipelineTask)
debug_verify_replay (bool)
model_path (str)
served_model_name (str | None)
weight_path (list[Path])
quantization_encoding (Literal['float32', 'float16', 'bfloat16', 'q4_k', 'q4_0', 'q6_k', 'float8_e4m3fn', 'float4_e2m1fnx2', 'gptq'] | None)
huggingface_model_revision (str)
huggingface_weight_revision (str)
trust_remote_code (bool)
subfolder (str | None)
device_specs (list[DeviceSpec])
force_download (bool)
vision_config_overrides (dict[str, Any])
rope_type (Literal['none', 'normal', 'neox', 'longrope', 'yarn'] | None)
sliding_window (int | None)
enable_echo (bool)
chat_template (Path | None)
use_subgraphs (bool)
data_parallel_degree (int)
pool_embeddings (bool)
max_length (int | None)
kv_cache (KVCacheConfig)
pipeline_role (Literal['prefill_and_decode', 'prefill_only', 'decode_only'])
max_batch_size (int | None)
max_queue_size_tg (int | None)
min_batch_size_tg (int | None)
ep_size (int)
ep_use_allreduce (bool)
eplb_profile (bool)
ce_delay_ms (float)
enable_prioritize_first_decode (bool)
enable_chunked_prefill (bool)
enable_in_flight_batching (bool)
eplb_replicas_per_gpu (int)
max_num_steps (int)
max_batch_input_tokens (int)
use_experimental_kernels (str)
use_vendor_blas (str)
use_vendor_ccl (str)
custom_architectures (list[str])
execute_empty_batches (bool)
max_batch_total_tokens (int | None)
device_graph_capture (bool | None)
force (bool)
kvcache_ce_watermark (float)
decode_stall_timeout_s (float | None)
decode_request_ttl_s (float | None)
enable_overlap_scheduler (bool)
allow_unsupported_logprobs (bool)
allow_extra_request_fields (bool)
prefer_module_v3 (bool)
reasoning_parser (str | None)
tool_parser (str | None)
emit_reasoning_content (bool)
temperature (float | None)
thinking_temperature (float | None)
max_vision_cache_entries (int)
denoising_cache (DenoisingCacheConfig)
in_dtype (DType)
out_dtype (DType)
enable_structured_output (bool)
structured_output_backend (str)
enable_variable_logits (bool)
enable_penalties (bool)
enable_min_tokens (bool)
sample_on_host (bool)
gpu_profiling (Literal['off', 'on', 'detailed'])
profiling_enabled (bool)
profiling_output_path (str | None)
profiling_dynolog_enabled (bool)
profiling_warmup_steps (Annotated[int, Ge(ge=0)])
profiling_active_steps (Annotated[int, Ge(ge=1)])
profiling_periodic_flush_seconds (Annotated[int, Ge(ge=1)])
lora (LoRAConfig | None)
speculative (SpeculativeConfig | None)
draft_model (MAXModelConfig | None)

`allow_extra_request_fields`

allow_extra_request_fields: bool

source

`allow_unsupported_logprobs`

allow_unsupported_logprobs: bool

source

`ce_delay_ms`

ce_delay_ms: float

source

`chat_template`

chat_template: Path | None

source

`config_file`

config_file: str | None

source

Path to the configuration file.

`custom_architectures`

custom_architectures: list[str]

source

`data_parallel_degree`

data_parallel_degree: int

source

`debug_verify_replay`

debug_verify_replay: bool

source

`decode_request_ttl_s`

decode_request_ttl_s: float | None

source

`decode_stall_timeout_s`

decode_stall_timeout_s: float | None

source

`denoising_cache`

denoising_cache: DenoisingCacheConfig

source

`device_graph_capture`

device_graph_capture: bool | None

source

`device_specs`

device_specs: list[DeviceSpec]

source

`draft_model`

draft_model: MAXModelConfig | None

source

`emit_reasoning_content`

emit_reasoning_content: bool

source

`enable_chunked_prefill`

enable_chunked_prefill: bool

source

`enable_echo`

enable_echo: bool

source

`enable_in_flight_batching`

enable_in_flight_batching: bool

source

`enable_min_tokens`

enable_min_tokens: bool

source

`enable_overlap_scheduler`

enable_overlap_scheduler: bool

source

`enable_penalties`

enable_penalties: bool

source

`enable_prioritize_first_decode`

enable_prioritize_first_decode: bool

source

`enable_structured_output`

enable_structured_output: bool

source

`enable_variable_logits`

enable_variable_logits: bool

source

`ep_size`

ep_size: int

source

`ep_use_allreduce`

ep_use_allreduce: bool

source

`eplb_profile`

eplb_profile: bool

source

`eplb_replicas_per_gpu`

eplb_replicas_per_gpu: int

source

`execute_empty_batches`

execute_empty_batches: bool

source

`force`

force: bool

source

`force_download`

force_download: bool

source

`from_flat_kwargs()`

classmethod from_flat_kwargs(**kwargs)

source

Construct a PipelineArgs from a flat CLI kwargs namespace.

Routes flat kwargs (the same format accepted by PipelineConfig.from_flat_kwargs()) into the appropriate flat fields of PipelineArgs. Delegates to PipelineConfig.from_flat_kwargs() for the heavy-lifting of routing and sub-config construction, then extracts the user-facing fields.

Parameters:: **kwargs (Any) – Flat keyword arguments, e.g. model_path, kv_cache_size, enable_lora.
Returns:: A PipelineArgs populated from the flat kwargs.
Return type:: Self

`from_pipeline_config()`

classmethod from_pipeline_config(pipeline_config)

source

Construct a PipelineArgs from an existing PipelineConfig.

Extracts the user-facing flat fields from a PipelineConfig and returns a PipelineArgs populated from them.

This exists to let from_flat_kwargs() reuse PipelineConfig.from_flat_kwargs()’s flat-kwarg routing logic (parsing --model-override, building the draft model config, etc.) instead of duplicating it. It is not a general round-trip: pipeline_config is expected to be freshly constructed and not yet resolved. Resolution-derived state (e.g. an applied dtype cast recorded by MAXModelConfig.resolve()) is not preserved – PipelineArgs is deliberately isolated from resolution mutations (see #90128), so passing an already-resolved pipeline_config here will silently drop that state.

Parameters:: pipeline_config (PipelineConfig) – The source PipelineConfig to extract from. Should not have had PipelineConfig.resolve() called on it.
Returns:: A PipelineArgs populated from the given config.
Return type:: Self

`gpu_profiling`

gpu_profiling: GPUProfilingMode

source

`huggingface_model_revision`

huggingface_model_revision: str

source

`huggingface_weight_revision`

huggingface_weight_revision: str

source

`in_dtype`

in_dtype: DType

source

`kv_cache`

kv_cache: KVCacheConfig

source

`kvcache_ce_watermark`

kvcache_ce_watermark: float

source

`lora`

lora: LoRAConfig | None

source

`main_architecture_name`

property main_architecture_name: str

source

Returns the HuggingFace architecture class name for the main model.

Reads architectures[0] from the model’s HuggingFace config without constructing a full PipelineConfig.

Raises:: ValueError – If the architecture name cannot be determined.

`max_batch_input_tokens`

max_batch_input_tokens: int

source

`max_batch_size`

max_batch_size: int | None

source

`max_batch_total_tokens`

max_batch_total_tokens: int | None

source

`max_length`

max_length: int | None

source

`max_num_steps`

max_num_steps: int

source

`max_queue_size_tg`

max_queue_size_tg: int | None

source

`max_vision_cache_entries`

max_vision_cache_entries: int

source

`min_batch_size_tg`

min_batch_size_tg: int | None

source

`model_config`

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'strict': False}

source

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

`model_override`

model_override: list[str]

source

`model_path`

model_path: str

source

`model_post_init()`

model_post_init(context, /)

source

This function is meant to behave like a BaseModel method to initialise private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

self (BaseModel) – The BaseModel instance.
context (Any) – The context.

Return type:

None

`out_dtype`

out_dtype: DType

source

`pipeline_role`

pipeline_role: PipelineRole

source

`pool_embeddings`

pool_embeddings: bool

source

`prefer_module_v3`

prefer_module_v3: bool

source

`profiling_active_steps`

profiling_active_steps: int

source

`profiling_dynolog_enabled`

profiling_dynolog_enabled: bool

source

`profiling_enabled`

profiling_enabled: bool

source

`profiling_output_path`

profiling_output_path: str | None

source

`profiling_periodic_flush_seconds`

profiling_periodic_flush_seconds: int

source

`profiling_warmup_steps`

profiling_warmup_steps: int

source

`quantization_encoding`

quantization_encoding: SupportedEncoding | None

source

`reasoning_parser`

reasoning_parser: str | None

source

`rope_type`

rope_type: RopeType | None

source

`sample_on_host`

sample_on_host: bool

source

`section_name`

section_name: str | None

source

Optional section name for comprehensive/multi-section config files.

If not provided, values are loaded from the YAML top-level (treating the file as an “individual config” file).

`served_model_name`

served_model_name: str | None

source

`sliding_window`

sliding_window: int | None

source

`speculative`

speculative: SpeculativeConfig | None

source

`structured_output_backend`

structured_output_backend: str

source

`subfolder`

subfolder: str | None

source

`task`

task: PipelineTask

source

`temperature`

temperature: float | None

source

`thinking_temperature`

thinking_temperature: float | None

source

`tool_parser`

tool_parser: str | None

source

`trust_remote_code`

trust_remote_code: bool

source

`use_experimental_kernels`

use_experimental_kernels: str

source

`use_subgraphs`

use_subgraphs: bool

source

`use_vendor_blas`

use_vendor_blas: str

source

`use_vendor_ccl`

use_vendor_ccl: str

source

`vision_config_overrides`

vision_config_overrides: dict[str, Any]

source

`weight_path`

weight_path: list[Path]

source

PipelineArgs

PipelineArgs​

allow_extra_request_fields​

allow_unsupported_logprobs​

ce_delay_ms​

chat_template​

config_file​

custom_architectures​

data_parallel_degree​

debug_verify_replay​

decode_request_ttl_s​

decode_stall_timeout_s​

denoising_cache​

device_graph_capture​

device_specs​

draft_model​

emit_reasoning_content​

enable_chunked_prefill​

enable_echo​

enable_in_flight_batching​

enable_min_tokens​

enable_overlap_scheduler​

enable_penalties​

enable_prioritize_first_decode​

enable_structured_output​

enable_variable_logits​

ep_size​

ep_use_allreduce​

eplb_profile​

eplb_replicas_per_gpu​

execute_empty_batches​

force​

force_download​

from_flat_kwargs()​

from_pipeline_config()​

gpu_profiling​

huggingface_model_revision​

huggingface_weight_revision​

in_dtype​

kv_cache​

kvcache_ce_watermark​

lora​

main_architecture_name​

max_batch_input_tokens​

max_batch_size​

max_batch_total_tokens​

max_length​

max_num_steps​

max_queue_size_tg​

max_vision_cache_entries​

min_batch_size_tg​

model_config​

model_override​

model_path​

model_post_init()​

out_dtype​

pipeline_role​

pool_embeddings​

prefer_module_v3​

profiling_active_steps​

profiling_dynolog_enabled​

profiling_enabled​

profiling_output_path​

profiling_periodic_flush_seconds​

profiling_warmup_steps​

quantization_encoding​

reasoning_parser​

rope_type​

sample_on_host​

section_name​

served_model_name​

sliding_window​

speculative​

structured_output_backend​

subfolder​

task​

temperature​

thinking_temperature​

tool_parser​

trust_remote_code​

use_experimental_kernels​