IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /max/get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Python class

SupportedArchitecture

SupportedArchitecture​

class max.pipelines.lib.registry.SupportedArchitecture(name, example_repo_ids, default_encoding, supported_encodings, pipeline_model, task, tokenizer, default_weights_format, context_type, config, weight_adapters=<factory>, multi_gpu_supported=False, input_modalities=<factory>, required_arguments=<factory>, context_validators=<factory>, supports_empty_batches=False, requires_max_batch_context_length=False, tool_parser=None, batching=None, reasoning_parser=None, default_structured_output_backend=None, supports_overlap_scheduler=True, supports_device_graph_capture=True, memory_planner=None, cascade_pipeline_factory=None, pipeline_cls=None)

source

Bases: object

Represents a model architecture configuration for MAX pipelines.

Defines the components and settings required to support a specific model architecture within the MAX pipeline system. Each SupportedArchitecture instance encapsulates the model implementation, tokenizer, supported encodings, and other architecture-specific configuration.

New architectures should be registered into the PipelineRegistry using the register() method.

Example:

my_architecture = SupportedArchitecture(
    name="MyModelForCausalLM",  # Must match your Hugging Face model class name
    example_repo_ids=[
        "your-org/your-model-name",  # Add example model repository IDs
    ],
    default_encoding="q4_k",
    supported_encodings={
        "q4_k",
        "bfloat16",
        # Add other encodings your model supports
    },
    pipeline_model=MyModel,
    tokenizer=TextTokenizer,
    context_type=TextContext,
    config=MyModelConfig,  # Architecture-specific config class
    default_weights_format=WeightsFormat.safetensors,
    weight_adapters={
        WeightsFormat.safetensors: weight_adapters.convert_safetensor_state_dict,
        # Add other weight formats if needed
    },
    multi_gpu_supported=True,  # Set based on your implementation capabilities
    required_arguments={"some_arg": True},
    task=PipelineTask.TEXT_GENERATION,
)

Parameters:

batching​

batching: type[Any] | None = None

source

Optional batch processor for input/output handling.

When set, must be a BatchProcessor subclass. The processor class is applied to pipeline_model at registration time via batch_processor_cls. Ragged text models should subclass RaggedBatchProcessor.

cascade_pipeline_factory​

cascade_pipeline_factory: Callable[[PipelineConfig], object] | None = None

source

Optional cascade pipeline factory for this architecture.

A CascadePipeline subclass (from max.experimental.cascade) that accepts a PipelineConfig in its constructor. The experimental cascade server resolves the architecture and constructs cascade_pipeline_factory(config), so cascade pipeline selection is driven entirely by the architecture rather than by PipelineTask.

The return is annotated object rather than CascadePipeline because the cascade layer sits above max.pipelines; importing the base class here to tighten the annotation would invert that dependency (the cascade server narrows the constructed value back to CascadePipeline). None means the architecture has no cascade pipeline yet.

config​

config: type[ArchConfig]

source

The architecture-specific configuration class for the model.

This class must implement the ArchConfig protocol, providing an initialize method that creates a configuration instance from a PipelineConfig. For models with KV cache, this should be a class implementing ArchConfigWithKVCache to enable KV cache memory estimation.

context_type​

context_type: type[TextContext] | type[EmbeddingsContext]

source

The context class type that this architecture uses for managing request state and inputs.

This should be a class (not an instance) that implements either the TextContext or EmbeddingsContext protocol, defining how the pipeline processes and tracks requests.

context_validators​

context_validators: list[Callable[[TextContext | TextAndVisionContext | PixelContext], None]]

source

A list of callable validators that verify context inputs before model execution.

These validators are called during context creation to ensure inputs meet model-specific requirements. Validators should raise InputError for invalid inputs, providing early error detection before expensive model operations.

def validate_single_image(context: TextContext | TextAndVisionContext) -> None:
    if isinstance(context, TextAndVisionContext):
        if context.pixel_values and len(context.pixel_values) > 1:
            raise InputError(f"Model supports only 1 image, got {len(context.pixel_values)}")

my_architecture = SupportedArchitecture(
    # ... other fields ...
    context_validators=[validate_single_image],
)

default_encoding​

default_encoding: SupportedEncoding

source

The default quantization encoding to use when no specific encoding is requested.

default_structured_output_backend​

default_structured_output_backend: str | None = None

source

Optional default structured output backend for this architecture.

When set (e.g., "llguidance" or "xgrammar"), the pipeline config will use this value for sampling.structured_output_backend if the user did not explicitly configure one. This allows architectures that work better with a specific backend to override the global default.

If None, the global default from SamplingConfig is used.

default_weights_format​

default_weights_format: WeightsFormat

source

The weights format expected by the pipeline_model.

example_repo_ids​

example_repo_ids: list[str]

source

A list of Hugging Face repository IDs that use this architecture for testing and validation purposes.

input_modalities​

input_modalities: set[InputModality]

source

The set of input modalities this architecture accepts.

Defaults to text-only. Multimodal architectures should declare all supported input types explicitly, e.g. {InputModality.TEXT, InputModality.IMAGE} for vision-language models.

memory_planner​

memory_planner: type[MemoryPlanner] | None = None

source

Optional MemoryPlanner subclass for this architecture.

When set, PipelineConfig uses the planner to estimate weight size, activation memory, signal-buffer memory, and vision cache entry bytes. Autoregressive text-generation models should set this to PagedMemoryPlanner (or a subclass with architecture-specific overrides).

None means the architecture manages its own memory estimation (e.g. diffusion pipelines that skip KV cache estimation entirely).

multi_gpu_supported​

multi_gpu_supported: bool = False

source

Whether the architecture supports multi-GPU execution.

name​

name: str

source

The name of the model architecture that must match the Hugging Face model class name.

pipeline_cls​

pipeline_cls: type | None = None

source

Optional pipeline class overriding the task-based default from get_pipeline_for_task().

Most architectures leave this None and are driven by the standard task pipelines. Set it when an architecture needs a bespoke generation loop that the stock one-token-per-step TextGenerationPipeline cannot express — for example block-diffusion text generation, which runs an encoder pass plus an inner denoising loop and emits a whole token block per scheduler step. The value must be a TextGenerationPipeline subclass (or compatible) selected after pipeline_config.resolve().

pipeline_model​

pipeline_model: PipelineModelType

source

The model class that defines the graph structure and execution logic.

Accepts either a PipelineModel subclass (for LLM and other token-generation architectures) or a PipelineExecutor subclass (for newer executor-based architectures such as diffusion pipelines).

reasoning_parser​

reasoning_parser: str | None = None

source

Optional default reasoning parser name for this architecture.

The name must correspond to a parser registered via max.pipelines.lib.reasoning.register(). When set, the pipeline config will fall back to this value for runtime.reasoning_parser if the user did not explicitly configure one. Different model architectures emit reasoning content in different formats (e.g., Kimi K2.5 wraps reasoning in <think>...</think>), so the appropriate default is architecture-specific.

If None, no reasoning parser is enabled by default and the user must opt in by setting runtime.reasoning_parser explicitly.

required_arguments​

required_arguments: dict[str, bool | int | float]

source

A dictionary specifying required values for PipelineConfig options.

requires_max_batch_context_length​

requires_max_batch_context_length: bool = False

source

Whether the architecture requires a max batch context length to be specified.

If True and max_batch_context_length is not specified, we will default to the max sequence length of the model.

supported_encodings​

supported_encodings: set[SupportedEncoding]

source

A dictionary of supported quantization encodings.

supports_device_graph_capture​

supports_device_graph_capture: bool = True

source

Whether this architecture supports auto-enabling device graph capture.

When False, device graph capture is not auto-enabled for this architecture even when otherwise eligible. Users can still force-enable via --device-graph-capture --force.

supports_empty_batches​

supports_empty_batches: bool = False

source

Whether the architecture can handle empty batches during inference.

When set to True, the pipeline can process requests with zero-sized batches without errors. This is useful for certain execution modes and expert parallelism. Most architectures do not require empty batch support and should leave this as False.

supports_overlap_scheduler​

supports_overlap_scheduler: bool = True

source

Whether this architecture supports auto-enabling the overlap scheduler.

When False, the overlap scheduler is not auto-enabled for this architecture even when otherwise eligible. Users can still force-enable via --enable-overlap-scheduler --force.

task​

task: PipelineTask

source

The pipeline task type that this architecture supports.

tokenizer​

tokenizer: Callable[..., PipelineTokenizer[Any, Any, Any]]

source

A callable that returns a PipelineTokenizer instance for preprocessing model inputs.

tokenizer_cls​

property tokenizer_cls: type[PipelineTokenizer[Any, Any, Any]]

source

Returns the tokenizer class for this architecture.

tool_parser​

tool_parser: str | Callable[[HuggingFaceRepo], str] | None = None

source

Optional default tool parser for this architecture.

Either a registered parser name (str), or a callable that takes the model’s HuggingFaceRepo handle (carrying repo_id, revision, subfolder, and trust_remote_code) and returns a registered parser name. Use the callable form when one architecture name covers multiple checkpoint revisions with different tool-call grammars (for example, DeepSeek V3 vs V3.1). The callable is invoked once during pipeline config resolution and the resulting string is stored on runtime.tool_parser.

The returned name must correspond to a parser registered via max.pipelines.lib.tool_parsing.register(). When set, the pipeline config falls back to this value for runtime.tool_parser if the user did not explicitly configure one.

If None, no tool parser is enabled by default and the serving layer falls back to its baseline parser.

weight_adapters​

weight_adapters: dict[WeightsFormat, WeightsAdapter]

source

A dictionary of weight format adapters for converting checkpoints from different formats to the default format.

Models

Cookbook

Puzzles

Blog

Community

Contact