What's new in MAX

Here's everything you should know about what's changed in each release.

v25.3 nightly

This version is still a work in progress.

See how to install the nightly release.

Highlights
Documentation
MAX Serve
MAX models
- max-pipelines CLI
MAX Engine
GPU programming
Mojo

v25.2

Highlights
Documentation
MAX Serve
MAX models
- max-pipelines CLI
MAX Engine
GPU programming
Mojo

Highlights

GPU programming with Mojo

Lots of new APIs are now available to enable both low-level GPU programming and abstracted programming patterns that simplify the code required to write GPU kernels for your AI models.
Multi-GPU support

When running massive LLMs such as Llama-3.3-70B-Instruct, MAX now uses tensor parallelism to distribute work across multiple GPUs.
Expanded library of MAX models

We're rapidly growing our library of base model architectures that MAX can accelerate with MAX Serve. Check out our MAX model repository.
Support for NVIDIA Hopper GPUs

You can now deploy MAX on NVIDIA H100 and H200 instances.

Documentation

New examples for programming with GPUs:

New examples for writing custom ops:
- fused_attention demonstrates complex GPU programming using MAX abstractions for a practical use in AI model development.
- matrix_multiplication includes a series of progressive optimizations for matrix multiplications on GPUs.
- histogram shows how to implement the histogram pattern as a custom op.
New examples for GPU programming in Mojo using the new MAX Driver API.

These use a Mojo programming model that should look familiar to CUDA C programmers, showing how to define and dispatch GPU functions within a single Mojo file. These examples recreate the first three samples from the popular textbook "Programming Massively Parallel Processors", showing how basic concepts translate from CUDA into Mojo. Additionally, a Mandelbrot set calculation example that parallels a similar one in the existing custom ops examples.

MAX Serve

Extended MAX Serve batch scheduling to account for the prefix cache. The scheduler can now create larger batches when many prompt tokens are already cached, improving throughput up to 10% in some benchmarks.
Added support for in-flight batching, allowing token generation requests to be scheduled alongside context encoding requests to reduce inter-token latency. This behavior can be controlled by CLI argument --enable-in-flight-batch.
Added support for copy-on-write on KV blocks when using PagedAttention with Prefix Caching. This improves the prefix cache hit rate and prefill performance in some scenarios.
MAX Serve now supports transformers v.4.49.0, with a patch to avoid graph breaks when using torch.compile on Llama models.
Added support for recording HTTP traffic out to a file for diagnostics or later replay.

MAX models

Added support for executing LlamaForCausalLM architecture models on multiple GPUs. The model uses tensor parallelism automatically when passing multiple device IDs to the --devices CLI argument. Try running Llama-3.3-70B-Instruct on 4 GPUs with:

max-pipelines generate --model-path=meta-llama/Llama-3.3-70B-Instruct \
  --quantization-encoding bfloat16 \
  --devices gpu:0,1,2,3 \
  --prompt="Design a self-sustaining colony on Neptune's moon Triton with a myth/science fusion name, three quantum tech breakthroughs, one ethical debate, a neon-lit cultural ritual, and a hidden flaw—presented in bullet points 🌌💡."
max-pipelines generate --model-path=meta-llama/Llama-3.3-70B-Instruct \
  --quantization-encoding bfloat16 \
  --devices gpu:0,1,2,3 \
  --prompt="Design a self-sustaining colony on Neptune's moon Triton with a myth/science fusion name, three quantum tech breakthroughs, one ethical debate, a neon-lit cultural ritual, and a hidden flaw—presented in bullet points 🌌💡."

In the event of an unhandled exception, we now use the standard Python traceback format instead of using pretty-printed Rich tracebacks.
Added support for models microsoft/Phi-3.5-mini-instruct, microsoft/phi-4, LGAI-EXAONE/EXAONE-3.0-7.8B-Instruct and LGAI-EXAONE/EXAONE-3.5-2.4B-Instruct.
Model weights are now downloaded in parallel.
We now need to explicitly import LLM from max.entrypoints.llm rather than the previous max.entrypoints import.
Added constraints on whitespace during Structured Output. This reduces tokens counts and improves model adherence.
Added jump ahead decoding during Structured Output. This auto-completes tokens when a singular path forward is identified, improving single completion times by up to ~20% for long prompts.

We now support GPTQ quantization for models that run on the GPU. This is handled transparently when the model weights are specified. For example, this runs Llama 3.1 8B using int4-quantized GPTQ weights:

max-pipelines generate \
--model-path hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4 \
--prompt "Why is the sky blue?" \
--max-batch-size 1 --max-length 10000 \
--quantization-encoding gptq
max-pipelines generate \
--model-path hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4 \
--prompt "Why is the sky blue?" \
--max-batch-size 1 --max-length 10000 \
--quantization-encoding gptq

This reduces the total memory consumption of this model from ~16 GB to ~5 GB, allowing it to fit in the RAM on consumer-grade GPUs.

Added support for the Phi3ForCausalLM model architecture (such as microsoft/phi-4). For example:

max-pipelines generate \
  --model-path=microsoft/phi-4 \
  --prompt="Write bubble sort in mojo"
max-pipelines generate \
  --model-path=microsoft/phi-4 \
  --prompt="Write bubble sort in mojo"

Added support for the OlmoForCausalLM model architecture (such as allenai/OLMo-1B-0724-hf). For example:

max-pipelines generate \
  --model-path=allenai/OLMo-1B-0724-hf \
  --prompt="Write bubble sort in mojo"
max-pipelines generate \
  --model-path=allenai/OLMo-1B-0724-hf \
  --prompt="Write bubble sort in mojo"

Added support for the GraniteForCausalLM model architecture (such as ibm-granite/granite-3.1-8b-instruct). For example:

max-pipelines generate \
  --model-path=ibm-granite/granite-3.1-8b-instruct \
  --prompt="Write bubble sort in mojo"
max-pipelines generate \
  --model-path=ibm-granite/granite-3.1-8b-instruct \
  --prompt="Write bubble sort in mojo"

The max.pipelines.dataprocessing.tokenizer and max.pipelines.dataprocessing.gguf_utils modules have been removed.
The previously deprecated PipelineConfig.architecture field and its corresponding --architecture CLI argument have been removed.

`max-pipelines` CLI

The --devices CLI argument now supports a comma-separated list of GPU IDs prefixed with gpu: like --devices=gpu:0,1,2,3. We no longer support the previous --devices=gpu-<N> format.
Removed --model-path PipelineConfig option and CLI argument in favor of --model-path.
Consolidated --model-path and --weight-path. If valid --weight-path(s) are provided, they'll now override --model-path, which in turn handles both local and remote (HF) cases. If we cannot derive the weights from the --weight-path(s), we'll now fall back to the --model-path, which has to be set explicitly by the user.
Added --huggingface-revision option, to allow selecting a non-default branch or a specific commit in a Hugging Face model repository.

MAX Engine

The MAX graph compiler now has kernel caching. This is a significant improvement to our compilation pipeline. Here are the key highlights:
- Up to 28% faster compilation times when making iterative changes to models
- Improved caching between different but similar models (up to 27% faster)
- Lays foundation for future caching optimizations
What does this mean for you? Faster development cycles! When you're working on model pipelines and making changes to the graph, the graph compiler will now intelligently reuse kernels that haven't changed, significantly reducing compilation times.

The improvements are particularly noticeable during iterative development, with compilation times dropping from ~80s to ~57s in some cases of compiling Llama3.1-8B for 4 GPUs. Even when compiling different models from the same family (like Llama/Granite variants), you'll see significant speedups on subsequent compilations.

Driver APIs

Added Accelerator.can_access(other: Device) -> bool method to check if one device can directly access memory of another device.
Fixed a bug in max.driver.tensor.load_max_tensor() for bfloat16 dtype, which would cause an error about mmap size being too large.
max.driver.Tensor.item() now works on any single-element tensor (previously restricted to rank-0 tensors).
Added Device.synchronize(), which ensures all operations on the device complete before returning.

Removed MojoCallContextPtr in favor of DeviceContextPtr. MojoCallContextPtr only contained a DeviceContextPtr, so this change directly exposes the DeviceContextPtr. Custom ops using MojoCallContextPtr now directly take a DeviceContextPtr argument:

    @staticmethod
    fn execute[
        type: DType, rank: Int
    ](
        output: OutputTensor[type=type, rank=rank],
        input: InputTensor[type=type, rank=rank],
        ctx: MojoCallContextPtr,
    ):
    @staticmethod
    fn execute[
        type: DType, rank: Int
    ](
        output: OutputTensor[type=type, rank=rank],
        input: InputTensor[type=type, rank=rank],
        ctx: MojoCallContextPtr,
    ):

becomes

    @staticmethod
    fn execute[
        type: DType, rank: Int
    ](
        output: OutputTensor[type=type, rank=rank],
        input: InputTensor[type=type, rank=rank],
        ctx: DeviceContextPtr,
    ):
    @staticmethod
    fn execute[
        type: DType, rank: Int
    ](
        output: OutputTensor[type=type, rank=rank],
        input: InputTensor[type=type, rank=rank],
        ctx: DeviceContextPtr,
    ):

You can now skip compiling a GPU kernel first before enqueueing it, and pass a function directly to ctx.enqueue_function[func](...):

fn func():
    print("Hello from GPU")

@register("custom_op")
struct CustomOp:

    @staticmethod
    fn execute(ctx: DeviceContextPtr) raises:
        var dev_ctx = ctx.get_device_context()
        dev_ctx.enqueue_function[func](grid_dim=1, block_dim=1)
fn func():
    print("Hello from GPU")

@register("custom_op")
struct CustomOp:

    @staticmethod
    fn execute(ctx: DeviceContextPtr) raises:
        var dev_ctx = ctx.get_device_context()
        dev_ctx.enqueue_function[func](grid_dim=1, block_dim=1)

However, if you're reusing the same function and parameters multiple times, this incurs some overhead of around 50-500 nanoseconds per enqueue. So you can still compile the function first and pass it to ctx.enqueue_function in this scenario:

var compiled_func = ctx.compile_function[func]()
# Multiple kernel launches with the same function/parameters
ctx.enqueue_function(compiled_func, grid_dim=1, block_dim=1)
ctx.enqueue_function(compiled_func, grid_dim=1, block_dim=1)
var compiled_func = ctx.compile_function[func]()
# Multiple kernel launches with the same function/parameters
ctx.enqueue_function(compiled_func, grid_dim=1, block_dim=1)
ctx.enqueue_function(compiled_func, grid_dim=1, block_dim=1)

Changed Accelerator and CPU were factory methods that created an object of type Device in Python which were accelerators and cpus in the C++ implementation. This change pushes the Accelerator and CPU type concepts up to Python, so that they are not methods, but types now.

This allows type annotations in Python. For example, a list of accelerators used to be defined like this:
```
graph_devices: list[DeviceRef]
```
```
graph_devices: list[DeviceRef]
```
Now it can be defined like this:
```
graph_devices: list[Accelerator]
```
```
graph_devices: list[Accelerator]
```
Elementwise operations (e.g. __add__ and friends) have been removed from Tensor (that is, tensor_internal.Tensor). This Tensor type is being phased out; please reduce usage in favor of LayoutTensor.

Graph APIs

Moved the nn package up a level to become max.nn.
Added ops.chunk to support chunking tensors along an axis.
Added support for while loops with ops.while_loop.
Added support for conditional execution with ops.cond.
Added axis reduction overloads for ops.min and ops.max, e.g., ops.min(tensor, axis=-1).
The gelu function now accepts an approximate keyword. The keyword controls the gelu approximation with none, tanh, and fast approximations accepted. The tanh approximation computes the gelu via the formula $gelu(x) = 0.5 * x * (1.0 + tanh(0.7978845608028654 * (x + 0.044715 * x**3)))$ whereas the fast approximation computes the gelu via the $sigmoid(1.702 * x) * x$ approximation.
Removed the roundeven operation from the Python API. The round operation now has the same behavior as roundeven, so there is no need for both to exist.
Added helpers to create analogous tensors from buffer types and vice versa.
Added max.nn.Module, a base class for writing layers and constructing networks of layers (e.g. using max.nn.Sequential). Currently, this class supports graph building by ensuring that all weight names are unique and systematically generated. This class also supports managing the weight values with the module.state_dict() and module.load_state_dict() methods. More functionality and documentation will be added in future releases.

Custom ops

Changes have been made to the way that custom ops are registered: rather than using the num_dps_outputs attribute on @compiler.register to specify the number of outputs, that number is now inferred from the signature of the custom operation. Inputs to the operation now use the InputTensor type and outputs from the operation use OutputTensor, instead of the previous ManagedTensorSlice for both. This eliminates the need for a manual num_dps_outputs attribute, and makes it safer to work with these inputs and outputs by preventing accidental writes to input tensors. The new interface looks something like the following:

@compiler.register("add_one_custom")
struct AddOneCustom:
    @staticmethod
    fn execute[
        target: StringLiteral,
    ](
        out: OutputTensor,
        x: InputTensor[type = out.type, rank = out.rank],
        ctx: DeviceContextPtr,
    ) raises:
        @parameter
        @always_inline
        fn elementwise_add_one[
            width: Int
        ](idx: IndexList[x.rank]) -> SIMD[x.type, width]:
            return x.load[width](idx) + 1

        foreach[elementwise_add_one, target=target](out, ctx)
@compiler.register("add_one_custom")
struct AddOneCustom:
    @staticmethod
    fn execute[
        target: StringLiteral,
    ](
        out: OutputTensor,
        x: InputTensor[type = out.type, rank = out.rank],
        ctx: DeviceContextPtr,
    ) raises:
        @parameter
        @always_inline
        fn elementwise_add_one[
            width: Int
        ](idx: IndexList[x.rank]) -> SIMD[x.type, width]:
            return x.load[width](idx) + 1

        foreach[elementwise_add_one, target=target](out, ctx)

The foreach function now raises to be able to handle errors within an elementwise calculation.

Hopper kernels

State-of-the-Art Kernels in Mojo for H100/H200 GPUs
- Hopper Architecture Matrix Multiplication Kernels: The implementation achieved performance comparable to NVIDIA's highly optimized cuBLAS library. These kernels take full advantage of the Tensor Cores in Hopper architecture GPUs to accelerate the fundamental matrix multiplication operations that underpin deep learning workloads.
- Multi-GPU AllReduce Implementation: The AllReduce operation is critical for distributed inference across multiple GPUs, as it efficiently aggregates gradients. The Mojo implementation surpassed NVIDIA's NCCL library in performance benchmarks. This improvement reduces communication overhead during distributed inference.
- MAX Attention Kernel with Flash Attention 3: This implementation incorporates the latest Flash Attention 3 algorithm and extends it, which significantly accelerates the computation of attention mechanisms in transformer models. The MAX attention kernel optimizes memory access patterns and computational steps, reducing both the memory footprint and execution time of attention operations. This is particularly important for LLMs where attention calculations represent a substantial portion of the computational workload.

GPU programming

Added the Mojo max.driver API to enable dispatching GPU functions from Mojo.

Check out examples for GPU programming in Mojo, which use this new API.

Mojo

Mojo is a crucial component of the MAX stack that enables all of MAX's performance-oriented code across hardware. For all the updates to the Mojo language, standard library, and tools, see the Mojo changelog.

v25.1.1 (2025-02-19)

Fix performance issues in autoregressive models with paged attention by setting sensible default values for --max-num-steps that are platform-specific.

✨ Highlights

Custom ops for GPUs

Our new custom op API allows you to extend MAX Engine with new graph operations written in Mojo that execute on either CPU or GPU, providing full composability and extensibility for your models. See more in the section about GPU programming.
Enhanced support for agentic workflows

MAX Serve now supports function calling, which allows you to instruct your model to interact with other systems, such as retrieve data and execute external tasks. Learn more about function calling and tool use.

MAX Serve now supports structured output (also known as constrained decoding) for MAX models on GPU. This allows you to enforce the output format from a model using an input schema that defines the output structure. Learn more about structured output.
Extended model architecture support
- MAX Serve now supports multimodal models that take both text and image inputs. For example, see how to deploy Llama 3.2 Vision.
- MAX Serve now supports text embedding models. Learn how to deploy a text embedding model.
New max-pipelines CLI tool

Instead of cloning our GitHub repo to access our latest GenAI models, you can instead install the max-pipelines CLI tool and quickly run an inference or deploy an endpoint. Learn more in the max-pipelines docs.

Documentation

New tutorials:

Other docs:

MAX Serve

The /v1/completions REST endpoint now supports:
- Pre-tokenized prompts.
- Image inputs for multimodal models such as Llama-3.2-11B-Vision-Instruct. For an example, see how to generate image descriptions with Llama 3.2 Vision.
  
  Known issue: You might receive faulty results because some parts of the text prompt get ignored for certain input combinations. We've identified the problem and will have a fix in a subsequent nightly release.
- Function calling and tool use, which allows you to instruct your model to interact with other systems, such as retrieve data and execute external tasks. Learn more about function calling and tool use.
- Structured output (also known as constrained decoding), which allows you to enforce the output format from a model using a JSON schema and the response_format field. To enable constrained decoding pass --enable-structured-output when running the server. However, this feature currently works for MAX models on GPU only (support for PyTorch models and CPU is in progress). Learn more about structured output.
Added support for the /v1/embeddings API endpoint, allowing you to generate vector representations using embedding models. See how to deploy a text embedding model.
Max Serve can evict requests when the number of available pages in the PagedAttention KVCache is limited. Before, the KV manager would throw an OOM error when a batch that cannot fit in the cache was scheduled.

MAX models

Added the max-pipelines CLI tool that simplifies the process to run inference with GenAI models (specified with a Hugging Face repo ID) and deploy them to a local endpoint with MAX Serve.

Previously, running or serving these models required cloning the modular/max GitHub repo and then running commands such as magic run llama3.

These model-specific commands like llama3 and replit commands have been removed. They're now standardized and subsumed by flags like --model-path in the max-pipelines tool. Arguments such as --max-length and --weight-path are also still supported by max-pipelines.

To view a list of supported model architectures from Hugging Face, run max-pipelines list.
Added support for PagedAttention, which improves memory efficiency by partitioning the KV cache into smaller blocks, reducing fragmentation and enabling larger inference batches. You can enable it with --cache-strategy=paged and --kv-cache-page-size with a value that's a multiple of 128.
Added support for prefix caching in all cases where PagedAttention is supported. This allows for more efficient usage of KVCache and improved prefill performance for workloads with common prefixes. You can enable it by setting --enable-prefix-caching. For more information, see Prefix caching with PagedAttention.
Batch size and max length are now inferred from available memory and the HF Models' default values for max length, respectively. If a configuration leads to an OOM, then we provide recommendations (to the best of our ability) to the user to fit the model into memory.
Added support for heterogeneous KV caches for multi-modal models, such as Llama Vision, which cache different KV states for self and cross attention layers.

Added support for embedding models, starting with MPNet. For example:

max-pipelines generate \
  --model-path=sentence-transformers/all-mpnet-base-v2 \
  --prompt="Encode this sentence."
max-pipelines generate \
  --model-path=sentence-transformers/all-mpnet-base-v2 \
  --prompt="Encode this sentence."

Also see how to deploy a text embedding model.

Added support for image and text multimodal models:
- max-pipelines generate now accepts image input with --image_url.
- Added an experimental Pixtral pipeline you can run as follows:
  max-pipelines generate \ --model-path=mistral-community/pixtral-12b \ --prompt="What is in this image? [IMG]" \ --image_url=/images/artwork/max-serve-cloud.png
  max-pipelines generate \ --model-path=mistral-community/pixtral-12b \ --prompt="What is in this image? [IMG]" \ --image_url=/images/artwork/max-serve-cloud.png
  The pipeline is automatically used for all models implementing the LlavaForConditionalGeneration architecture.
  
  The implementation currently has a limit of one image. We plan support an arbitrary number of images of mixed sizes soon.
- Added an experimental Llama Vision pipeline you can run as follows:
  max-pipelines generate \ --model-path=meta-llama/Llama-3.2-11B-Vision-Instruct \ --prompt="<|image|><|begin_of_text|>What is in this image?" \ --image_url=/images/artwork/max-serve-cloud.png
  max-pipelines generate \ --model-path=meta-llama/Llama-3.2-11B-Vision-Instruct \ --prompt="<|image|><|begin_of_text|>What is in this image?" \ --image_url=/images/artwork/max-serve-cloud.png
  The pipeline is automatically used for all models implementing the MllamaForConditionalGeneration architecture.
  
  Note: This model is gated and requires that you set the HF_TOKEN environment variable. See Llama-3.2-11B-Vision-Instruct.
- See how to generate image descriptions with Llama 3.2 Vision.

Added support for the Qwen2ForCausalLM model architecture (such as Qwen/Qwen2.5-7B-Instruct). For example:

max-pipelines generate \
  --model-path=Qwen/Qwen2.5-7B-Instruct \
  --prompt="Write bubble sort in python" \
  --quantization-encoding bfloat16
max-pipelines generate \
  --model-path=Qwen/Qwen2.5-7B-Instruct \
  --prompt="Write bubble sort in python" \
  --quantization-encoding bfloat16

Added support for offline batched inference for text-based LLMs, allowing you to load a model and run inference with a batch of inputs directly from Python, instead of relying on an HTTP interface. For an example, see examples/offline-inference/basic.py.
The --max-cache-batch-size flag has been deprecated in favor of --max-batch-size. Using --max-cache-batch-size now emits a deprecation warning and will stop working in a future release.
The --use-gpu flag has been deprecated in favor of --devices=cpu, --devices=gpu, or --devices=gpu-0,gpu-1,.... If the device isn't specified, the model runs on the first available GPU, or CPU if no GPUs are available.

MAX Engine

Improved internal kernel compilation speed 1.5 - 4X across different models.

We've revamped our GPU compilation process so that all kernels in a program are compiled together into a single LLVM module, then split into separate kernels afterward. This ensures shared code between kernel entry points is only compiled once. For example, we observe a 3.7x speed up for Llama3.1-8b GPU startup time.
Improved initial model execution speed on NVIDIA GPUs.

Instead of compiling to PTX and performing just-in-time compilation during runtime, we now generate CUBIN binaries directly. While this increases initial compilation time, it significantly improves execution speed.
The kernels have been further tuned for performance on NVIDIA A100 GPUs.

Graph APIs

You can now write custom operations (ops) in Mojo, and add them to a graph constructed in Python, using custom() and inplace_custom().

For more detail, see the section below about GPU programming.
Cached compiled MAX graphs that make use of custom operations now get invalidated when the implementation of the custom operations change.
Graph.add_weight() now takes an explicit device argument. This enables explicitly passing GPU-resident weights to session.load() via the weights registry to initialize the model.
max.graph.Weight now inherits from TensorValue, allowing you to call weight.cast() or weight.T. As such, the TensorValue no longer accepts Weight for the value argument.

Pipeline APIs

TextTokenizer.new_context() now supports tool definitions passed through its request argument (via TokenGeneratorRequest.tools).

It also now supports JSON schemas passed through its request argument (via TokenGeneratorRequest.response_format).
Removed the default num_steps value for TokenGenerator.next_token(), ensuring users pass a value, reducing the potential for silent errors.
KVCacheStrategy now defaults to MODEL_DEFAULT.

As opposed to the previous setting which always used the "continuous" caching strategy, KV caching strategy is now defaulted on an architecture-specific basis to ensure the most optimized caching strategy is used.
The Linear layer now has a create() class method that automatically creates specializations of Linear for non-quantized, k-quant, or GPTQ layers.
Added nn.Conv1D for audio models like Whisper.

GPU programming

This release includes all new APIs to program on GPUs. The way to write code for GPUs is to create custom operations with GPU functions that you can load into a MAX graph. This foundational API includes a few key components:

Mojo APIs to write custom op functions:
- The @compiler.register decorator is applied to a Mojo struct that implements a custom op in an execute() function—for either CPU or GPU—and a shape() function that defines the custom op's output tensor.
- The max.tensor package adds essential Mojo APIs for writing custom ops, such as:
  - The foreach() function, which efficiently executes an element-wise computation in parallel on either a GPU or CPU.
  - The ManagedTensorSlice type defines the input and output tensors for the custom op.
Python APIs to load custom ops into a model:
- The custom() and inplace_custom() functions allow you to add the previously-defined Mojo custom op to a MAX graph written in Python.
- The InferenceSession constructor accepts the custom op implementation as a Mojo package in the custom_extensions argument.

For more detail, see the tutorial to build custom ops for GPUs, or check out this simple example of a custom op.

Additionally, we've added a new gpu package to the Mojo standard library that provides low-level programming constructs for working with GPUs. These APIs let you do things that you can’t currently do with the high-level foreach() abstraction above. The Mojo gpu APIs allow you to manually manage interaction between the CPU host and GPU device, manage memory between devices, synchronize threads, and more. For some examples, see vector_addition.mojo and top_k.mojo.

Mojo

Mojo is a crucial component of the MAX stack that enables all of MAX’s performance-oriented code across hardware. For all the updates to the Mojo language, standard library, and tools, see the Mojo changelog.

v24.6 (2024-12-17)

This is a huge update that offers a first look at our serving library for MAX on GPUs!

Highlights
Documentation
MAX Serve
MAX models
MAX Engine
Mojo

Also check out our blog post introducing MAX 24.6.

✨ Highlights

MAX Engine on GPUs preview

We’re excited to share a preview of MAX Engine on GPUs. We’ve created a few tutorials that demonstrate MAX’s ability to run GenAI models with our next-generation MAX graph compiler on NVIDIA GPU architectures (including A100, A10, L4, and L40 GPUs). You can experience it today by deploying Llama 3 on an A100 GPU.
MAX Serve preview

This release also includes an all-new serving interface called MAX Serve. It's a Python-based serving layer that supports both native MAX models when you want a high-performance deployment, and off-the-shelf PyTorch LLMs from Hugging Face when you want to explore and experiment—all with GPU support. It provides an OpenAI-compatible REST endpoint for inference requests, and a Prometheus-compatible metrics endpoint. You can use a magic command to start a local server , or use our ready-to-deploy MAX container to start an endpoint in the cloud. Try it now with an LLM from Hugging Face.
Upgraded MAX models

As we continue to build our Python-based MAX Graph API that allows you to build high-performance GenAI models, we’ve made a ton of performance improvements to the existing models and added a few new models to our GitHub repo. All the Python-based MAX models now support GPUs and broad model architectures. For example, llama3 adds compatibility for the LlamaForCausalLM family, which includes over 20,000 model variants and weights on Hugging Face.

Documentation

New tutorials:

Other new docs:

Also, our documentation is now available for MAX nightly builds! If you’re building with a MAX nightly release, you can switch to see the nightly docs using a toggle to the right of the search bar.

MAX Serve

This release includes a preview of our Python-based serving library called MAX Serve. It simplifies the process to deploy your own inference server with consistent and reliable performance.

MAX Serve currently includes the following features:

Deploys locally and to the cloud with our MAX container image, or with the magic CLI.
An OpenAI-compatible server with streaming /chat/completion and /completion endpoints for LLM inference requests.
Prometheus-compatible metrics endpoint with LLM KPIs (TTFT and ITL) for monitoring and evaluating performance.
Supports most TextGeneration Hugging Face Hub models.
Multiprocess HTTP/model worker architecture to maximize CPU core utilization by distributing multiple incoming requests across multiple processes, ensuring both high throughput and responsiveness.
Continuous heterogeneous batching to combine multiple incoming requests into a single inference (no waiting to fill a batch size) and improve total throughput.

There’s much more still in the works for MAX Serve, but you can try it today with our tutorials to Deploy Llama 3 on GPU with MAX Serve and Deploy a PyTorch model from Hugging Face.

Known issues:

While this release is enough to support typical chatbot applications, this release does not yet support the function-calling portion of the OpenAI API specification needed to enable robust agentic workflows.
Sampling is still limited and doesn’t currently respect temperature or other sampling-related API request input.
Structured generation is not supported.
Support for multi-modal models is still nascent.

MAX models

All of our Python-based GenAI models on GitHub now support GPUs!

As we add more models, we’re also building a robust set of libraries and infrastructure that make it easier to build and deploy a growing library of LLMs. Some of which is available in a new max.pipelines package and some of it is alongside the models on GitHub. Here are just some of the highlights:

Deep integration with the Hugging Face ecosystem for a quick-to-deploy experience, such as using HF Model Hub tools to fetch config files, support for weights in safetensor format, support for HF tokenizers, and more. (We also support GGUF weight formats.)
Expanded set of model abstractions for use by different LLM architectures:
- Attention layers (including highly optimized implementations with configurable masking, like AttentionWithRope). The optimized attention layers include variants that accept an attention mask. More memory-efficient variants that don’t take a mask instead take a “mask functor” argument to the kernel, which implements masking without materializing a mask by computing a mask value from input coordinates on the fly.
- Transformers such as Transformer and TransformerBlock. These include an initial implementation of ragged tensors—tensors for which each dimension can have a different size, avoiding the use of padding tokens by flattening a batch of sequences of differing lengths.
- Common layers such as RMSNorm , Embedding, and Sequential.
- KV cache management helpers, like ContinuousBatchingKVCacheManager.
- Low-level wrappers over optimized kernels like fused_qk_ragged_rope. These are custom fused kernels that update the KV cache in place. Although they are custom, they reuse the underlying kernel implementation by passing in lambda functions used to retrieve inputs and write to outputs in place.
Added generalized interfaces for text generation such as TokenGenerator and PipelineModel, which provide modularity within the models and serving infrastructure. Also added a plug-in mechanism (PipelineRegistry) to more quickly define new models, tokenizers, and other reusable components. For example, anything that conforms to TokenGenerator can be served using the LLM infrastructure within MAX Serve. We then used this interface to create the following:
- An optimized TextGenerationPipeline that can be combined with any compatible graph and has powerful performance features like graph-based multi-step scheduling, sampling, KV cache management, ragged tensor support, and more.
- A generic HFTextGenerationPipeline that can run any Hugging Face model for which we don’t yet have an optimized implementation in eager mode.
Models now accept weights via a weights registry, which is passed to the session.load() method’s weights_registry argument. The decoupling of weights and model architecture allows implementing all of the different fine-tunes for a given model with the same graph. Furthermore, because the underlying design is decoupled, we can later expose the ability to compile a model once and swap weights out on the fly, without re-compiling the model.
Added generic implementations of common kernels, which allow you to plug-in different batching strategies (ragged or padded), KV cache management approaches (continuous batching), masking (causal, sliding window, etc.), and position encoding (RoPE or ALIBI) without having to re-write any kernel code. (More about this in a future release.)
Multi-step scheduling to run multiple token-generation steps on GPU before synchronizing to the CPU.

Updated models:

Significant performance upgrades for Llama 3, and expanded compatibility with the LlamaForCausalLM models family. For example, it also supports Llama 3.2 1B and 3B text models.

New models:

Mistral NeMo (and other MistralForCausalLM models)
Replit Code V1.5 3B

Known issues:

The Q4 quantized models currently work on CPU only.
Using a large setting for top-k with the Llama 3.1 model may lead to segmentation faults for certain workloads when run on NVIDIA GPUs. This should be resolved in the latest nightly MAX builds.
The models currently use a smaller default context window than the max_seq_len specified in the Hugging Face configuration files for a given model. This can be manually adjusted by setting the --max-length parameter to the desired context length when serving a model.
Some variants of the supported core models (like LlamaForCausalLM with different number of heads, head sizes, etc.) might not be fully optimized yet. We plan to fully generalize our implementations in a future release.

MAX Engine

MAX Engine includes a lot of the core infrastructure that enables MAX to accelerate AI models on any hardware, such as the graph compiler, runtime, kernels, and the APIs to interact with it all, and it all works without external dependencies such as PyTorch or CUDA.

This release includes a bunch of performance upgrades to our graph compiler and runtime. We’ve added support for NVIDIA GPU architectures (including A100, A10, L4, and L40 GPUs), and built out new infrastructure so we can quickly add support for other GPU hardware.

Engine API changes:

InferenceSession now accepts a custom_extensions constructor argument, same as load(), to specify model extension libraries.
The Model object is now callable to run an inference.

Breaking changes:

Model.execute() signature changed to support GPUs.
- The execute() function currently doesn’t accept keyword arguments. Instead you can pass tensors as a driver.Tensor, int, float, bool, np.generic, or DLPackArray (DLPack). Note that both PyTorch and NumPy arrays implement the DLPack protocol, which means you can also pass either of those types to execute().
- execute_legacy() preserves the semantics of execute() with support for keyword arguments to help with migration, but will be removed in a future release. execute_legacy() doesn't support GPUs.
- Calling execute() with positional arguments still works the same.

Driver APIs

MAX Driver (the max.driver module) is a new component of MAX Engine that’s still a work in progress. It provides primitives for working with heterogeneous hardware systems (GPUs and CPUs), such as to allocate on-device memory, transfer data between host and device, query device stats, and more. It’s a foundation on which other components of MAX Engine operate (for example, InferenceEngine now uses driver.Tensor to handle model inputs and outputs).

Driver API changes:

Added CUDA() device to open an NVIDIA GPU.
Added support for fp16 and bfloat16 dtypes.
Expanded functionality for max.driver.Device, with new class methods and properties. We are still working on building this out to support more accelerator features.
driver.Tensor (and the InferenceSession.load() argument weights_registry ) now supports zero-copy interoperability with NumPy arrays and PyTorch tensors, using DLPack / DLPackArray.
driver.Tensor has new methods, such as from_dlpack(), element_size() , to(), to_numpy(), view(), zeros(), and more.

MAX Driver APIs are still changing rapidly and not yet ready for general use. We’ll publish more documentation in a future release.

Known issues:

MAX Driver is currently limited to managing just one NVIDIA GPU at a time (it does not yet support multi-GPU). It also does not yet support remote devices.
DLPack support is not complete. For example, streams are not yet supported.

Graph compiler

When you load a model into MAX Engine, the graph compiler is the component that inspects and optimizes all graph operations (ops) to deliver the best run time performance on each device.

This release includes various graph compiler improvements:

Major extensions to support NVIDIA GPUs (and other devices in the future), including async copies and caching of JIT’d kernels.
The runtime now performs scheduling to enable GPU compute overlap with the CPU.
New transformations to the Mojo kernels to enable a number of optimizations, including specialization on tensor dimensions, specialization on target hardware, specialization on non-tensor dimension input to kernels, automatic kernel fusion between operators, and more.
New algebraic simplifications and algorithms for ops such as horizontal fusion of matrix multiplications.
New CPU-side primitives for device management that are automatically transformed and optimized to reduce overhead (MAX does not need to use things like CUDA Graphs).
Updated memory planning to preallocate device memory (hoist computation from inference runtime to initialization time) and reduce per-inference overhead.

Graph APIs

The graph compiler is also exposed through the MAX Graph APIs (the max.graph package), which allow you to build high-performance GenAI models in Python.

Graph API changes:

Python stack traces from model execution failures now include a trace to the original op-creation, allowing for easier debugging during development.
The max.graph APIs now include preliminary support for symbolic algebraic expressions using AlgebraicDim, enabling more powerful support for checked dynamic shapes. This allows -Dim("x") - 4. Furthermore, the algebraic expressions simplify to a canonical form, so that for example -Dim("x") - 4 == -(Dim("x") + 4) holds.
More advanced dtype promotion now allows TensorValue math operators to just work when used with NumPy arrays and python primitives.
TensorValue has new methods, such as broadcast_to(), cast(), flatten(), permute(), and more.
Added BufferValue, which allows for device-resident tensors that are read and mutated within the graph.
DType has new methods/properties, align, size_in_bytes, and is_float().
Value constructor accepts more types for value.
TensorValue constructor accepts more types for value.
TensorValue.rebind() accepts a new message argument.

Breaking changes:

Graph.add_weight() now accepts Weight and returns TensorValue. Weight is essentially a named placeholder for a tensor that knows its name, dtype, shape, and optionally device and quantization encoding. Graph.add_weight() stages an op in the graph that is populated by a named weight in the weights registry passed to session.load.
The Weight constructor arguments changed; added align , dtype , and shape; removed assign , filepath, offset, and value.
The ops.scalar() method was removed along with the is_static() and is_symbolic() methods from all graph.type objects.
- Instead of ops.scalar(), use ops.constant().
- Instead of is_static() and is_symbolic(), use isinstance(dim, SymbolicDim) and isinstance(dim, StaticDim).

The MAX Graph APIs are not ready for general use but you can experiment with it now by following this tutorial. We'll add more documentation when we finish some API redesigns.

Custom op registration

Although the APIs to write custom operators (ops) isn’t ready for general use, this release includes a significant redesign that lays the groundwork. You might notice some associated APIs in this release and more APIs in the nightlies, so here’s a little about the work in progress:

The custom op APIs will allow you to extend MAX Engine with new ops written in Mojo, providing full composability and extensibility for your models. It’s the exact same API we use to write MAX Engine’s built-in ops such as matmul. That means your custom ops can benefit from all our compiler optimization features such as kernel fusion—your ops are treated the same as all the ops included “in the box.”
The new API requires far less adornment at the definition site to enable the MAX model compiler to optimize custom ops along with the rest of the graph (compared to our previous version that used NDBuffer).
Custom ops support “destination passing style” for tensors.
The design composes on top of Mojo’s powerful meta programming, as well as the kernel libraries abstractions for composable kernels.

We’ll publish more documentation when the custom op API is ready for general use. Check out the MAX repo’s nightly branch to see the latest custom op examples.

Known issues:

Custom ops don't have type or lifetime checking. They also don't reason about mutability. Expect lots of sharp corners and segfaults if you hold them wrong while we improve this!

Numeric kernels

The GPU kernels for MAX Engine are built from the ground up in Mojo with no dependencies on external vendor code or libraries. This release includes the following kernel improvements:

AttenGen: a novel way to express attention pattern that’s able to express different attention masks, score functions, as well as caching strategies.
State-of-the-art matrix multiplication algorithms with optimizations such as the following:
- Pipelining and double-buffering to overlap data transfer and computation and to hide memory access latency (for both global and shared memory).
- Thread swizzling to avoid shared memory bank conflicts associated with tensor core layouts.
- Block swizzling to increase L2 cache locality.
SplitK/StreamK GEMM algorithms: divides the computation along the shared K dimension into smaller matrices which can then be executed independently on streaming multiprocessors (such as CUDA cores). These algorithms are ideal for matrices with large K dimension but small M dimension.
Large context length MHA: uses SplitK/StreamK to implement the attention mechanism and eliminate the need of a huge score matrix, which drastically reduces memory usage/traffic to enable large context length.
DualGemm: accelerates the multi-layer perceptron (MLP) layers where the left-hand side (LHS) is shared between two matrix multiplications.

Known issues:

The MAX kernels are optimized for bfloat16 on GPUs.
Convolution on GPU is not performance optimized yet.
Although v24.6 technically runs on H100, it doesn’t include performance-optimized kernels for that device yet and it isn’t recommended.

Mojo

v24.5 (2024-09-13)

✨ Highlights

Mojo and MAX are magical! We've created a new package and virtual environment manager, magic, for MAX and Mojo. Check it out!
New Llama3.1 pipeline built with the new MAX Graph Python API.
We have not one, but two new Python APIs that we're introducing in this release:
- MAX Graph Python API
- MAX Driver Python API

⭐️ New

Added repeat_interleave graph op.
Added caching for MAX graph models. This means that graph compilation is cached and the executable model is retrieved from cache on the 2nd and subsequent runs. Note that the model cache is architecture specific and isn't portable across different targets.
Support for Python 3.12.

MAX Graph Python API

This Python API will ultimately provide the same low-level programming interface for high-performance inference graphs as the Mojo API. As with the Mojo API, it's an API for graph-building only, and it does not implement support for training.

You can take a look at how the API works in the MAX Graph Python API reference.

MAX Driver Python API

The MAX Driver API allows you to interact with devices (such as CPUs and GPUs) and allocate memory directly onto them. With this API, you interact with this memory as tensors.

Note that this API is still under development, with support for non-host devices, such as GPUs, planned for a future release.

To learn more, check out the MAX Driver Python APIreference.

MAX C API

New APIs for adding torch metadata libraries:

M_setTorchMetadataLibraryPath
M_setTorchMetadataLibraryPtr

🦋 Changed

MAX Engine performance

Compared to v24.4, MAX Engine v24.5 generates tokens for Llama an average of 15%-48% faster.

MAX C API

Simplified the API for adding torch library paths, which now only takes one path per API call, but can be called multiple times to add paths to the config:

M_setTorchLibraries -> M_setTorchLibraryPath

⚠️ Deprecated

The max command line tool is no longer supported and will be removed in a future release.

❌ Removed

Dropped support for Ubuntu 20.04. If you're using Ubuntu, we currently support Ubuntu 22.04 LTS only.
Dropped support for Python 3.8.
Removed built-in PyTorch libraries from the max package. See the FAQ for information on supported torch versions.

v24.4 (2024-06-07)

🔥 Legendary

MAX is now available on macOS! Try it now.
New quantization APIs for MAX Graph. You can now build high-performance graphs in Mojo that use the latest quantization techniques, enabling even faster performance and more system compatibility for large models.

Learn more in the guide to quantize your graph weights.

⭐️ New

MAX Mojo APIs

Added AI pipeline examples in the max repo, with Mojo implementations for common transformer layers, including quantization support.
- New Llama3 pipeline built with MAX Graph.
- New Replit Code pipeline built with MAX Graph.
- New TinyStories pipeline (based on TinyLlama) that offers a simple demo of the MAX Graph quantization API.
Added Mojo API inference example with the TorchScript BERT model.
Added max.graph.checkpoint package to save and load model weights.

All weights are stored in a TensorDict. You can save and load a TensorDict to disk with save() and load() functions.
Added MAX Graph quantization APIs:
- Added quantization encodings BFloat16Encoding, Q4_0Encoding, Q4_KEncoding, and Q6_KEncoding.
- Added the QuantizationEncoding trait so you can build custom quantization encodings.
- Added Graph.quantize() to create a quantized tensor node.
- Added qmatmul() to perform matrix-multiplication with a float32 and a quantized matrix.
Added some MAX Graph ops:

Added a layer() context manager and current_layer() function to aid in debugging during graph construction. For example:

with graph.layer("foo"):
    with graph.layer("bar"):
        print(graph.current_layer())  # prints "foo.bar"
        x = graph.constant[DType.int64](1)
        graph.output(x)
with graph.layer("foo"):
    with graph.layer("bar"):
        print(graph.current_layer())  # prints "foo.bar"
        x = graph.constant[DType.int64](1)
        graph.output(x)

This adds a path foo.bar to the added nodes, which will be reported during errors.

Added format_system_stack() function to format the stack trace, which we use to print better error messages from error().
Added TensorMap.keys() to get all the tensor key names.

MAX C API

Miscellaneous new APIs:

M_cloneCompileConfig()
M_copyAsyncTensorMap()
M_tensorMapKeys() and M_deleteTensorMapKeys()
M_setTorchLibraries()

🦋 Changed

MAX Mojo API

EngineNumpyView.data() and EngineTensorView.data() functions that return a type-erased pointer were renamed to unsafe_ptr().
TensorMap now conforms to CollectionElement trait to be copyable and movable.
custom_nv() was removed, and its functionality moved into custom() as an function overload, so it can now output a list of tensor symbols.

v24.3 (2024-05-02)

🔥 Legendary

You can now write custom ops for your models with Mojo!

Learn more about MAX extensibility.

🦋 Changed

Added support for named dynamic dimensions. This means you can specify when two or more dimensions in your model's input are dynamic but their sizes at run time must match each other. By specifying each of these dimension sizes with a name (instead of using None to indicate a dynamic size), the MAX Engine compiler can perform additional optimizations. See the notes below for the corresponding API changes that support named dimensions.
Simplified all the APIs to load input specs for models, making them more consistent.

MAX Engine performance

Compared to v24.2, MAX Engine v24.3 shows an average speedup of 10% on PyTorch models, and an average 20% speedup on dynamically quantized ONNX transformers.

MAX Graph API

The max.graph APIs are still changing rapidly, but starting to stabilize.

See the updated guide to build a graph with MAX Graph.

AnyMoType renamed to Type, MOTensor renamed to TensorType, and MOList renamed to ListType.
Removed ElementType in favor of using DType.
Removed TypeTuple in favor of using List[Type].
Removed the Module type so you can now start building a graph by directly instantiating a Graph.
Some new ops in max.ops, including support for custom ops.

See how to create a custom op in MAX Graph.

MAX Engine Python API

Redesigned InferenceSession.load() to replace the confusing options argument with a custom_ops_path argument.

As a result, CommonLoadOptions, TorchLoadOptions, and TensorFlowLoadOptions have all been removed.
TorchInputSpec now supports named dynamic dimensions (previously, dynamic dimension sizes could be specified only as None). This lets you tell MAX which dynamic dimensions are required to have the same size, which helps MAX better optimize your model.

MAX Engine Mojo API

InferenceSession.load_model() was renamed to load().
Redesigned InferenceSession.load() to replace the confusing config argument with a custom_ops_path argument for use when loading a custom op, and an input_specs argument for use when loading TorchScript models.

Doing so removed LoadOptions and introduced the new InputSpec type to define the input shape/type of a model (instead of LoadOptions).
New ShapeElement type to allow for named dynamic dimensions (in InputSpec).
max.engine.engine module was renamed to max.engine.info.

MAX Engine C API

M_newTorchInputSpec() now supports named dynamic dimensions (via new dimNames argument).

❌ Removed

Removed TensorFlow support in the MAX SDK, so you can no longer load a TensorFlow SavedModel for inference. However, TensorFlow is still available for enterprise customers.

We removed TensorFlow because industry-wide TensorFlow usage has declined significantly, especially for the latest AI innovations. Removing TensorFlow also cuts our package size by over 50% and accelerates the development of other customer-requested features. If you have a production use-case for a TensorFlow model, please contact us.
Removed the Python CommonLoadOptions, TorchLoadOptions, and TensorFlowLoadOptions classes. See note above about InferenceSession.load() changes.
Removed the Mojo LoadOptions type. See the note above about InferenceSession.load() changes.

v24.2.1 (2024-04-11)

You can now import more MAX Graph functions from max.graph.ops instead of using max.graph.ops.elementwise. For example:

from max.graph import ops

var relu = ops.relu(matmul)
from max.graph import ops

var relu = ops.relu(matmul)

v24.2 (2024-03-28)

MAX Engine now supports TorchScript models with dynamic input shapes.

No matter what the input shapes are, you still need to specify the input specs for all TorchScript models.
The Mojo standard library is now open source!

Read more about it in this blog post.
And, of course, lots of Mojo updates, including implicit traits, support for keyword arguments in Python calls, a new List type (previously DynamicVector), some refactoring that might break your code, and much more.

For details, see the Mojo changelog.

v24.1.1 (2024-03-18)

This is a minor release that improves error reports.

v24.1 (2024-02-29)

The first release of the MAX platform is here! 🚀

This is a preview version of the MAX platform. That means it is not ready for production deployment and designed only for local development and evaluation.

Because this is a preview, some API libraries are still in development and subject to change, and some features that we previously announced are not quite ready yet. But there is a lot that you can do in this release!

This release includes our flagship developer tools, currently for Linux only:

MAX Engine: Our state-of-the-art graph compiler and runtime library that executes models from PyTorch and ONNX, with incredible inference speed on a wide range of hardware.
- API libraries in Python, C, and Mojo to run inference with your existing models. See the API references.
- The max benchmark tool, which runs MLPerf benchmarks on any compatible model without writing any code.
- The max visualize tool, which allows you to visualize your model in Netron after partially lowering in MAX Engine.
- An early look at the MAX Graph API, our low-level library for building high-performance inference graphs.
MAX Serving: A preview of our serving wrapper for MAX Engine that provides full interoperability with existing AI serving systems (such as Triton) and that seamlessly deploys within existing container infrastructure (such as Kubernetes).
- A Docker image that runs MAX Engine as a backend for NVIDIA Triton Inference Server.
Mojo: The world's first programming language built from the ground-up for AI developers, with cutting-edge compiler technology that delivers unparalleled performance and programmability for any hardware.
- The latest version of Mojo, the standard library, and the mojo command line tool. These are always included in MAX, so you don't need to download any separate packages.
- The Mojo changes in each release are often quite long, so we're going to continue sharing those in the existing Mojo changelog.

Additionally, we've started a new GitHub repo for MAX, where we currently share a bunch of code examples for our API libraries, including some large model pipelines. You can also use this repo to report issues with MAX.

v25.3 nightly​

v25.2​

Highlights​

Documentation​

MAX Serve​

MAX models​

max-pipelines CLI​

MAX Engine​

Driver APIs​

Graph APIs​

Custom ops​

Hopper kernels​

GPU programming​

Mojo​

v25.1.1 (2025-02-19)​

v25.1 (2025-02-13)​

✨ Highlights​

Documentation​

MAX Serve​

MAX models​

MAX Engine​

Graph APIs​

Pipeline APIs​

GPU programming​

Mojo​

v24.6 (2024-12-17)​

✨ Highlights​

Documentation​

MAX Serve​

MAX models​

MAX Engine​

Driver APIs​

Graph compiler​

Graph APIs​

Custom op registration​

Numeric kernels​

Mojo​

v24.5 (2024-09-13)​

✨ Highlights​

⭐️ New​

MAX Graph Python API​

MAX Driver Python API​

MAX C API​

🦋 Changed​

MAX Engine performance​

MAX C API​

⚠️ Deprecated​

❌ Removed​

v24.4 (2024-06-07)​

🔥 Legendary​

⭐️ New​

MAX Mojo APIs​

MAX C API​

🦋 Changed​

MAX Mojo API​

v24.3 (2024-05-02)​

🔥 Legendary​

🦋 Changed​

MAX Engine performance​

MAX Graph API​

MAX Engine Python API​

MAX Engine Mojo API​

MAX Engine C API​

❌ Removed​

v24.2.1 (2024-04-11)​

v24.2 (2024-03-28)​

v24.1.1 (2024-03-18)​

v24.1 (2024-02-29)​

v25.3 nightly

v25.2

Highlights

Documentation

MAX Serve

MAX models

`max-pipelines` CLI

MAX Engine

Driver APIs

Graph APIs

Custom ops

Hopper kernels

GPU programming

Mojo

v25.1.1 (2025-02-19)

v25.1 (2025-02-13)

✨ Highlights

Documentation

MAX Serve

MAX models

MAX Engine

Graph APIs

Pipeline APIs

GPU programming

Mojo

v24.6 (2024-12-17)

✨ Highlights

Documentation

MAX Serve

MAX models

MAX Engine

Driver APIs

Graph compiler

Graph APIs

Custom op registration

Numeric kernels

Mojo

v24.5 (2024-09-13)

✨ Highlights

⭐️ New

MAX Graph Python API

MAX Driver Python API

MAX C API

🦋 Changed

MAX Engine performance

MAX C API

⚠️ Deprecated

❌ Removed

v24.4 (2024-06-07)

🔥 Legendary

⭐️ New

MAX Mojo APIs

MAX C API

🦋 Changed

MAX Mojo API

v24.3 (2024-05-02)

🔥 Legendary

🦋 Changed

MAX Engine performance

MAX Graph API

MAX Engine Python API

MAX Engine Mojo API

MAX Engine C API

❌ Removed

v24.2.1 (2024-04-11)

v24.2 (2024-03-28)

v24.1.1 (2024-03-18)

v24.1 (2024-02-29)