v24.3 (2024-05-02)

🔥 Legendary

You can now write custom ops for your models with Mojo!

Learn more about MAX extensibility.

Added support for named dynamic dimensions. This means you can specify when two or more dimensions in your model's input are dynamic but their sizes at run time must match each other. By specifying each of these dimension sizes with a name (instead of using None to indicate a dynamic size), the MAX Engine compiler can perform additional optimizations. See the notes below for the corresponding API changes that support named dimensions.
Simplified all the APIs to load input specs for models, making them more consistent.

Compared to v24.2, MAX Engine v24.3 shows an average speedup of 10% on PyTorch models, and an average 20% speedup on dynamically quantized ONNX transformers.

The max.graph APIs are still changing rapidly, but starting to stabilize.

AnyMoType renamed to Type, MOTensor renamed to TensorType, and MOList renamed to ListType.
Removed ElementType in favor of using DType.
Removed TypeTuple in favor of using List[Type].
Removed the Module type so you can now start building a graph by directly instantiating a Graph.
Some new ops in max.ops, including support for custom ops.

See how to create a custom op in MAX Graph.

Redesigned InferenceSession.load() to replace the confusing options argument with a custom_ops_path argument.

As a result, CommonLoadOptions, TorchLoadOptions, and TensorFlowLoadOptions have all been removed.
TorchInputSpec now supports named dynamic dimensions (previously, dynamic dimension sizes could be specified only as None). This lets you tell MAX which dynamic dimensions are required to have the same size, which helps MAX better optimize your model.

InferenceSession.load_model() was renamed to load().
Redesigned InferenceSession.load() to replace the confusing config argument with a custom_ops_path argument for use when loading a custom op, and an input_specs argument for use when loading TorchScript models.

Doing so removed LoadOptions and introduced the new InputSpec type to define the input shape/type of a model (instead of LoadOptions).
New ShapeElement type to allow for named dynamic dimensions (in InputSpec).
max.engine.engine module was renamed to max.engine.info.

M_newTorchInputSpec() now supports named dynamic dimensions (via new dimNames argument).

Removed TensorFlow support in the MAX SDK, so you can no longer load a TensorFlow SavedModel for inference. However, TensorFlow is still available for enterprise customers.

We removed TensorFlow because industry-wide TensorFlow usage has declined significantly, especially for the latest AI innovations. Removing TensorFlow also cuts our package size by over 50% and accelerates the development of other customer-requested features. If you have a production use-case for a TensorFlow model, please contact us.
Removed the Python CommonLoadOptions, TorchLoadOptions, and TensorFlowLoadOptions classes. See note above about InferenceSession.load() changes.
Removed the Mojo LoadOptions type. See the note above about InferenceSession.load() changes.