Mojođ„ programming manual
Mojo is a programming language that is as easy to use as Python but with the performance of C++ and Rust. Furthermore, Mojo provides the ability to leverage the entire Python library ecosystem.
Mojo achieves this feat by utilizing next-generation compiler technologies with integrated caching, multithreading, and cloud distribution technologies. Furthermore, Mojoâs autotuning and compile-time metaprogramming features allow you to write code that is portable to even the most exotic hardware.
More importantly, Mojo allows you to leverage the entire Python ecosystem so you can continue to use tools you are familiar with. Mojo is designed to become a superset of Python over time by preserving Pythonâs dynamic features while adding new primitives for systems programming. These new system programming primitives will allow Mojo developers to build high-performance libraries that currently require C, C++, Rust, CUDA, and other accelerator systems. By bringing together the best of dynamic languages and systems languages, we hope to provide a unified programming model that works across levels of abstraction, is friendly for novice programmers, and scales across many use cases from accelerators through to application programming and scripting.
This document is an introduction to the Mojo programming language, fit for consumption by Mojo programmers. It assumes knowledge of Python and systems programming concepts but it does not expect the reader to be a compiler nerd. At the moment, Mojo is still a work in progress and the documentation is targeted to developers with systems programming experience. As the language grows and becomes more broadly available, we intend for it to be friendly and accessible to everyone, including beginner programmers. Itâs just not there today.
Using the Mojo compiler
You can run a Mojo program from a terminal just like you can with Python. So if you have a file named hello.mojo
(or hello.đ„
âyes, the file extension can be an emoji!), just type mojo hello.mojo
:
$ cat hello.đ„def main():
print("hello world")
for x in range(9, 0, -3):
print(x)
$ mojo hello.đ„
hello world9
6
3
$
Again, you can use either the emoji or the .mojo
suffix.
If you are interested in diving into the internal implementation details of Mojo, it can be instructive to look at types in the standard library, example code in notebooks, blogs and other sample code.
Basic systems programming extensions
Given our goal of compatibility and Pythonâs strength with high-level applications and dynamic APIs, we donât have to spend much time explaining how those portions of the language work. On the other hand, Pythonâs support for systems programming is mainly delegated to C, and we want to provide a single system that is great in that world. As such, this section breaks down each major component and feature and describes how to use them with examples.
let
and var
declarations
Inside a def
in Mojo, you may assign a value to a name and it implicitly creates a function scope variable just like in Python. This provides a very dynamic and low-ceremony way to write code, but it is a challenge for two reasons:
- Systems programmers often want to declare that a value is immutable for type-safety and performance.
- They may want to get an error if they mistype a variable name in an assignment.
To support this, Mojo provides scoped runtime value declarations: let
is immutable, and var
is mutable. These values use lexical scoping and support name shadowing:
def your_function(a, b):
let c = a
= b # error: c is immutable
c
if c != b:
var c = b
stuff()
let
and var
declarations support type specifiers as well as patterns, and late initialization:
def your_function():
let x: SI8 = 42
let y: SI64 = 17
let z: SI8
if x != 0:
= 1
z else:
= foo()
z use(z)
Note that let
and var
are completely opt-in when in def
declarations. You can still use implicitly declared values as with Python, and they get function scope as usual.
struct
types
Mojo is based on MLIR and LLVM, which offer a cutting-edge compiler and code generation system used in many programming languages. This lets us have better control over data organization, direct access to data fields, and other ways to improve performance. An important feature of modern systems programming languages is the ability to build high-level and safe abstractions on top of these complex, low-level operations without any performance loss. In Mojo, this is provided by the struct
type.
A struct
in Mojo is similar to a Python class
: they both support methods, fields, operator overloading, decorators for metaprogramming, etc. Their differences are as follows:
Python classes are dynamic: they allow for dynamic dispatch, monkey-patching (or âswizzlingâ), and dynamically binding instance properties at runtime.
Mojo structs are static: they are bound at compile-time (you cannot add methods at runtime). Structs allow you to trade flexibility for performance while being safe and easy to use.
Hereâs a simple definition of a struct:
@value
struct MyPair:
var first: Int
var second: Int
def __lt__(self, rhs: MyPair) -> Bool:
return self.first < rhs.first or
self.first == rhs.first and
(self.second < rhs.second)
Syntactically, the biggest difference compared to a Python class
is that all instance properties in a struct
must be explicitly declared with a var
or let
declaration.
In Mojo, the structure and contents of a âstructâ are set in advance and canât be changed while the program is running. Unlike in Python, where you can add, remove, or change attributes of an object on the fly, Mojo doesnât allow that for structs. This means you canât use del
to remove a method or change its value in the middle of running the program.
However, the static nature of struct
has some great benefits! It helps Mojo run your code faster. The program knows exactly where to find the structâs information and how to use it without any extra steps or delays.
Mojoâs structs also work really well with features you might already know from Python, like operator overloading (which lets you change how math symbols like +
and -
work with your own data). Furthermore, all the âstandard typesâ (like Int
, Bool
, String
and even Tuple
) are made using structs. This means theyâre part of the standard set of tools you can use, rather than being hardwired into the language itself. This gives you more flexibility and control when writing your code.
If youâre wondering what the inout
means on the self
argument: this indicates that the argument is mutable and changes made inside the function are visible to the caller. For details, see below about inout arguments.
Int
vs int
In Mojo, you might notice that we use Int
(with a capital âIâ), which is different from Pythonâs int
(with a lowercase âiâ). This difference is on purpose, and itâs actually a good thing!
In Python, the int
type can handle really big numbers and has some extra features, like checking if two numbers are the same object. But this comes with some extra baggage that can slow things down. Mojoâs Int
is different. Itâs designed to be simple, fast, and tuned for your computerâs hardware to handle quickly.
We made this choice for two main reasons:
We want to give programmers who need to work closely with computer hardware (systems programmers) a transparent and reliable way to interact with hardware. We donât want to rely on fancy tricks (like JIT compilers) to make things faster.
We want Mojo to work well with Python without causing any issues. By using a different name (Int instead of int), we can keep both types in Mojo without changing how Pythonâs int works.
As a bonus, Int
follows the same naming style as other custom data types you might create in Mojo. Additionally, Int
is a struct
thatâs included in Mojoâs standard set of tools.
Strong type checking
Even though you can still use flexible types like in Python, Mojo lets you use strict type checking. Type-checking can make your code more predictable, manageable, and secure.
One of the primary ways to employ strong type checking is with Mojoâs struct
type. A struct
definition in Mojo defines a compile-time-bound name, and references to that name in a type context are treated as a strong specification for the value being defined. For example, consider the following code that uses the MyPair
struct shown above:
def pairTest() -> Bool:
let p = MyPair(1, 2)
return p < 4 # gives a compile-time error
When you run this code, youâll get a compile-time error telling you that â4â cannot be converted to MyPair
, which is what the RHS of MyPair.__lt__
requires.
This is a familiar experience when working with systems programming languages, but itâs not how Python works. Python has syntactically identical features for MyPy type annotations, but they are not enforced by the compiler: instead, they are hints that inform static analysis. By tying types to specific declarations, Mojo can handle both the classical type annotation hints and strong type specifications without breaking compatibility.
Type checking isnât the only use-case for strong types. Since we know the types are accurate, we can optimize the code based on those types, pass values in registers, and be as efficient as C for argument passing and other low-level details. This is the foundation of the safety and predictability guarantees Mojo provides to systems programmers.
Overloaded functions and methods
Like Python, you can define functions in Mojo without specifying argument data types and Mojo will handle them dynamically. This is nice when you want expressive APIs that just work by accepting arbitrary inputs and let dynamic dispatch decide how to handle the data. However, when you want to ensure type safety, as discussed above, Mojo also offers full support for overloaded functions and methods.
This allows you to define multiple functions with the same name but with different arguments. This is a common feature seen in many languages, such as C++, Java, and Swift.
When resolving a function call, Mojo tries each candidate and uses the one that works (if only one works), or it picks the closest match (if it can determine a close match), or it reports that the call is ambiguous if it canât figure out which one to pick. In the latter case, you can resolve the ambiguity by adding an explicit cast on the call site. Letâs look at an example:
struct Array[T: AnyType]:
fn __getitem__(self, idx: Int) -> T: ...
fn __getitem__(self, idx: Range) -> ArraySlice: ...
You can overload methods in structs and classes and overload module-level functions.
Mojo doesnât support overloading solely on result type, and doesnât use result type or contextual type information for type inference, keeping things simple, fast, and predictable. Mojo will never produce an âexpression too complexâ error, because its type-checker is simple and fast by definition.
Again, if you leave your argument names without type definitions, then the function behaves just like Python with dynamic types. As soon as you define a single argument type, Mojo will look for overload candidates and resolve function calls as described above.
fn
definitions
The extensions above are the cornerstone that provides low-level programming and provide abstraction capabilities, but many systems programmers prefer more control and predictability than what def
in Mojo provides. To recap, def
is defined by necessity to be very dynamic, flexible and generally compatible with Python: arguments are mutable, local variables are implicitly declared on first use, and scoping isnât enforced. This is great for high level programming and scripting, but is not always great for systems programming. To complement this, Mojo provides an fn
declaration which is like a âstrict modeâ for def
.
Alternative: instead of using a new keyword like
fn
, we could instead add a modifier or decorator like@strict def
. However, we need to take new keywords anyway and there is little cost to doing so. Also, in practice in systems programming domains,fn
is used all the time so it probably makes sense to make it first class.
As far as a caller is concerned, fn
and def
are interchangeable: there is nothing a def
can provide that a fn
cannot (and vice versa). The difference is that a fn
is more limited and controlled on the inside of its body (alternatively: pedantic and strict). Specifically, fn
s have a number of limitations compared to def
functions:
Argument values default to being immutable in the body of the function (like a
let
), instead of mutable (like avar
). This catches accidental mutations, and permits the use of non-copyable types as arguments.Argument values require a type specification (except for
self
in a method), catching accidental omission of type specifications. Similarly, a missing return type specifier is interpreted as returningNone
instead of an unknown return type. Note that both can be explicitly declared to returnobject
, which allows one to opt-in to the behavior of adef
if desired.Implicit declaration of local variables is disabled, so all locals must be declared. This catches name typos and dovetails with the scoping provided by
let
andvar
.Both support raising exceptions, but this must be explicitly declared on a
fn
with theraises
keyword.
Programming patterns will vary widely across teams, and this level of strictness will not be for everyone. We expect that folks who are used to C++ and already use MyPy-style type annotations in Python to prefer the use of fn
s, but higher level programmers and ML researchers to continue to use def
. Mojo allows you to freely intermix def
and fn
declarations, e.g. implementing some methods with one and others with the other, and allows each team or programmer to decide what is best for their use-case.
For more about argument behavior in Mojo functions, see the section below about Argument passing control and memory ownership.
The __copyinit__
and __moveinit__
special methods
Mojo supports full âvalue semanticsâ as seen in languages like C++ and Swift, and it makes defining simple aggregates of fields very easy with its @value
decorator (described in more detail below).
For advanced use cases, Mojo allows you to define custom constructors (using Pythonâs existing __init__
special method), custom destructors (using the existing __del__
special method) and custom copy and move constructors using the new __copyinit__
and __moveinit__
special methods.
These low-level customization hooks can be useful when doing low level systems programming, e.g. with manual memory management. For example, consider a dynamic string type that needs to allocate memory for the string data when constructed and destroy it when the value is destroyed:
struct MyString:
var data: Pointer[UI8]
# StringRef has a data + length field
def __init__(inout self, input: StringRef):
let data = Pointer[UI8].alloc(input.length+1)
input.data, input.length)
data.memcpy(input.length] = 0
data[self.data = Pointer[UI8](data)
def __del__(owned self):
self.data.free()
This MyString
type is implemented using low-level functions to show a simple example of how this works - a more realistic implementation would use short string optimizations, etc. However, if you go ahead and try this out, you might be surprised:
fn useStrings():
var a: MyString = "hello"
print(a) # Should print "hello"
var b = a # ERROR: MyString doesn't implement __copyinit__
= "Goodbye"
a print(b) # Should print "hello"
print(a) # Should print "Goodbye"
The Mojo compiler doesnât allow us to copy the string from a
to b
because it doesnât know how to. MyString
contains an instance of Pointer
(which is equivalent to a low-level C pointer) and Mojo doesnât know what kind of data it points to or how to copy it. More generally, some types (like atomic numbers) cannot be copied or moved around because their address provides an identity just like a class instance does.
In this case, we do want our string to be copyable. To enable this, we implement the __copyinit__
special method, which is conventionally implemented like this:
struct MyString:
...def __copyinit__(inout self, existing: Self):
self.data = Pointer(strdup(existing.data.address))
With this implementation, our code above works correctly, and the b = a
copy produces a logically distinct instance of the string with its own lifetime and data. The copy is made with the C-style strdup()
function as instructed by the lines of code above. Mojo also supports the __moveinit__
method, which allows both Rust-style moves (which take a value when a lifetime ends) and C++-style moves (where the contents of a value are removed, but the destructor still runs) and allows defining custom move logic. For more discussion about value lifetimes, see the Value lifecycle section below.
Mojo provides full control over the lifetime of a value, including the ability to make types copyable, move-only, and not-movable. This is more control than languages like Swift and Rust offer, which require values to at least be movable. If you are curious how existing
can be passed into the __copyinit__
method without itself creating a copy, check out the section on Borrowed arguments below.
Argument passing control and memory ownership
In both Python and Mojo, much of the language revolves around function calls: a lot of the (apparently) built-in behaviors are implemented in the standard library with âdunderâ (double-underscore) methods. Inside these magic functions is where a lot of memory ownership is determined through argument passing.
Letâs review some details about how Python and Mojo pass arguments:
All values passed into a Python
def
function use reference semantics. This means the function can modify mutable objects passed into it and those changes are visible outside the function. However, the behavior is sometimes surprising for the uninitiated, because you can change the object that an argument points to and that change is not visible outside the function.All values passed into a Mojo
def
function use value semantics by default. Compared to Python, this is an important difference: A Mojodef
function receives a copy of all argumentsâit can modify arguments inside the function, but the changes are not visible outside the function.All values passed into a Mojo
fn
function are immutable references by default. This means the function can read the original object (it is not a copy), but it cannot modify the object at all.
This convention for immutable argument passing in a Mojo fn
is called âborrowing.â In the following sections, weâll explain how you can change the argument passing behavior in Mojo, for both def
and fn
functions.
Why argument conventions are important
In Python, all fundamental values are references to objectsâas described above, a Python function can modify the original object. Thus, Python developers are used to thinking about everything as reference semantic. However, at the CPython or machine level, you can see that the references themselves are actually passed by-copyâPython copies a pointer and adjusts reference counts.
This Python approach provides a comfortable programming model for most people, but it requires all values to be heap-allocated (and results are occasionally surprising results due to reference sharing). Mojo classes (TODO: will) follow the same reference-semantic approach for most objects, but this isnât practical for simple types like integers in a systems programming context. In these scenarios, we want the values to live on the stack or even in hardware registers. As such, Mojo structs are always inlined into their container, whether that be as the field of another type or into the stack frame of the containing function.
This raises some interesting questions: How do you implement methods that need to mutate self
of a structure type, such as __iadd__
? How does let
work, and how does it prevent mutation? How are the lifetimes of these values controlled to keep Mojo a memory-safe language?
The answer is that the Mojo compiler uses dataflow analysis and type annotations to provide full control over value copies, aliasing of references, and mutation control. These features are similar in many ways to features in the Rust language, but they work somewhat differently in order to make Mojo easier to learn, and they integrate better into the Python ecosystem without requiring a massive annotation burden.
In the following sections, youâll learn about how you can control memory ownership for objects passed into Mojo fn
functions.
Immutable arguments (borrowed
)
A borrowed object is an immutable reference to an object that a function receives, instead of receiving a copy of the object. So the callee function has full read-and-execute access to the object, but it cannot modify it (the caller still has exclusive âownershipâ of the object).
Although this is the default behavior for fn
arguments, you can explicitly define it with the borrowed
keyword if youâd like (you can also apply borrowed
to def
arguments):
fn use_something_big(borrowed a: SomethingBig, b: SomethingBig):
"""'a' and 'b' are passed the same, because 'borrowed' is the default."""
a.print_id() b.print_id()
This default applies to all arguments uniformly, including the self
argument of methods. This is much more efficient when passing large values or when passing expensive values like a reference-counted pointer (which is the default for Python/Mojo classes), because the copy constructor and destructor donât have to be invoked when passing the argument. Here is a more elaborate example building on the code above:
# A type that is so expensive to copy around we don't even have a
# __copyinit__ method.
struct SomethingBig:
var id_number: Int
var huge: InlinedArray[Int, 100000]
fn __init__(inout self): âŠ
# self is passed inout for mutation as described above.
fn set_id(inout self, number: Int):
self.id_number = number
# Arguments like self are passed as borrowed by default.
fn print_id(self): # Same as: fn print_id(borrowed self):
print(self.id_number)
fn try_something_big():
# Big thing sits on the stack: after we construct it it cannot be
# moved or copied.
let big = SomethingBig()
# We still want to do useful things with it though!
big.print_id()# Do other things with it.
use_something_big(big, big)
Because the default argument convention for fn
functions is borrowed
, Mojo has simple and logical code that does the right thing by default. For example, we donât want to copy or move all of SomethingBig
just to invoke the print_id()
method, or when calling use_something_big()
.
This borrowed argument convention is similar in some ways to passing an argument by const&
in C++, which avoids a copy of the value and disables mutability in the callee. However, the borrowed convention differs from const&
in C++ in two important ways:
The Mojo compiler implements a borrow checker (similar to Rust) that prevents code from dynamically forming mutable references to a value when there are immutable references outstanding, and it prevents multiple mutable references to the same value. You are allowed to have multiple borrows (as the call to
use_something_big
does above) but you cannot pass something by mutable reference and borrow at the same time. (TODO: Not currently enabled).Small values like
Int
,Float
, andSIMD
are passed directly in machine registers instead of through an extra indirection (this is because they are declared with the@register_passable
decorator). This is a significant performance enhancement when compared to languages like C++ and Rust, and moves this optimization from every call site to being declarative on a type.
Similar to Rust, Mojoâs borrow checker enforces the exclusivity of invariants. The major difference between Rust and Mojo is that Mojo does not require a sigil on the caller side to pass by borrow. Also, Mojo is more efficient when passing small values, and Rust defaults to moving values instead of passing them around by borrow. These policy and syntax decisions allow Mojo to provide an easier-to-use programming model.
Mutable arguments (inout
)
On the other hand, if you define an fn
function and want an argument to be mutable (so that changes to the argument inside the function are visible outside the function), you must declare the argument as mutable with the inout
keyword.
Consider the following example, in which the __iadd__
function tries to modify self
:
struct Int:
# self and rhs are both immutable in __add__.
fn __add__(self, rhs: Int) -> Int: ...
# ... but this cannot work for __iadd__
fn __iadd__(self, rhs: Int):
self = self + rhs # ERROR: cannot assign to self!
The problem here is that self
is immutable because this is a Mojo fn
function, so it canât change the internal state of the argument. The solution is to declare that the argument is mutable by adding the inout
keyword on the self
argument name:
struct Int:
# ...
fn __iadd__(inout self, rhs: Int):
self = self + rhs # OK
Tip: When you see inout
, it means that any changes made to the argument inside the function are visible outside the function.
Now the self
argument is mutable in the function and any changes are visible in the callerâeven if the caller has a non-trivial computation to access it, like an array subscript:
fn show_mutation():
var x = 42
+= 1
x print(x) # prints 43 of course
var a = InlinedFixedVector[16, Int](...)
4] = 7
a[4] += 1 # Mutate an element within the InlinedFixedVector
a[print(a[4]) # Prints 8
let y = x
+= 1 # ERROR: Cannot mutate 'let' value y
Mojo implements the in-place mutation of the above InlinedFixedVector
element by emitting a call to __getitem__
into a temporary buffer, followed by a store with __setitem__
after the call. Mutation of the let
value fails because it isnât possible to form a mutable reference to an immutable value. Similarly, the compiler rejects attempts to use a subscript with an inout
argument if it implements __getitem__
but not __setitem__
.
Of course, you can declare multiple inout
arguments. For example, you can define and use a swap function like this:
fn swap(inout lhs: Int, inout rhs: Int):
let tmp = lhs
= rhs
lhs = tmp
rhs
fn show_swap():
var x = 42
var y = 12
swap(x, y)print(x) # Prints 12
print(y) # Prints 42
A very important aspect of this system is that it all composes correctly.
Notice that we donât call this argument passing âby reference.â Although the inout
convention is conceptually the same, we donât call it by-reference passing because the implementation may actually pass values using pointers.
Transfer arguments (owned
and ^
)
The final argument convention that Mojo supports is the owned
argument convention. This convention is used for functions that want to take exclusive ownership over a value, and it is often used with the postfixed ^
operator.
For example, imagine youâre working with a move-only type like a unique pointer. While the borrow convention makes it easy to work with the unique pointer without ceremony, at some point you might want to transfer ownership to some other function. This is what the ^
âtransferâ operator does:
fn usePointer():
let ptr = SomeUniquePtr(...)
# Perfectly fine to pass to borrowing function.
use(ptr) ^) # Pass ownership of the `ptr` value to another function.
take_ptr(ptr
# ERROR: ptr is no longer valid here! use(ptr)
For movable types, the ^
operator ends the lifetime of a value binding and transfers the value ownership to something else (in this case, the take_ptr()
function). To support this, you can define functions as taking owned
arguments. For example, you define take_ptr()
like so:
fn take_ptr(owned p: SomeUniquePtr):
use(p)
Because it is declared owned
, the take_ptr()
function knows it has unique access to the value. This is very important for things like unique pointers, and itâs useful when you want to avoid copies.
For example, you will notably see the owned
convention on destructors and on consuming move constructor. For example, our MyString
type from earlier can be defined as follows:
struct MyString:
var data: Pointer[UI8]
# StringRef has a data + length field
def __init__(inout self, input: StringRef): ...
def __copyinit__(inout self, existing: Self): ...
def __moveinit__(inout self, owned existing: Self):
self.data = existing.data
def __del__(owned self):
self.data.free()
Specifying owned
in the __del__
function is important because you must own a value to destroy it.
Comparing def
and fn
argument passing
Mojoâs def
function is essentially just sugaring for the fn
function:
A
def
argument without an explicit type annotation defaults toObject
.A
def
argument without a convention keyword (such asinout
orowned
) is passed by implicit copy into a mutable var with the same name as the argument. (This requires that the type have a__copyinit__
method.)
For example, these two functions have the same behavior:
def example(inout a: Int, b: Int, c):
# b and c use value semantics so they're mutable in the function
...
fn example(inout a: Int, b_in: Int, c_in: Object):
# b_in and c_in are immutable references, so we make mutable shadow copies
var b = b_in
var c = c_in
...
The shadow copies typically add no overhead, because references for small types like Object
are cheap to copy. The expensive part is adjusting the reference count, but thatâs eliminated by a move optimization.
Python integration
Itâs easy to use Python modules you know and love in Mojo. You can import any Python module into your Mojo program and create Python types from Mojo types.
Importing Python modules
To import a Python module in Mojo, just call Python.import_module()
with the module name:
from PythonInterface import Python
# This is equivalent to Python's `import numpy as np`
let np = Python.import_module("numpy")
# Now use numpy as if writing in Python
= np.array([1, 2, 3])
a print(a)
Yes, this imports Python NumPy, and you can import any other Python module.
Currently, you cannot import individual members (such as a single Python class or function)âyou must import the whole Python module and then access members through the module name.
Importing local Python modules
If you have some local Python code you want to use in Mojo, just add the directory to the Python path and then import the module.
For example, suppose you have a Python file like this:
mypython.py
import numpy as np
def my_algorithm(a, b):
= np.random.rand(a, a)
array_a return array_a + b
Hereâs how you can import it and use it in Mojo:
mojo-code.mojo
from PythonInterface import Python
"path/to/module")
Python.add_to_path(let mypython = Python.import_module("mypython")
let c = mypython.my_algorithm(2, 3)
print(c)
Thereâs no need to worry about memory management when using Python in Mojo. Everything just works because Mojo was designed for Python from the beginning.
Mojo types in Python
Mojo primitive types implicitly convert into Python objects. Today we support lists, tuples, integers, floats, booleans, and strings.
For example, given this Python function that prints Python types:
mypython2.py
def type_printer(my_list, my_tuple, my_int, my_string, my_float):
print(type(my_list))
print(type(my_tuple))
print(type(my_int))
print(type(my_string))
print(type(my_float))
You can import it and pass it Mojo types, no problem:
mojo-code.mojo
from PythonInterface import Python
"/path/to/module")
Python.add_to_path(let mypython2 = Python.import_module("mypython2")
0, 3], (False, True), 4, "orange", 3.4) mypython2.type_printer([
It will output the types after implicit conversion to Python types:
<class 'list'>
<class 'tuple'>
<class 'int'>
<class 'str'>
<class 'float'>
Mojo doesnât have a standard Dictionary yet, so it is not yet possible to create a Python dictionary from a Mojo dictionary. You can work with Python dictionaries in Mojo though! To create a Python dictionary, use the dict
method:
from PythonInterface import Python
from PythonObject import PythonObject
from IO import print
from Range import range
def main() -> None:
let dictionary = Python.dict()
"fruit"] = "apple"
dictionary["starch"] = "potato"
dictionary[let keys: PythonObject = ["fruit", "starch", "protein"]
let N: Int = keys.__len__().to_index()
print(N, "items")
for i in range(N):
if Python.is_type(dictionary.get(keys[i]), Python.none()):
print(keys[i], "is not in dictionary")
else:
print(keys[i], "is included")
The output:
3 items
fruit is included
starch is included
protein is not in dictionary
Parameterization: compile-time metaprogramming
One of Pythonâs most amazing features is its extensible runtime metaprogramming features. This has enabled a wide range of libraries and provides a flexible and extensible programming model that Python programmers everywhere benefit from. Unfortunately, these features also come at a cost: because they are evaluated at runtime, they directly impact run-time efficiency of the underlying code. Because they are not known to the IDE, it is difficult for IDE features like code completion to understand them and use them to improve the developer experience.
Outside the Python ecosystem, static metaprogramming is also an important part of development, enabling the development of new programming paradigms and advanced libraries. There are many examples of prior art in this space, with different tradeoffs, for example:
Preprocessors (e.g. C preprocessor, Lex/YACC, etc) are perhaps the heaviest handed. They are fully general but the worst in terms of developer experience and tools integration.
Some languages (like Lisp and Rust) support (sometimes âhygienicâ) macro expansion features, enabling syntactic extension and boilerplate reduction with somewhat better tooling integration.
Some older languages like C++ have very large and complex metaprogramming languages (templates) that are a dual to the runtime language. These are notably difficult to learn and have poor compile times and error messages.
Some languages (like Swift) build many features into the core language in a first-class way to provide good ergonomics for common cases at the expense of generality.
Some newer languages like Zig integrate a language interpreter into the compilation flow, and allow the interpreter to reflect over the AST as it is compiled. This allows many of the same features as a macro system with better extensibility and generality.
For Modularâs work in AI, high-performance machine learning kernels, and accelerators, we need high abstraction capabilities provided by advanced metaprogramming systems. We needed high-level zero-cost abstractions, expressive libraries, and large-scale integration of multiple variants of algorithms. We want library developers to be able to extend the system, just like they do in Python, providing an extensible developer platform.
That said, we are not willing to sacrifice developer experience (including compile times and error messages) nor are we interested in building a parallel language ecosystem that is difficult to teach. We can learn from these previous systems but also have new technologies to build on top of, including MLIR and fine-grained language-integrated caching technologies.
As such, Mojo supports compile-time metaprogramming built into the compiler as a separate stage of compilationâafter parsing, semantic analysis, and IR generation, but before lowering to target-specific code. It uses the same host language for runtime programs as it does for metaprograms, and leverages MLIR to represent and evaluate these programs predictably.
Letâs take a look at some simple examples.
About âparametersâ: Python developers use the words âargumentsâ and âparametersâ fairly interchangeably for âthings that are passed into functions.â We decided to reclaim âparameterâ and âparameter expressionâ to represent a compile-time value in Mojo, and continue to use âargumentâ and âexpressionâ to refer to runtime values. This allows us to align around words like âparameterizedâ and âparametricâ for compile-time metaprogramming.
Defining parameterized types and functions
Mojo structs and functions may each be parameterized, but an example can help motivate why we care. Letâs look at a SIMD type, which represents a low-level vector register in hardware that holds multiple instances of a scalar data-type. Hardware accelerators these days are getting exotic datatypes, and it isnât uncommon to work with CPUs that have 512-bit or longer SIMD vectors. There is a lot of diversity in hardware (including many brands like SSE, AVX-512, NEON, SVE, RVV, etc.) but many operations are common and used by numerics and ML kernel developersâthe SIMD
type exposes them to Mojo programmers.
Here is a (cut down) version of the SIMD
API in the Mojo standard library:
struct SIMD[type: DType, size: Int]:
var value: ⊠# Some low-level MLIR stuff here
# Create a new SIMD from a number of scalars
fn __init__(inout self, *elems: SIMD[type, 1]): ...
# Fill a SIMD with a duplicated scalar value.
@staticmethod
fn splat(x: SIMD[type, 1]) -> SIMD[type, size]: ...
# Cast the elements of the SIMD to a different elt type.
fn cast[target: DType](self) -> SIMD[target, size]: ...
# Many standard operators are supported.
fn __add__(self, rhs: Self) -> Self: ...
Parameters in Mojo are declared in square brackets using an extended version of the PEP695 syntax. They are named and have types like normal values in a Mojo program, but they are evaluated at compile-time instead of runtime by the target program. The runtime program may use the value of parametersâbecause the parameters are resolved at compile-time before they are needed by the runtime programâbut the compile-time parameter expressions may not use runtime values.
In the case of the SIMD
excerpt above, there are three declared parameters: the SIMD
struct is parameterized by a type
parameter and a size
parameter. The cast
method is further parameterized with a target
parameter. Because SIMD
is a parameterized type, the type of a self
argument carries the parametersâthe full type name is SIMD[type, size]
. While it is always valid to write this out (as shown in the return type of splat()
), this can be verbose, so we recommend using the Self
type (from PEP673) like the __add__
example does.
Using parameterized types and functions
For the SIMD
type, size
specifies the number of elements in a SIMD vector, and type
specifies the element typeâfor example, you might use a â4xFloatâ to represent a small floating-point vector or a â32xbfloat16â on an AVX-512 system with the âbfloat16â machine learning type:
fn funWithSIMD():
# Make a vector of 4 floats.
let small_vec = SIMD[DType.f32, 4](1.0, 2.0, 3.0, 4.0)
# Make a big vector containing 1.0 in bfloat16 format.
let big_vec = SIMD[DType.bf16, 32].splat(1.0)
# Do some math and convert the elements to float32.
let bigger_vec = (big_vec+big_vec).cast[DType.f32]()
# You can write types out explicitly if you want of course.
let bigger_vec2 : SIMD[DType.f32, 32] = bigger_vec
Note that the cast()
method needs an additional parameter to indicate what type to cast to: that is handled by parameterizing the call to cast()
. The example above shows the use of concrete types, but the major power of parameters comes from the ability to define parametric algorithms and types. For example, itâs quite easy to define parametric algorithms, such as those that are length- and DType-agnostic:
fn rsqrt[width: Int, dt: DType](x: SIMD[dt, width]) -> SIMD[dt, width]:
return 1 / sqrt(x)
The Mojo compiler is fairly smart about type inference with parameters. Note that this function is able to call the parametric sqrt()
function without specifying the parameters, the compiler infers its parameters as if you wrote sqrt[width,type](x)
explicitly. Also note that rsqrt()
chose to define its first parameter named width
but the SIMD type names it size
without challenge.
Parameter expressions are just Mojo code
All parameters and parameter expressions are typed using the same type system as the runtime program: Int
and DType
are implemented in the Mojo standard library as structs. Parameters are quite powerful, supporting the use of expressions with operators, function calls at compile-time, and more, just like a runtime program. This enables the use of many âdependent typeâ features. For example, you might want to define a helper function to concatenate two SIMD vectors:
fn concat[ty: DType, len1: Int, len2: Int](
-> SIMD[ty, len1+len2]:
lhs: SIMD[ty, len1], rhs: SIMD[ty, len2])
...
fn use_vectors(a: SIMD[DType.f32, 4], b: SIMD[DType.f16, 8]):
let x = concat(a, a) # Length = 8
let y = concat(b, b) # Length = 16
Note how the resulting length is the sum of the input vector lengths, and you can express that with a simple +
operation. For a more complex example, take a look at the SIMD.shuffle()
method in the standard library: it takes two input SIMD values, a vector shuffle mask as a list, and returns a SIMD that matches the length of the shuffle mask.
Powerful compile-time programming
While simple expressions are useful, sometimes you want to write imperative compile-time logic with control flow. For example, the isclose()
function in the Mojo Math
module uses exact equality for integers but âcloseâ comparison for floating-point. You can even do compile-time recursion. For instance, here is an example âtree reductionâ algorithm that sums all elements of a vector recursively into a scalar:
struct SIMD[type: DType, size: Int]:
...fn reduce_add(self) -> SIMD[type, 1]:
@parameter
if size == 1:
return self[0]
elif size == 2:
return self[0] + self[1]
# Extract the top/bottom halves, add them, sum the elements.
let lhs = self.slice[size // 2](0)
let rhs = self.slice[size // 2](size // 2)
return (lhs + rhs).reduce_add()
This makes use of the @parameter if
feature, which is an if
statement that runs at compile-time. It requires that its condition be a valid parameter expression, and ensures that only the live branch of the if
statement is compiled into the program.
Mojo types are just parameter expressions
While weâve shown how you can use parameter expressions within types, in both Python and Mojo, type annotations can themselves be arbitrary expressions. Types in Mojo have a special metatype type, allowing type-parametric algorithms and functions to be defined. For example, you can define an algorithm like the C++ std::vector
class like this:
struct DynamicVector[type: AnyType]:
...fn reserve(inout self, new_capacity: Int): ...
fn push_back(inout self, value: type): ...
fn pop_back(inout self): ...
fn __getitem__(self, i: Int) -> type: ...
fn __setitem__(inout self, i: Int, value: type): ...
fn use_vector():
var v = DynamicVector[Int]()
17)
v.push_back(42)
v.push_back(0] = 123
v[print(v[1]) # Prints 42
print(v[0]) # Prints 123
Notice that the type
parameter is used as the formal type for the value
arguments and the return type of the __getitem__
function. Parameters allow the DynamicVector
type to provide different APIs based on the different use-cases. There are many other cases that benefit from more advanced use cases. For example, the parallel processing library defines the parallelForEachN
algorithm, which executes a closure N times in parallel, feeding in a value from the context. That value can be of any type:
fn parallelize[
arg_type: AnyType,fn(Int, arg_type) -> None,
func:
](rt: Runtime, num_work_items: Int, arg: arg_type):# Not actually parallel: see Functional.mojo for real impl.
for i in range(num_work_items):
func(i, arg)
This is possible because the func
parameter is allowed to refer to the earlier arg_type
parameter, and that refines its type in turn.
Another example where this is important is with variadic generics, where an algorithm or data structure may need to be defined over a list of heterogeneous types:
struct Tuple[*ElementTys: AnyType]:
var _storage : *ElementTys
Note: we donât have enough metatype helpers in place yet, but we should be able to write something like this in the future, though overloading is still a better way to handle this:
struct Array[T: AnyType]:
fn __getitem__[IndexType: AnyType](self, idx: IndexType)
-> (ArraySlice[T] if issubclass(IndexType, Range) else T):
...
alias
: named parameter expressions
It is very common to want to name compile-time values. Whereas var
defines a runtime value, and let
defines a runtime constant, we need a way to define a compile-time temporary value. For this, Mojo uses an alias
declaration. For example, the DType
struct implements a simple enum using aliases for the enumerators like this (the actual internal implementation details vary a bit):
struct DType:
var value : UI8
alias invalid = DType(0)
alias bool = DType(1)
alias si8 = DType(2)
alias ui8 = DType(3)
alias si16 = DType(4)
alias ui16 = DType(5)
...alias f32 = DType(15)
This allows clients to use DType.f32
as a parameter expression (which also works as a runtime value) naturally. Note that this is invoking the runtime constructor for DType at compile-time.
Types are another common use for alias: because types are compile-time expressions, it is handy to be able to do things like this:
alias F32 = SIMD[DType.f32, 1]
alias UI8 = SIMD[DType.ui8, 1]
var x : F32 # F32 works like a "typedef"
Like var
and let
, aliases obey scope, and you can use local aliases within functions as youâd expect.
Autotuning / Adaptive compilation
Mojo parameter expressions allow you to write portable parametric algorithms like you can do in other languages, but when writing high-performance code you still have to pick concrete values to use for the parameters. For example, when writing high-performance numeric algorithms, you might want to use memory tiling to accelerate the algorithm, but the dimensions to use depend highly on the available hardware features, the sizes of the cache, what gets fused into the kernel, and many other fiddly details.
Even vector length can be difficult to manage, because the vector length of a typical machine depends on the datatype, and some datatypes like bfloat16
donât have full support on all implementations. Mojo helps by providing an autotune
function in the standard library. For example if you want to write a vector-length-agnostic algorithm to a buffer of data, you might write it like this:
from Autotune import autotune
def exp_buffer_impl[dt: DType](data: ArraySlice[dt]):
# Pick vector length for this dtype and hardware
alias vector_len = autotune(1, 4, 8, 16, 32)
# Use it as the vectorization length
vectorize[exp[dt, vector_len]](data)
When compiling instantiations of this code, Mojo forks compilation of this algorithm and decides which value to use by measuring what works best in practice for the target hardware. It evaluates the different values of the vector_len
expression and picks the fastest one according to a user-defined performance evaluator. Because it measures and evaluates each option individually, it might pick a different vector length for F32 than for SI8, for example. This simple feature is pretty powerful - going beyond simple integer constants - because functions and types are also parameter expressions.
Users can instrument the search of exp_buffer_impl
by providing a performance evaluator and using the search
standard library function. search
takes an evaluator and a forked function and returns the fastest implementation selected by the evaluator as a parameter result.
from Autotune import search
fn exp_buffer[dt: DType](data: ArraySlice[dt]):
# Forward declare the result parameter.
alias best_impl: fn(ArraySlice[dt]) -> None
# Perform search!
search[fn(ArraySlice[dt]) -> None,
exp_buffer_impl[dt],-> best_impl
exp_evaluator[dt]
]()
# Call the selected implementation
best_impl(data)
In this example, we provided exp_evaluator
to the search function as the performance evaluator. Performance evaluators are invoked with a list of candidate functions and should return the index of the best one. Mojoâs standard library provides a Benchmark
module that you can use to time functions.
from Benchmark import Benchmark
fn exp_evaluator[dt: DType](
fn(ArraySlice[dt]) -> None],
fns: Pointer[
num: Int
):var best_idx = -1
var best_time = -1
for i in range(num):
= fns[i]
candidate let buf = Buffer[dt]()
# Benchmark this candidate.
fn setup():
buf.fill_random()fn wrapper():
candidate(buf)let cur_time = Benchmark(2).run[wrapper, setup]()
# Track the index of the fastest candidate.
if best_idx < 0:
= i
best_idx = cur_time
best_time elif best_time > cur_time:
= f_idx
best_idx = cur_time
best_time
# Return the fastest implementation.
return best_idx
Autotuning has an exponential runtime. It benefits from internal implementation details of the Mojo compiler stack (particularly MLIR, integrated caching, and distribution of compilation). This is a power-user feature and needs continued development and iteration over time.
âValue Lifecycleâ: Birth, life and death of a value
Now that we have an understanding of the different ingredients that can go into building functions and the types system, we can look at how to put them together to model important types that you may want to express in Mojo.
Many existing languages express design points with different tradeoffs: C++, for example, is very powerful but often accused of âgetting the defaults wrongâ which leads to bugs and mis-features. Swift is easy to work with, but has a less predictable model that copies values a lot and is dependent on an âARC optimizerâ for performance. Rust started with strong value ownership goals to satisfy its borrow checker, but relies on values being movable, which makes it challenging to express custom move constructors and can put a lot of stress on memcpy
performance. In Python, everything is a reference to a class, so it never really faces issues with types.
For Mojo, we benefit from learning from these existing systems, and aim to provide a model that is very powerful while still easy to learn and understand. We also donât want to require âbest effortâ and difficult-to-predict optimization passes built into a âsufficiently smartâ compiler.
To explore these issues, we look at different value classifications and the relevant Mojo features that go into expressing them, and build from the bottom-up. We use C++ as the primary comparison point in examples because it is widely known, but we occasionally reference other languages if they provide a better comparison point.
Types that cannot be instantiated
The most bare-bones type in Mojo is one that doesnât allow you to create instances of it: these types have no initializer at all, and if they have a destructor, it will never be invoked (because there cannot be instances to destroy):
struct NoInstances:
var state: Int # Pretty useless
alias my_int = Int
@staticmethod
fn print_hello():
print("hello world")
Mojo types do not get default constructors, move constructors, memberwise initializers or anything else by default, so it is impossible to create an instance of this NoInstances
type. In order to get them, you need to define an __init__
method or use a decorator that synthesizes an initializer. As shown, these types can be useful as ânamespacesâ because you can refer to static members like NoInstances.my_int
or NoInstances.print_hello()
even though you cannot instantiate an instance of the type.
Non-movable and non-copyable types
If we take a step up the ladder of sophistication, weâll get to types that can be instantiated, but once they are pinned to an address in memory, they cannot be implicitly moved or copied. This can be useful to implement types like atomic operations (such as std::atomic
in C++) or other types where the memory address of the value is its identity and is critical to its purpose:
struct Atomic:
var state: Int
fn __init__(inout self, state: Int = 0):
self.state = state
fn __iadd__(inout self, rhs: Int):
#...atomic magic...
fn get_value(self) -> Int:
return atomic_load_int(self.state)
This class defines an initializer but no copy or move constructors, so once it is initialized it can never be moved or copied. This is safe and useful because Mojoâs ownership system is fully âaddress correctâ - when this is initialized onto the stack or in the field of some other type, it never needs to move.
Note that Mojoâs approach controls only the built-in move operations, such as a = b
copies and the ^
transfer operator. One useful pattern you can use for your own types (like Atomic
above) is to add an explicit copy()
method (a non-âdunderâ method). This can be useful to make explicit copies of an instance when it is known safe to the programmer.
Unique âmove-onlyâ types
If we take one more step up the ladder of capabilities, we will encounter types that are âuniqueâ - there are many examples of this in C++, such as types like std::unique_ptr
or even a FileDescriptor
type that owns an underlying POSIX file descriptor. These types are pervasive in languages like Rust, where copying is discouraged, but âmoveâ is free. In Mojo, you can implement these kinds of moves by defining the __moveinit__
method to take ownership of a unique type. For example:
# This is a simple wrapper around POSIX-style fcntl.h functions.
struct FileDescriptor:
var fd: Int
# This is how we move our unique type.
fn __moveinit__(inout self, owned existing: Self):
self.fd = existing.fd
# This takes ownership of a POSIX file descriptor.
fn __init__(inout self, fd: Int):
self.fd = fd
fn __init__(inout self, path: String):
# Error handling omitted, call the open(2) syscall.
self = FileDescriptor(open(path, ...))
fn __del__(owned self):
self.fd) # pseudo code, call close(2)
close(
fn dup(self) -> Self:
# Invoke the dup(2) system call.
return Self(dup(self.fd))
fn read(...): ...
fn write(...): ...
The consuming move constructor (__moveinit__
) takes ownership of an existing FileDescriptor
, and moves its internal implementation details over to a new instance. This is because instances of FileDescriptor
may exist at different locations, and they can be logically moved aroundâstealing the body of one value and moving it into another.
Here is an egregious example that will invoke __moveinit__
multiple times:
fn egregious_moves(owned fd1: FileDescriptor):
# fd1 and fd2 have different addresses in memory, but the
# transfer operator moves unique ownership from fd1 to fd2.
let fd2 = fd1^
# Do it again, a use of fd2 after this point will produce an error.
let fd3 = fd2^
# We can do this all day...
let fd4 = fd3^
fd4.read(...)# fd4.__del__() runs here
Note how ownership of the value is transferred between various values that own it, using the postfix-^
âtransferâ operator, which destroys a previous binding and transfer ownership to a new constant. If you are familiar with C++, the simple way to think about the transfer operator is like std::move
, but in this case, we can see that it is able to move things without resetting them to a state that can be destroyed: in C++, if your move operator failed to change the old valueâs fd
instance, it would get closed twice.
Mojo tracks the liveness of values and allows you to define custom move constructors. This is rarely needed, but extremely powerful when it is. For example, some types like the llvm::SmallVector type
use the âinline storageâ optimization technique, and they may want to be implemented with an âinner pointerâ into their instance. This is a well-known trick to reduce pressure on the malloc memory allocator, but it means that a âmoveâ operation needs custom logic to update the pointer when that happens.
With Mojo, this is as simple as implementing a custom __moveinit__
method. This is something that is also easy to implement in C++ (though, with boilerplate in the cases where you donât need custom logic) but is difficult to implement in other popular memory-safe languages.
One additional note is that while the Mojo compiler provides good predictability and control, it is also very sophisticated. It reserves the right to eliminate temporaries and the corresponding copy/move operations. If this is inappropriate for your type, you should use explicit methods like copy()
instead of the dunder methods.
Types that support a âstealing moveâ
One challenge with memory-safe languages is that they need to provide a predictable programming model around what the compiler is able to track, and static analysis in a compiler is inherently limited. For example, while it is possible for a compiler to understand that the two array accesses in the first example below are to different array elements, it is (in general) impossible to reason about the second example:
std::pair<T, T> getValues1(MutableArray<T> &array) {
return { std::move(array[0]), std::move(array[1]) };
}
std::pair<T, T> getValues2(MutableArray<T> &array, size_t i, size_t j) {
return { std::move(array[i]), std::move(array[j]) };
}
The problem here is that there is simply no way (looking at just the function body above) to know or prove that the dynamic values of i
and j
are not the same. While it is possible to maintain dynamic state to track whether individual elements of the array are live, this often causes significant runtime expense (even when move/transfers are not used), which is something that Mojo and other systems programming languages are not keen to do. There are a variety of ways to deal with this, including some pretty complicated solutions that arenât always easy to learn.
Mojo takes a pragmatic approach to let Mojo programmers get their job done without having to work around its type system. As seen above, it doesnât force types to be copyable, movable, or even constructable, but it does want types to express their full contract, and it wants to enable fluent design patterns that programmers expect from languages like C++. The (well known) observation here is that many objects have contents that can be âstolenâ without needing to disable their destructor, either because they have a ânull stateâ (like an optional type or nullable pointer) or because they have a null value that is efficient to create and a no-op to destroy (e.g. std::vector
can have a null pointer for its data).
To support these use-cases, the ^
transfer operator supports arbitrary LValues, and when applied to one, it invokes the âstealing move constructor.â This constructor must set up the new value to be in a live state, and it can mutate the old value, but it must put the old value into a state where its destructor still works. For example, if we want to put our FileDescriptor
into a vector and move out of it, we might choose to extend it to know that -1
is a sentinel which means that it is ânullâ. We can implement this like so:
# This is a simple wrapper around POSIX-style fcntl.h functions.
struct FileDescriptor:
var fd: Int
# This is the new key capability.
fn __moveinit__(inout self, inout existing: Self):
self.fd = existing.fd
= -1 # neutralize 'existing'.
existing.fd
fn __moveinit__(inout self, owned existing: Self): # as above
fn __init__(inout self, fd: Int): # as above
fn __init__(inout self, path: String): # as above
fn __del__(owned self):
if self.fd != -1:
self.fd) # pseudo code, call close(2) close(
Notice how the âstealing moveâ constructor takes the file descriptor from an existing value and mutates that value so that its destructor wonât do anything. This technique has tradeoffs and is not the best for every type. We can see that it adds one (inexpensive) branch to the destructor because it has to check for the sentinel case. It is also generally considered bad form to make types like this nullable because a more general feature like an Optional[T]
type is a better way to handle this.
Furthermore, we plan to implement Optional[T]
in Mojo itself, and Optional
needs this functionality. We also believe that the library authors understand their domain problem better than language designers do, and generally prefer to give library authors full power over that domain. As such you can choose (but donât have to) to make your types participate in this behavior in an opt-in way.
Copyable types
The next step up from movable types are copyable types. Copyable types are also very common - programmers generally expect things like strings and arrays to be copyable, and every Python Object reference is copyable - by copying the pointer and adjusting the reference count.
There are many ways to implement copyable types. One can implement reference semantic types like Python or Java, where you propagate shared pointers around, one can use immutable data structures that are easily shareable because they are never mutated once created, and one can implement deep value semantics through lazy copy-on-write as Swift does. Each of these approaches has different tradeoffs, and Mojo takes the opinion that while we want a few common sets of collection types, we can also support a wide range of specialized ones that focus on particular use cases.
In Mojo, you can do this by implementing the __copyinit__
method. Here is an example of that using a simple String
in pseudo-code:
struct MyString:
var data: Pointer[UI8]
# StringRef is a pointer + length and works with StringLiteral.
def __init__(inout self, input: StringRef):
self.data = ...
# Copy the string by deep copying the underlying malloc'd data.
def __copyinit__(inout self, existing: Self):
self.data = strdup(existing.data)
# This isn't required, but optimizes unneeded copies.
def __moveinit__(inout self, owned existing: Self):
self.data = existing.data
def __del__(owned self):
self.data.address)
free(
def __add__(self, rhs: MyString) -> MyString: ...
This simple type is a pointer to a ânull-terminatedâ string data allocated with malloc, using old-school C APIs for clarity. It implements the __copyinit__
, which maintains the invariant that each instance of MyString
owns its underlying pointer and frees it upon destruction. This implementation builds on tricks weâve seen above, and implements a __moveinit__
constructor, which allows it to completely eliminate temporary copies in some common cases. You can see this behavior in this code sequence:
fn test_my_string():
var s1 = MyString("hello ")
var s2 = s1 # s2.__copyinit__(s1) runs here
print(s1)
var s3 = s1^ # s3.__moveinit__(s1) runs here
print(s2)
# s2.__del__() runs here
print(s3)
# s3.__del__() runs here
In this case, you can see both why a copy constructor is needed: without one, the duplication of the s1
value into s2
would be an error - because you cannot have two live instances of the same non-copyable type. The move constructor is optional but helps the assignment into s3
: without it, the compiler would invoke the copy constructor from s1, then destroy the old s1
instance. This is logically correct but introduces extra runtime overhead.
Mojo destroys values eagerly, which allows it to transform copy+destroy pairs into single move operations, which can lead to much better performance than C++ without requiring the need for pervasive micromanagement of std::move
.
Trivial types
The most flexible types are ones that are just âbags of bitsâ. These types are âtrivialâ because they can be copied, moved, and destroyed without invoking custom code. Types like these are arguably the most common basic type that surrounds us: things like integers and floating point values are all trivial. From a language perspective, Mojo doesnât need special support for these, it would be perfectly fine for type authors to implement these things as no-ops, and allow the inliner to just make them go away.
There are two reasons that approach would be suboptimal: one is that we donât want the boilerplate of having to define a bunch of methods on trivial types, and second, we donât want the compile-time overhead of generating and pushing around a bunch of function calls, only to have them inline away to nothing. Furthermore, there is an orthogonal concern, which is that many of these types are trivial in another way: they are tiny, and should be passed around in the registers of a CPU, not indirectly in memory.
As such, Mojo provides a struct decorator that solves all of these problems. You can implement a type with the @register_passable("trivial")
decorator, and this tells Mojo that the type should be copyable and movable but that it has no user-defined logic for doing this. It also tells Mojo to prefer to pass the value in CPU registers, which can lead to efficiency benefits.
TODO: This decorator is due for reconsideration. Lack of custom logic copy/move/destroy logic and âpassability in a registerâ are orthogonal concerns and should be split. This former logic should be subsumed into a more general @value("trivial")
decorator, which is orthogonal from @register_passable
.
@value
decorator
Mojoâs approach (described above) provides simple and predictable hooks that give you the ability to express exotic low-level things like Atomic
correctly. This is great for control and for a simple programming model, but most structs we all write are simple aggregations of other types, and we donât want to have to write a lot of boilerplate for them! To solve this, Mojo provides a @value
decorator for structs that synthesizes the boilerplate for you. @value
can be thought of as an extension of Pythonâs @dataclass
handling the new __moveinit__
and __copyinit__
Mojo methods.
The @value
decorator takes a look at the fields of your type, and generates members that are missing. Consider a simple struct like this, for example:
@value
struct MyPet:
var name: String
var age: Int
Mojo will notice that you do not have a memberwise initializer, a move constructor or a copy constructor and will synthesize these for you as if you had written:
fn __init__(inout self, owned name: String, age: Int):
self.name = name^
self.age = age
fn __copyinit__(inout self, existing: Self):
self.name = existing.name
self.age = existing.age
fn __moveinit__(inout self, owned existing: Self):
self.name = existing.name^
self.age = existing.age
If your type contains any move-only fields, it cannot (and therefore will not) generate a copy constructor for you of course. Mojo only synthesizes these for you when they donât exist, so it is ok to override its behavior by defining your own version of these. For example, it is fairly common to want to define a custom copy constructor but use the default memberwise and move constructor.
There is no way to suppress the generation of specific methods or customize generation at this time, but we can add arguments to the @value
generator to do this if there is demand.
Note that the @value
decorator only works on types whose members are copyable and/or movable. If you have something like Atomic
in your struct, then it probably isnât a value type, and you donât want these members anyway.
Behavior of destructors
Any struct in Mojo can have a destructor, which is automatically run when the values lifetime ends. For example, a simple string might look like this (in pseudo code):
struct MyString:
var data: Pointer[UI8]
def __init__(inout self, input: StringRef): ...
def __add__(self, rhs: MyString) -> MyString: ...
def __del__(owned self):
self.data.address) free(
The Mojo compiler automatically invokes the destructor when the value is dead and provides strong guarantees about when the destructor is run. Mojo uses static compiler analysis to reason about your code and decide when to insert calls to the destructor. For example:
fn use_strings():
var a = MyString("hello a")
var b = MyString("hello b")
print(a)
# a.__del__() runs here
print(b)
# b.__del__() runs here
= MyString("temporary a")
a # a.__del__() runs here
other_stuff()
= MyString("final a")
a print(a)
# a.__del__() runs here
In the code above, youâll see that the a
and b
values are created early on, and each initialization of a value is matched with a call to a destructor. Notice also where the calls are happening: in the b
variable. For example, Mojo keeps the value live across the (unrelated) print of the a
variable until the print of the b
variable and destroys it immediately after that call. The a
value is destroyed immediately after its first print, and immediately after reassigning it a new (unused) temporary value, and after its final print.
Mojo destroys values using an âAs Soon As Possibleâ (ASAP) policy, behaving like a hyper-active garbage collector that is run after every call - and when we say every call, we mean it! Code that uses internal expressions (like a+b+c+d
) will destroy the intermediate expressions eagerly when they are not needed - destruction is not deferred to the end of the statement like in C++. Mojo fully understands control flow, including loops, ifs, and try/except of course.
Now, this may be surprising to a C++ programmer: this invalidates the use of the RAII pattern that C++ programmers use widely. So, why does Mojo destroy things so eagerly instead of using C++-style scoped destruction? Well Iâm glad you asked, there are many good reasons!
The Mojo design has a number of strong advantages over the C++ model:
- Recall that Python doesnât really have scopes beyond the whole function, and Mojo needs to provide a workable model that behaves correctly in the presence of Python-style âdefâs. 2. Because Python doesnât provide strong guarantees on object destruction, it doesnât encourage the RAII pattern. To solve for the RAII pattern, Mojo (and Python) provides a
with
statement that provides scoped access to resources, which is more deliberate and more syntactically clear than RAII. 3. The Mojo approach eliminates the need for types to implement re-assignment operators, likeoperator=(const T&)
andoperator=(T&&)
in C++, making it easier to define types and eliminating a concept. 4. Mojo does not allow mutable references to overlap with other mutable references or with immutable borrows. One major way that it provides a predictable programming model is by making sure that references to objects die as soon as possible, avoiding confusing situations where the compiler thinks a value could still be alive and interfere with another value, but that isnât clear to the user. 5. Destroying values at last-use composes nicely with âmoveâ optimization, which transforms a âcopy+delâ pair into a âmoveâ operation, a generalization of C++ move optimizations like NRVO. 6. Destroying values at end-of-scope in C++ is problematic for some common patterns like tail recursion because the destructor calls happen after the tail call. This can be a significant performance and memory problem for certain functional programming patterns.
The Mojo approach is more similar to how Rust and Swift work, because they both have strong value ownership tracking and provide memory safety. One difference is that their implementation requires the use of a dynamic âdrop flagâ - they maintain hidden shadow variables to keep track of the state of your values to provide safety. These are often optimized away, but the Mojo approach eliminates this overhead entirely, making the generated code faster and avoiding ambiguity.
Field sensitive lifetime management
In addition to Mojoâs lifetime analysis being fully control flow aware, it is also fully field sensitive (each field of a structure is tracked independently). It separately keeps track of whether a âwhole objectâ is initialized with an initializer or destroyed with a whole object destructor. For example, consider this code:
struct TwoStrings:
var str1: MyString
var str2: MyString
fn __init__(inout self): ...
fn __del__(owned self): ...
fn use_two_strings():
var ts = TwoStrings()
# ts.str1.__del__() runs here
other_stuff()
= MyString("hello a") # Overwrite ts.str1
ts.str1 print(ts.str1)
# ts.__del__() runs here
Note that the ts.str1
field is immediately destroyed after being set up, because Mojo knows that it will be overwritten down below. You can also see this when using the transfer operator, for example:
fn consume_and_use_two_strings():
var ts = TwoStrings()
^)
consume(ts.str1
# ts is partially initialized here!
other_stuff()
= MyString() # All together now
ts.str1 # This is ok
use(ts) # ts.__del__() runs here
Notice that the code transfers ownership of one of the fields: for the duration of other_stuff()
, the str1
field is completely uninitialized because ownership was transferred to consume()
. Fortunately for the code above, str1
is reinitialized before it is used by the use()
function - and if it werenât, Mojo would reject the code with an uninitialized field error.
Mojoâs rule on this is powerful and intentionally straight-forward: fields can be temporarily transferred, but the âwhole objectâ must be constructed with the aggregate typeâs initializer and destroyed with the aggregate destructor. This means that it isnât possible to create an object by initializing its fields, nor is it possible to tear down an object by destroying its fields:
fn consume_and_use_two_strings():
var ts = TwoStrings()
^)
consume(ts.str1^)
consume(ts.str2# Error: cannot run the 'ts' destructor without initialized fields.
var ts2 : TwoStrings
= MyString() # All together now
ts2.str1 = MyString() # All together now
ts2.str2 # Error: 'ts2' isn't fully initialized use(ts2)
While we could allow patterns like this to happen, we reject this because âa value is more than a sum of its partsâ. Consider a FileDescriptor
that contains a POSIX file descriptor as an integer value. For example - there is a big difference between destroying the integer (a no-op!) and destroying the FileDescriptor
(it might call the close()
system call). Because of this, we require all full-value initialization to go through initializers and be destroyed with their full-value destructor.
For what itâs worth, Mojo does internally have an equivalent of the Rust mem::forget
function, which explicitly disables a destructor and has a corresponding internal feature for âblessingâ an object, but they arenât exposed for user consumption at this point.
Field lifetimes in __init__
The behavior of an __init__
method works almost like any other method - there is a small bit of magic: it knows that the fields of an object are uninitialized, but it believes the full object is initialized. This means that you can use âselfâ as a whole object as soon as all the fields are initialized:
struct TwoStrings:
var str1: MyString
var str2: MyString
fn __init__(inout self, cond: Bool, other: MyString):
self.str1 = MyString()
if cond:
self.str2 = other
self) # Safe to use immediately!
use(# self.str2.__del__(): destroyed because overwritten below.
self.str2 = self.str1
self) # Safe to use immediately! use(
Similarly, it is completely safe for initializers in Mojo to completely overwrite self
, e.g. by delegating to other initializers:
struct TwoStrings:
var str1: MyString
var str2: MyString
fn __init__(inout self): ...
fn __init__(inout self, cond: Bool, other: MyString):
self = TwoStrings() # basic
self.str1 = MyString("fancy")
Field lifetimes of owned
arguments in __del__
and __moveinit__
A final bit of magic exists for the âownedâ arguments of a destructor and move initializer. To recap, these methods are defined like this:
struct TwoStrings:
var str1: MyString
var str2: MyString
fn __init__(...)
fn __moveinit__(inout self, owned existing: Self): ...
fn __del__(owned self): ...
These methods face an interesting but obscure problem: both of these methods are in charge of dismantling the owned existing
/self
value, either in destroying sub-elements that have to do with them, or using them to implement deletion logic for their own type. The move constructor wants to create a new self
instance by stealing parts from an existing instance. As such, they both want to own and transform elements of the owned
value and definitely donât want the owned valueâs destructor to run. The most egregious example of this is the __del__
method, which would turn into an infinite loop.
To solve this problem, Mojo handles these two methods specially by assuming that their whole values are destroyed upon reaching any return from the method. This means that the whole object may be used before the field values are transferred. For example, this works as you expect:
struct TwoStrings:
var str1: MyString
var str2: MyString
fn __init__(...)
fn __moveinit__(inout self, owned existing: Self): ...
fn __del__(owned self):
self) # Self is still whole
log(# self.str2.__del__(): Mojo destroys str2 since it isn't used
^)
consume(str1# Everything has now been transferred, no destructor is run on self.
You should not generally have to think about this, but if you have logic with inner pointers into members, you may need to keep them alive for some logic within the destructor or move initializer itself. You can do this by assigning to the discard pattern:
fn __del__(owned self):
self) # Self is still whole
log(
^)
consume(str1= self.str2
_ # self.str2.__del__(): Mojo destroys str2 after its last use.
In this case, if consume()
implicitly refers to some value in str2
somehow, this will ensure that str2
isnât destroyed until the last use when it is accessed by the _
pattern.
Lifetimes
TODO: Explain how returning references work, tied into lifetimes which dovetail with parameters. This is not enabled yet.
Type traits
This is a feature very much like Rust traits or Swift protocols or Haskell type classes. Note, this is not implemented yet.
Advanced/Obscure Mojo features
This section describes power-user features that are important for building the bottom-est level of the standard library. This level of the stack is inhabited by narrow features that require experience with compiler internals to understand and utilize effectively.
@register_passable
struct decorator
The default model for working with values is they live in memory, so they have an identity, which means they are passed indirectly to and from functions (equivalently, they are passed âby referenceâ at the machine level). This is great for types that cannot be moved, and is a safe default for large objects or things with expensive copy operations. However, it is inefficient for tiny things like a single integer or floating point number.
To solve this, Mojo allows structs to opt-in to being passed in a register instead of passing through memory with the @register_passable
decorator. Youâll see this decorator on types like Int
in the standard library:
@register_passable("trivial")
struct Int:
var value: __mlir_type.`!pop.scalar<index>`
fn __init__(value: __mlir_type.`!pop.scalar<index>`) -> Self:
return Self {value: value}
...
The basic @register_passable
decorator does not change the fundamental behavior of a type: it still needs to have a __copyinit__
method to be copyable, may still have a __init__
and __del__
methods, etc. The major effect of this decorator is on internal implementation details: @register_passable
types are typically passed in machine registers (subject to the details of the underlying architecture).
There are only a few observable effects of this decorator to the typical Mojo programmer:
@register_passable
types are not able to hold instances of types that are not themselves@register_passable
.Instances of
@register_passable
types do not have predictable identity, and so theself
pointer is not stable/predictable (e.g. in hash tables).@register_passable
arguments and result are exposed to C and C++ directly, instead of being passed by-pointer.The
__init__
and__copyinit__
methods of this type are implicitly static (like__new__
in Python) and returns its result by-value instead of takinginout self
.
We expect that this decorator will be used pervasively on core standard library types, but is safe to ignore for general application level code.
The Int
example above actually uses the âtrivialâ variant of this decorator. It changes the passing convention as described above but also disallows copy and move constructors and destructors (synthesizing them all trivially).
TODO: Trivial needs to be decoupled to its own decorator since it applies to memory types as well.
@always_inline
decorator
@always_inline("nodebug")
: same thing but without debug information so you donât step into the + method on Int.
@parameter
decorator
The @parameter
decorator can be placed on nested functions that capture runtime values to create âparametricâ capturing closures. This is an unsafe feature in Mojo, because we do not currently model the lifetimes of capture-by-reference. A particular aspect of this feature is that it allows closures that capture runtime values to be passed as parameter values.
Magic operators
C++ code has a number of magic operators that intersect with value lifecycle, things like âplacement newâ, âplacement deleteâ and âoperator=â that reassign over an existing value. Mojo is a safe language when you use all its language features and compose on top of safe constructs, but of any stack is a world of C-style pointers and rampant unsafety. Mojo is a pragmatic language, and since we are interested in both interoperating with C/C++ and in implementing safe constructs like String directly in Mojo itself, we need a way to express unsafe things.
The Mojo standard library Pointer[element_type]
type is implemented with an underlying !pop.pointer<element_type>
type in MLIR, and we desire a way to implement these C++-equivalent unsafe constructs in Mojo. Eventually, these will migrate to all being methods on the Pointer type, but until then, some need to be exposed as built-in operators.
Direct access to MLIR
Mojo provides full access to the MLIR dialects and ecosystem. Please take a look at the Low level IR in Mojo to learn how to use the __mlir_type
, __mlir_op
, and __mlir_type
constructs. All of the built-in and standard library APIs are implemented by just calling the underlying MLIR constructs, and in doing so, Mojo effectively serves as syntax sugar on top of MLIR.