Testing
Mojo includes a framework for developing and executing unit tests. The Mojo
testing framework consists of a set of assertions defined as part of the Mojo
standard library and the mojo test command line
tool.
Get started
Let's start with a simple example of writing and running Mojo tests.
1. Write tests
For your first example of using the Mojo testing framework, create a file named
test_quickstart.mojo containing the following code:
# Content of test_quickstart.mojo
from testing import assert_equal
def inc(n: Int) -> Int:
return n + 1
def test_inc_zero():
# This test contains an intentional logical error to show an example of
# what a test failure looks like at runtime.
assert_equal(inc(0), 0)
def test_inc_one():
assert_equal(inc(1), 2)
# Content of test_quickstart.mojo
from testing import assert_equal
def inc(n: Int) -> Int:
return n + 1
def test_inc_zero():
# This test contains an intentional logical error to show an example of
# what a test failure looks like at runtime.
assert_equal(inc(0), 0)
def test_inc_one():
assert_equal(inc(1), 2)
In this file, the inc() function is the test target. The functions whose
names begin with test_ are the tests. Usually the target should be in a
separate source file from its tests, but you can define them in the same file
for this simple example.
A test function fails if it raises an error when executed, otherwise it
passes. The two tests in this example use the assert_equal() function,
which raises an error if the two values provided are not equal.
The implementation of test_inc_zero() contains an intentional logical error
so that you can see an example of a failed test when you execute it in the
next step of this tutorial.
2. Execute tests
Then in the directory containing the file, execute the following command in your shell:
mojo test test_quickstart.mojo
mojo test test_quickstart.mojo
You should see output similar to this (note that this example elides the full filesystem paths from the output shown):
Testing Time: 1.193s
Total Discovered Tests: 2
Passed : 1 (50.00%)
Failed : 1 (50.00%)
Skipped: 0 (0.00%)
******************** Failure: 'ROOT_DIR/test_quickstart.mojo::test_inc_zero()' ********************
Unhandled exception caught during execution
Error: At ROOT_DIR/test_quickstart.mojo:8:17: AssertionError: `left == right` comparison failed:
left: 1
right: 0
********************
Testing Time: 1.193s
Total Discovered Tests: 2
Passed : 1 (50.00%)
Failed : 1 (50.00%)
Skipped: 0 (0.00%)
******************** Failure: 'ROOT_DIR/test_quickstart.mojo::test_inc_zero()' ********************
Unhandled exception caught during execution
Error: At ROOT_DIR/test_quickstart.mojo:8:17: AssertionError: `left == right` comparison failed:
left: 1
right: 0
********************
The output starts with a summary of the number of tests discovered, passed, failed, and skipped. Following that, each failed test is reported along with its error message.
Next steps
- The
testingmodule describes the assertion functions available to help implement tests. - Writing unit tests shows how to write unit tests and organize them into test files.
- The
mojo testcommand describes how to execute and collect lists of tests. - The public GitHub repo contains an example project to demonstrate unit testing. Several of the examples shown later are based on this project.
The testing module
The Mojo standard library includes a testing
module that defines several assertion functions for implementing tests. Each
assertion returns None if its condition is met or raises an error if it isn't.
assert_true(): Asserts that the input value isTrue.assert_false(): Asserts that the input value isFalse.assert_equal(): Asserts that the input values are equal.assert_not_equal(): Asserts that the input values are not equal.assert_almost_equal(): Asserts that the input values are equal up to a tolerance.
The boolean assertions report a basic error message when they fail.
from testing import *
assert_true(False)
from testing import *
assert_true(False)
Unhandled exception caught during execution
Error: At Expression [1] wrapper:14:16: AssertionError: condition was unexpectedly False
Unhandled exception caught during execution
Error: At Expression [1] wrapper:14:16: AssertionError: condition was unexpectedly False
Each function also accepts an optional msg keyword argument for providing a
custom message to include if the assertion fails.
assert_true(False, msg="paradoxes are not allowed")
assert_true(False, msg="paradoxes are not allowed")
Unhandled exception caught during execution
Error: At Expression [2] wrapper:14:16: AssertionError: paradoxes are not allowed
Unhandled exception caught during execution
Error: At Expression [2] wrapper:14:16: AssertionError: paradoxes are not allowed
For comparing floating point values you should use assert_almost_equal(),
which allows you to specify either an absolute or relative tolerance.
result = 10 / 3
assert_almost_equal(result, 3.33, atol=0.001, msg="close but no cigar")
result = 10 / 3
assert_almost_equal(result, 3.33, atol=0.001, msg="close but no cigar")
Unhandled exception caught during execution
Error: At Expression [3] wrapper:15:24: AssertionError: 3.3333333333333335 is not close to 3.3300000000000001 with a diff of 0.0033333333333334103 (close but no cigar)
Unhandled exception caught during execution
Error: At Expression [3] wrapper:15:24: AssertionError: 3.3333333333333335 is not close to 3.3300000000000001 with a diff of 0.0033333333333334103 (close but no cigar)
The testing module also defines a context
manager,
assert_raises(), to assert that
a given code block correctly raises an expected error.
def inc(n: Int) -> Int:
if n == Int.MAX:
raise Error("inc overflow")
return n + 1
print("Test passes because the error is raised")
with assert_raises():
_ = inc(Int.MAX)
print("Test fails because the error isn't raised")
with assert_raises():
_ = inc(Int.MIN)
def inc(n: Int) -> Int:
if n == Int.MAX:
raise Error("inc overflow")
return n + 1
print("Test passes because the error is raised")
with assert_raises():
_ = inc(Int.MAX)
print("Test fails because the error isn't raised")
with assert_raises():
_ = inc(Int.MIN)
Unhandled exception caught during execution
Test passes because the error is raised
Test fails because the error isn't raised
Error: AssertionError: Didn't raise at Expression [4] wrapper:18:23
Unhandled exception caught during execution
Test passes because the error is raised
Test fails because the error isn't raised
Error: AssertionError: Didn't raise at Expression [4] wrapper:18:23
The example above assigns the return value from inc() to a
discard pattern.
Without it, the Mojo compiler reports a warning that the return value is unused.
You can also provide an optional contains argument to assert_raises() to
indicate that the test passes only if the error message contains the substring
specified. Other errors are propagated, failing the test.
print("Test passes because the error contains the substring")
with assert_raises(contains="required"):
raise Error("missing required argument")
print("Test fails because the error doesn't contain the substring")
with assert_raises(contains="required"):
raise Error("invalid value")
print("Test passes because the error contains the substring")
with assert_raises(contains="required"):
raise Error("missing required argument")
print("Test fails because the error doesn't contain the substring")
with assert_raises(contains="required"):
raise Error("invalid value")
Unhandled exception caught during execution
Test passes because the error contains the substring
Test fails because the error doesn't contain the substring
Error: invalid value
Unhandled exception caught during execution
Test passes because the error contains the substring
Test fails because the error doesn't contain the substring
Error: invalid value
Writing unit tests
A Mojo unit test is simply a function that fulfills all of these requirements:
- Has a name that starts with
test_. - Accepts no arguments.
- Returns either
Noneor a value of typeobject. - Raises an error to indicate test failure.
- Is defined at the module scope, not as a Mojo struct method.
You can use either def or fn to define a test function. Because a test
function always raises an error to indicate failure, any test function defined
using fn must include the raises declaration.
Generally, you should use the assertion utilities from the Mojo standard library
testing module to implement your tests.
You can include multiple related assertions in the same test function. However,
if an assertion raises an error during execution then the test function returns
immediately, skipping any subsequent assertions.
You must define your Mojo unit tests in Mojo source files named with a test
prefix or suffix. You can organize your test files within a directory hierarchy,
but the test files must not be part of a Mojo package (that is, the test
directories should not contain __init__.mojo files).
Here is an example of a test file containing three tests for functions defined
in a source module named my_target_module (which is not shown here).
# File: test_my_target_module.mojo
from my_target_module import convert_input, validate_input
from testing import assert_equal, assert_false, assert_raises, assert_true
def test_validate_input():
assert_true(validate_input("good"), msg="'good' should be valid input")
assert_false(validate_input("bad"), msg="'bad' should be invalid input")
def test_convert_input():
assert_equal(convert_input("input1"), "output1")
assert_equal(convert_input("input2"), "output2")
def test_convert_input_error():
with assert_raises():
_ = convert_input("garbage")
# File: test_my_target_module.mojo
from my_target_module import convert_input, validate_input
from testing import assert_equal, assert_false, assert_raises, assert_true
def test_validate_input():
assert_true(validate_input("good"), msg="'good' should be valid input")
assert_false(validate_input("bad"), msg="'bad' should be invalid input")
def test_convert_input():
assert_equal(convert_input("input1"), "output1")
assert_equal(convert_input("input2"), "output2")
def test_convert_input_error():
with assert_raises():
_ = convert_input("garbage")
The unique identity of a unit test consists of the path of the test file and the
name of the test function, separated by ::. So the test IDs from the example
above are:
test_my_target_module.mojo::test_validate_input()test_my_target_module.mojo::test_convert_input()test_my_target_module.mojo::test_convert_error()
The mojo test command
The mojo command line interface includes the mojo test
command for running tests or collecting a list of tests.
Running tests
By default, the mojo test command runs the tests that you specify using one of
the following:
- A single test ID with either an absolute or relative file path, to run only that test.
- A single absolute or relative file path, to run all tests in that file.
- A single absolute or relative directory path, to recurse through that directory hierarchy and run all tests found.
If needed, you can optionally use the -I option one or more times to append
additional paths to the list of directories searched to import Mojo modules and
packages. Consider the example testing
project in
GitHub, which has the following directory structure:
.
├── src
│ ├── example.mojo
│ └── my_math
│ ├── __init__.mojo
│ └── utils.mojo
└── test
└── my_math
├── test_dec.mojo
└── test_inc.mojo
.
├── src
│ ├── example.mojo
│ └── my_math
│ ├── __init__.mojo
│ └── utils.mojo
└── test
└── my_math
├── test_dec.mojo
└── test_inc.mojo
From the project root directory, you can execute all of the tests in the test
directory like this:
mojo test -I src test
mojo test -I src test
Testing Time: 3.433s
Total Discovered Tests: 4
Passed : 4 (100.00%)
Failed : 0 (0.00%)
Skipped: 0 (0.00%)
Testing Time: 3.433s
Total Discovered Tests: 4
Passed : 4 (100.00%)
Failed : 0 (0.00%)
Skipped: 0 (0.00%)
You can run the tests contained in only the test_dec.mojo file like this:
mojo test -I src test/my_math/test_dec.mojo
mojo test -I src test/my_math/test_dec.mojo
Testing Time: 1.175s
Total Discovered Tests: 2
Passed : 2 (100.00%)
Failed : 0 (0.00%)
Skipped: 0 (0.00%)
Testing Time: 1.175s
Total Discovered Tests: 2
Passed : 2 (100.00%)
Failed : 0 (0.00%)
Skipped: 0 (0.00%)
And you can run a single test from a file by providing its fully qualified ID like this:
mojo test -I src 'test/my_math/test_dec.mojo::test_dec_valid()'
mojo test -I src 'test/my_math/test_dec.mojo::test_dec_valid()'
Testing Time: 0.66s
Total Discovered Tests: 1
Passed : 1 (100.00%)
Failed : 0 (0.00%)
Skipped: 0 (0.00%)
Testing Time: 0.66s
Total Discovered Tests: 1
Passed : 1 (100.00%)
Failed : 0 (0.00%)
Skipped: 0 (0.00%)
Collecting a list of tests
By including the --collect-only or --co option, you can use mojo test to
discover and print a list of tests.
Consider the example testing
project
directory structure shown in the Running tests section. The
following command produces a list of all of the tests defined in the test
directory hierarchy.
mojo test --co test
mojo test --co test
The output shows the hierarchy of directories, test files, and individual tests (note that this example elides the full filesystem paths from the output shown):
<ROOT_DIR/test/my_math>
<ROOT_DIR/test/my_math/test_dec.mojo>
<ROOT_DIR/test/my_math/test_dec.mojo::test_dec_valid()>
<ROOT_DIR/test/my_math/test_dec.mojo::test_dec_min()>
<ROOT_DIR/test/my_math/test_inc.mojo>
<ROOT_DIR/test/my_math/test_inc.mojo::test_inc_valid()>
<ROOT_DIR/test/my_math/test_inc.mojo::test_inc_max()>
<ROOT_DIR/test/my_math>
<ROOT_DIR/test/my_math/test_dec.mojo>
<ROOT_DIR/test/my_math/test_dec.mojo::test_dec_valid()>
<ROOT_DIR/test/my_math/test_dec.mojo::test_dec_min()>
<ROOT_DIR/test/my_math/test_inc.mojo>
<ROOT_DIR/test/my_math/test_inc.mojo::test_inc_valid()>
<ROOT_DIR/test/my_math/test_inc.mojo::test_inc_max()>
Producing JSON formatted output
By default mojo test produces concise, human-readable output. Alternatively
you can produce JSON formatted output more suitable for input to other tools by
including the --diagnostic-format json option.
For example, you can run the tests in the test_quickstart.mojo file shown
in the Get started section with JSON formatted output using this
command:
mojo test --diagnostic-format json test_quickstart.mojo
mojo test --diagnostic-format json test_quickstart.mojo
The output shows the detailed results for each individual test and summary results (note that this example elides the full filesystem paths from the output shown):
{
"children": [
{
"duration_ms": 60,
"error": "Unhandled exception caught during execution",
"kind": "executionError",
"stdErr": "",
"stdOut": "Error: At ROOT_DIR/test_quickstart.mojo:8:17: AssertionError: `left == right` comparison failed:\r\n left: 1\r\n right: 0\r\n",
"testID": "ROOT_DIR/test_quickstart.mojo::test_inc_zero()"
},
{
"duration_ms": 51,
"error": "",
"kind": "success",
"stdErr": "",
"stdOut": "",
"testID": "ROOT_DIR/test_quickstart.mojo::test_inc_one()"
}
],
"duration_ms": 1171,
"error": "",
"kind": "executionError",
"stdErr": "",
"stdOut": "",
"testID": "ROOT_DIR/test_quickstart.mojo"
}
{
"children": [
{
"duration_ms": 60,
"error": "Unhandled exception caught during execution",
"kind": "executionError",
"stdErr": "",
"stdOut": "Error: At ROOT_DIR/test_quickstart.mojo:8:17: AssertionError: `left == right` comparison failed:\r\n left: 1\r\n right: 0\r\n",
"testID": "ROOT_DIR/test_quickstart.mojo::test_inc_zero()"
},
{
"duration_ms": 51,
"error": "",
"kind": "success",
"stdErr": "",
"stdOut": "",
"testID": "ROOT_DIR/test_quickstart.mojo::test_inc_one()"
}
],
"duration_ms": 1171,
"error": "",
"kind": "executionError",
"stdErr": "",
"stdOut": "",
"testID": "ROOT_DIR/test_quickstart.mojo"
}
You can also produce JSON output for test collection as well. Consider the
example testing
project
directory structure shown in the Running tests section. The
following command collects a list in JSON format of all of the tests defined in
the test directory hierarchy:
mojo test --diagnostic-format json --co test
mojo test --diagnostic-format json --co test
The output will appear as follows (note that this example elides the full filesystem paths from the output shown):
{
"children": [
{
"children": [
{
"id": "ROOT_DIR/test/my_math/test_dec.mojo::test_dec_valid()",
"location": {
"endColumn": 5,
"endLine": 19,
"startColumn": 5,
"startLine": 19
}
},
{
"id": "ROOT_DIR/test/my_math/test_dec.mojo::test_dec_min()",
"location": {
"endColumn": 5,
"endLine": 24,
"startColumn": 5,
"startLine": 24
}
}
],
"id": "ROOT_DIR/test/my_math/test_dec.mojo"
},
{
"children": [
{
"id": "ROOT_DIR/test/my_math/test_inc.mojo::test_inc_valid()",
"location": {
"endColumn": 5,
"endLine": 19,
"startColumn": 5,
"startLine": 19
}
},
{
"id": "ROOT_DIR/test/my_math/test_inc.mojo::test_inc_max()",
"location": {
"endColumn": 5,
"endLine": 24,
"startColumn": 5,
"startLine": 24
}
}
],
"id": "ROOT_DIR/test/my_math/test_inc.mojo"
}
],
"id": "ROOT_DIR/test/my_math"
}
{
"children": [
{
"children": [
{
"id": "ROOT_DIR/test/my_math/test_dec.mojo::test_dec_valid()",
"location": {
"endColumn": 5,
"endLine": 19,
"startColumn": 5,
"startLine": 19
}
},
{
"id": "ROOT_DIR/test/my_math/test_dec.mojo::test_dec_min()",
"location": {
"endColumn": 5,
"endLine": 24,
"startColumn": 5,
"startLine": 24
}
}
],
"id": "ROOT_DIR/test/my_math/test_dec.mojo"
},
{
"children": [
{
"id": "ROOT_DIR/test/my_math/test_inc.mojo::test_inc_valid()",
"location": {
"endColumn": 5,
"endLine": 19,
"startColumn": 5,
"startLine": 19
}
},
{
"id": "ROOT_DIR/test/my_math/test_inc.mojo::test_inc_max()",
"location": {
"endColumn": 5,
"endLine": 24,
"startColumn": 5,
"startLine": 24
}
}
],
"id": "ROOT_DIR/test/my_math/test_inc.mojo"
}
],
"id": "ROOT_DIR/test/my_math"
}
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!