Model debugging

MAX includes built-in debugging tools that help you diagnose common issues during model development, such as numerical accuracy problems, correctness bugs, performance bottlenecks, and hard-to-trace runtime errors. This page explains how to configure debugging options.

If you want to enable MAX's debugging defaults right away, you can set the environment variable MODULAR_DEBUG=sensible before running your model:

MODULAR_DEBUG=sensible max serve --model modularai/Llama-3.1-8B-Instruct-GGUF

See the DebugConfig API reference to review all debugging options.

When debugging happens

To configure debugging effectively, you must set debugging options at the correct stage of the model pipeline. Debugging options take effect at one of the following stages:

Graph build time: Set debugging options before you instantiate a Graph object. Options configured after graph creation don't apply.
Model build time: Set debugging options before you compile the graph. Debugging is integrated during compilation, so you must rebuild the model for any changes to take effect.
Run time: Set debugging options before you run the model. You can modify these options between runs without rebuilding an already compiled model.

This matters when using the Python API, since Python execution is order-dependent. Environment variables and config files load before any processes run, so they're always in effect when needed.

For the timing requirements of each debugging option, see the DebugConfig API reference.

Configure debugging options

You can configure debugging options using a few different strategies. When you set the same options using multiple strategies, precedence applies from highest to lowest:

Using the Python API (highest)
Setting the MODULAR_DEBUG environment variable
Updating the modular.cfg file (lowest)

The following sections walk through how to set up debugging at each level.

Environment variable and config file options use kebab-case names (nan-check), while the Python API uses the underscored form (nan_check).

Use the Python API

The Python API provides the most direct way to enable debugging options. Most options live on InferenceSession.debug except for source_tracebacks, which lives on Graph.debug because it's consumed during graph construction.

Debug options are process-wide, so set them on the class (rather than an instance) before you create the corresponding object:

from max.engine import InferenceSession
from max.graph import Graph

# Set source_tracebacks before graph construction
Graph.debug.source_tracebacks = True
g = Graph("my_model")


# Set before session.load() compiles the graph
InferenceSession.debug.nan_check = True
InferenceSession.debug.op_log_level = "trace"

session = InferenceSession()
model = session.load(g)

# Set before calling execute()
InferenceSession.debug.device_sync_mode = True
outputs = model.execute(inputs_a)

InferenceSession.debug.device_sync_mode = False
outputs = model.execute(inputs_b)

# Clear every option set through the Python API
# Values from MODULAR_DEBUG and modular.cfg remain in effect
InferenceSession.debug.reset()

Set the `MODULAR_DEBUG` environment variable

Another option is to use an environment variable to enable your debugging options. This is the recommended way to set global options.

Set the MODULAR_DEBUG environment variable to a comma-separated list of debug option names. For options that take a value, use =.

export MODULAR_DEBUG=nan-check,device-sync-mode,source-tracebacks,assert-level=safe

Update your config file

You can also enable debugging options in a Modular config file.

Place the modular.cfg file in your project root or home directory. Then, add a [max-debug] section to the file. Here's an example configuration:

[max-debug]
nan-check = true
device-sync-mode = true
op-log-level = trace

Next steps

Now that you know how to set up debugging in MAX, review the full list of debugging options in the DebugConfig class. You'll find detailed specifications for each option (property) including the type and accepted values.

When debugging happens​

Configure debugging options​

Use the Python API​

Set the MODULAR_DEBUG environment variable​

Update your config file​

Next steps​