This is a preview of the Modular Inference Engine. It is not publicly available yet and APIs are subject to change.

If you’re interested, please sign up for early access.

#include "modular/c/types.h"


typedef struct M_Status M_Status

Contains the success or failure of an API call.

In general, any API that may fail will accept a M_Status function argument that is filled in with a meaningful error message on failure.

typedef struct M_RuntimeConfig M_RuntimeConfig

Specifies the Inference Engine configuration.

Configuration properties include the number of threads, artifact path, etc.

typedef struct M_RuntimeContext M_RuntimeContext

Contains information that needs to be shared between APIs.

It’s expected that there’s only one runtime context active in an inference session at a time.

typedef struct M_CompileConfig M_CompileConfig

Specifies the configuration required for model compilation.

typedef struct M_AsyncCompiledModel M_AsyncCompiledModel

Contains an async value to a compiled model.

M_compileModel() returns this type immediately to the caller. M_AsyncCompiledModel can be passed to other APIs that accept compiled models as a function parameter. This async value will eventually resolve to a compiled model or an error in the case of compilation failure.

typedef struct M_AsyncModel M_AsyncModel

Contains a future used for inference.

The future will resolve to a model ready for inference. Initialization APIs accept AsyncCompiledModel as an input and return a AsyncModel.

typedef enum M_Dtype M_Dtype

Represents all data types supported by the framework.

typedef struct M_AsyncTensor M_AsyncTensor

Contains an async value to a tensor for inference.

typedef struct M_TensorSpec M_TensorSpec

Contains the representation of a shape and an element type.

typedef struct M_AsyncTensorArray M_AsyncTensorArray

Contains a collection of tensors.

The collection of tensors is used to represent inputs and outputs when executing a model.


enum M_Dtype

Represents all data types supported by the framework.


enumerator M_UNKNOWN
enumerator M_INT8
enumerator M_INT16
enumerator M_INT32
enumerator M_INT64
enumerator M_UINT8
enumerator M_UINT16
enumerator M_UINT32
enumerator M_UINT64
enumerator M_FLOAT16
enumerator M_FLOAT32
enumerator M_DOUBLE
enumerator M_BOOL