C API
You can use the following C APIs to run inference with MAX Engine.
For a walkthrough, see how to run inference with C.
API headers
Each of the following pages represents one of the C API header files:
-
M_newCompileConfig()
M_cloneCompileConfig()
M_setModelPath()
M_newModelSource()
M_setModelSource()
M_enableVisualization()
M_compileModel()
M_waitForCompilation()
M_compileModelSync()
M_initModel()
M_getInputNames()
M_getOutputNames()
M_getTensorNameAt()
M_getModelInputSpecByName()
M_getModelOutputSpecByName()
M_waitForModel()
M_executeModelSync()
M_getNumModelInputs()
M_getNumModelOutputs()
M_validateInputTensorSpec()
M_freeModel()
M_freeCompiledModel()
M_freeCompileConfig()
M_freeModelSource()
M_exportCompiledModel()
-
M_newTensorSpec()
M_isDynamicRanked()
M_getDimAt()
M_getRank()
M_getDtype()
M_getName()
M_newAsyncTensorMap()
M_copyAsyncTensorMap()
M_getTensorMapSize()
M_borrowTensorInto()
M_createBorrowedTensor()
M_getTensorByNameFrom()
M_tensorMapKeys()
M_deleteTensorMapKeys()
M_getTensorFromValue()
M_getTensorNumElements()
M_getTensorType()
M_getTensorData()
M_getTensorSpec()
M_getTensorMapIterator()
M_advanceTensorMapIterator()
M_getNameFromMapIterator()
M_getTensorFromMapIterator()
M_isEndOfTensorMap()
M_freeTensor()
M_freeTensorNameArray()
M_freeTensorSpec()
M_freeAsyncTensorMap()
M_freeTensorMapIterator()
-
M_Status
M_RuntimeConfig
M_RuntimeContext
M_UInt64Counter
M_DoubleCounter
M_UInt64Histogram
M_DoubleHistogram
M_Int64Gauge
M_DoubleGauge
M_CustomMetricReader
M_CompileConfig
M_DeviceConfig
M_AsyncCompiledModel
M_AsyncModel
M_AsyncTensor
M_TensorNameArray
M_TensorSpec
M_AsyncTensorMap
M_TensorMapIterator
M_AsyncValue
M_Config
M_AsyncDict
M_AsyncList
M_AsyncTuple
M_AsyncNone
M_MaxContext
M_ModelSource
M_WeightsRegistry
M_Dtype
M_UNKNOWN
M_INT8
M_INT16
M_INT32
M_INT64
M_UINT8
M_UINT16
M_UINT32
M_UINT64
M_FLOAT16
M_FLOAT32
M_FLOAT64
M_BFLOAT16
M_BOOL
M_AllocatorType
kSystem
kCaching
M_ValueType
M_STRING_VALUE
M_DOUBLE_VALUE
M_LONG_VALUE
M_BOOL_VALUE
M_TENSOR_VALUE
M_LIST_VALUE
M_TUPLE_VALUE
M_DICT_VALUE
M_NONE_VALUE
M_UNKNOWN_VALUE
M_FrameworkFormat
M_MAX_GRAPH_FRAMEWORK_FORMAT
M_TORCHSCRIPT_MODULE_FRAMEWORK_FORMAT
M_TORCHSCRIPT_FUNCTION_FRAMEWORK_FORMAT
M_TORCH_MLIR_FRAMEWORK_FORMAT
M_ResultOutputStyle
M_COMPACT
M_FULL
M_BINARY
M_NONE
-
M_getValueByNameFrom()
M_getValueFromMapIterator()
M_freeValue()
M_getStringFromValue()
M_createStringAsyncValue()
M_getDoubleFromValue()
M_createDoubleAsyncValue()
M_getLongFromValue()
M_createLongAsyncValue()
M_getBoolFromValue()
M_createBoolAsyncValue()
M_borrowValueInto()
M_getValueType()
M_getDictFromValue()
M_createDictAsyncValue()
M_insertIntoDict()
M_getListFromValue()
M_createListAsyncValue()
M_appendToList()
M_getTupleFromValue()
M_borrowIntoTuple()
M_createTupleAsyncValue()
M_getDictSize()
M_getListSize()
M_getTupleSize()
M_getDictKey()
M_getDictValue()
M_getListValue()
M_getTupleValue()
M_createNoneAsyncValue()
M_freeDict()
M_freeList()
M_freeTuple()
M_freeNone()
Async API usage
Our C API allows for compiling and executing models asynchronously. In general, effective use of asynchronous APIs may be difficult, but rewarding for performance. To help with this, we’re going to explain some important concepts and mental models to keep in mind with the API.
Our APIs are async-safe unless stated otherwise, typically with a Sync
in the
function identifier name. For example, we have M_executeModel
and
M_executeModelSync()
.
Types
Our API describes the underlying async-holding types with a “value or error” concept. Conceptually, this means that the type is in one of three states:
Constructed
- the value is not yet there, but there is no errorAvailable
- the value is there and ready for useError
- the value is not there and there is an error
Synchronization points
When using async APIs, it is a good idea to be mindful of the synchronization
point APIs currently provided below. This is useful for discerning between the
Constructed
and Available
states mentioned above. After calling the
synchronization point, the input will never be in a Constructed
state: it will
always resolve to either being Available
or Error
.
M_waitForCompilation()
M_waitForModel()
M_waitForTensors
Errors
Errors surface immediately when using our synchronous APIs. Otherwise, in the
case of async APIs, errors will not surface until the next synchronization
point. You can query the error message by calling M_getError()
.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!
😔 What went wrong?