MAX Engine C API

You can use the following C APIs to run inference with MAX Engine.

For a walkthrough, see how to run inference with C.

API headers

Each of the following pages represents one of the C API header files:

• Common

  • M_version()
  • M_newStatus()
  • M_getError()
  • M_isError()
  • M_freeStatus()
  • M_sizeOf()
  • M_getDynamicDimensionValue()
  • M_getDynamicRankValue()

• Context

  • M_newRuntimeConfig()
  • M_setArtifactPath()
  • M_setNumThreads()
  • M_setAllocatorType()
  • M_setCPUAffinity()
  • M_getNumThreads()
  • M_getCPUAffinity()
  • M_enableCrashLog()
  • M_freeRuntimeConfig()
  • M_newRuntimeContext()
  • M_freeRuntimeContext()

• Custom ops

  • M_useKernelsFrom()

• Model

  • M_newCompileConfig()
  • M_setModelPath()
  • M_enableVisualization()
  • M_compileModel()
  • M_waitForCompilation()
  • M_compileModelSync()
  • M_initModel()
  • M_getInputNames()
  • M_getOutputNames()
  • M_getTensorNameAt()
  • M_getModelInputSpecByName()
  • M_getModelOutputSpecByName()
  • M_waitForModel()
  • M_executeModelSync()
  • M_getNumModelInputs()
  • M_getNumModelOutputs()
  • M_validateInputTensorSpec()
  • M_freeModel()
  • M_freeCompiledModel()
  • M_freeCompileConfig()

• Tensor

  • M_newTensorSpec()
  • M_isDynamicRanked()
  • M_getDimAt()
  • M_getRank()
  • M_getDtype()
  • M_getName()
  • M_newAsyncTensorMap()
  • M_getTensorMapSize()
  • M_borrowTensorInto()
  • M_createBorrowedTensor()
  • M_getTensorByNameFrom()
  • M_getTensorFromValue()
  • M_getTensorNumElements()
  • M_getTensorType()
  • M_getTensorData()
  • M_getTensorSpec()
  • M_getTensorMapIterator()
  • M_advanceTensorMapIterator()
  • M_getNameFromMapIterator()
  • M_getTensorFromMapIterator()
  • M_isEndOfTensorMap()
  • M_freeTensor()
  • M_freeTensorNameArray()
  • M_freeTensorSpec()
  • M_freeAsyncTensorMap()
  • M_freeTensorMapIterator()

• Types

  • M_Status
  • M_RuntimeConfig
  • M_RuntimeContext
  • M_CompileConfig
  • M_DeviceConfig
  • M_AsyncCompiledModel
  • M_AsyncModel
  • M_AsyncTensor
  • M_TensorNameArray
  • M_TensorSpec
  • M_AsyncTensorMap
  • M_TensorMapIterator
  • M_AsyncValue
  • M_Config
  • M_AsyncDict
  • M_AsyncList
  • M_AsyncTuple
  • M_AsyncNone
  • M_Dtype
  • M_UNKNOWN
  • M_INT8
  • M_INT16
  • M_INT32
  • M_INT64
  • M_UINT8
  • M_UINT16
  • M_UINT32
  • M_UINT64
  • M_FLOAT16
  • M_FLOAT32
  • M_FLOAT64
  • M_BOOL
  • M_AllocatorType
  • kSystem
  • kCaching
  • M_ValueType
  • M_STRING_VALUE
  • M_DOUBLE_VALUE
  • M_LONG_VALUE
  • M_BOOL_VALUE
  • M_TENSOR_VALUE
  • M_LIST_VALUE
  • M_TUPLE_VALUE
  • M_DICT_VALUE
  • M_NONE_VALUE
  • M_UNKNOWN_VALUE

• Value

  • M_getValueByNameFrom()
  • M_getValueFromMapIterator()
  • M_freeValue()
  • M_getStringFromValue()
  • M_createStringAsyncValue()
  • M_getDoubleFromValue()
  • M_createDoubleAsyncValue()
  • M_getLongFromValue()
  • M_createLongAsyncValue()
  • M_getBoolFromValue()
  • M_createBoolAsyncValue()
  • M_borrowValueInto()
  • M_getValueType()
  • M_getDictFromValue()
  • M_createDictAsyncValue()
  • M_insertIntoDict()
  • M_getListFromValue()
  • M_createListAsyncValue()
  • M_appendToList()
  • M_getTupleFromValue()
  • M_borrowIntoTuple()
  • M_createTupleAsyncValue()
  • M_getDictSize()
  • M_getListSize()
  • M_getTupleSize()
  • M_getDictKey()
  • M_getDictValue()
  • M_getListValue()
  • M_getTupleValue()
  • M_createNoneAsyncValue()
  • M_freeDict()
  • M_freeList()
  • M_freeTuple()
  • M_freeNone()

• PyTorch config

  • M_newTorchInputSpec()
  • M_setTorchInputSpecs()
  • M_freeTorchInputSpec()

• PyTorch types

  • M_TorchInputSpec

Async API usage

Our C API allows for compiling and executing models asyncronously. In general, effective use of asynchronous APIs may be difficult, but rewarding for performance. To help with this, we’re going to explain some important concepts and mental models to keep in mind with the API.

Our APIs are async-safe unless stated otherwise, typically with a Sync in the function identifier name. For example, we have M_executeModel and M_executeModelSync().

Types

Our API describes the underlying async-holding types with a “value or error” concept. Conceptually, this means that the type is in one of three states:

• Constructed - the value is not yet there, but there is no error
• Available - the value is there and ready for use
• Error - the value is not there and there is an error

Synchronization points

When using async APIs, it is a good idea to be mindful of the synchronization point APIs currently provided below. This is useful for discerning between the Constructed and Available states mentioned above. After calling the synchronization point, the input will never be in a Constructed state: it will always resolve to either being Available or Error.

• M_waitForCompilation()
• M_waitForModel()
• M_waitForTensors

Errors

Errors surface immediately when using our syncronous APIs. Otherwise, in the case of async APIs, errors will not surface until the next synchronization point. You can query the error message by calling M_getError().