Context
Preview 0.4.0
This is a preview of the Modular AI Engine. It is not publicly available yet and APIs are subject to change.
If you’re interested, please sign up for early access.
#include "modular/c/context.h"
Functions
-
M_RuntimeConfig *M_newRuntimeConfig()¶
Creates a new runtime config.
This configures runtime details such as the number of threads and log level.
By default, the config object’s number of threads will be set to
0
, which is internally used to refer to the number of physical processors in the first socket in the system. You can change this withM_setNumThreads()
.You need this as an argument for
M_newRuntimeContext()
.- Returns:
A pointer to the new runtime config. You are responsible for the memory associated with the pointer returned. You can deallocate the memory by calling
M_freeRuntimeConfig()
.
-
void M_setArtifactPath(M_RuntimeConfig *runtimeConfig, const char *path, M_Status *status)¶
Sets the location where a runtime stores its artifacts.
These artifacts are required for compilation and execution of models. If no artifact path is set, the runtime defaults to using a temporary directory.
- Parameters:
runtimeConfig – The runtime config.
path – The artifacts path.
status – The status object for reporting errors.
-
void M_setNumThreads(M_RuntimeConfig *config, size_t numThreads)¶
Sets the number of threads in a runtime’s threadpool.
- Parameters:
config – The runtime config.
numThreads – The number of threads.
-
size_t M_getNumThreads(M_RuntimeConfig *config)¶
Gets the number of threads in a runtime’s threadpool.
- Parameters:
config – The runtime config.
- Returns:
The number of threads in the the runtime’s threadpool. Otherwise,
0
ifM_setNumThreads()
has not been called.
-
void M_enableCrashLog(M_RuntimeConfig *config, const char *crashDir)¶
Enables crash logging and sets the location where crash dumps are stored. Note that this will install signal handlers to do so; ensure that this method is called last to unwind to previously registered handlers.
- Parameters:
crashDir – The crash dump directory.
-
void M_freeRuntimeConfig(M_RuntimeConfig *config)¶
Deallocates the memory for a runtime config. No-op if
config
is NULL.- Parameters:
config – The runtime config.
-
M_RuntimeContext *M_newRuntimeContext(const M_RuntimeConfig *config, M_Status *status)¶
Creates a runtime context.
The context is an application-level object that sets up various resources such as threadpool and allocators during inference. You need this before you can call
M_compileModel()
.It’s expected that there’s only one runtime context active in an inference session at a time. We recommended you create one context and use it throughout your application.
For example:
M_Status *status = M_newStatus(); M_RuntimeConfig *runtimeConfig = M_newRuntimeConfig(); M_RuntimeContext *context = M_newRuntimeContext(runtimeConfig, status); if (M_isError(status)) { logError(M_getError(status)); return EXIT_FAILURE; }
- Parameters:
config – The runtime config, from
M_newRuntimeConfig()
.status – The status object for reporting errors. It is filled with an error message if construction of the runtime context fails.
- Returns:
A pointer to the runtime context object. On success, this is a valid pointer. On failure, this is a
NULL
pointer with an error message in the status. You are responsible for the memory associated with the pointer returned. You can deallocate the memory by callingM_freeRuntimeContext()
.
-
void M_freeRuntimeContext(M_RuntimeContext *context)¶
Deallocates the memory for a runtime context. No-op if
context
is NULL.- Parameters:
context – The runtime context.