Context

Preview 0.4.0

This is a preview of the Modular AI Engine. It is not publicly available yet and APIs are subject to change.

If you’re interested, please sign up for early access.

#include "modular/c/context.h"

Functions

M_RuntimeConfig *M_newRuntimeConfig()¶

Creates a new runtime config.

This configures runtime details such as the number of threads and log level.

By default, the config object’s number of threads will be set to 0, which is internally used to refer to the number of physical processors in the first socket in the system. You can change this with M_setNumThreads().

You need this as an argument for M_newRuntimeContext().

Returns:

A pointer to the new runtime config. You are responsible for the memory associated with the pointer returned. You can deallocate the memory by calling M_freeRuntimeConfig().

void M_setArtifactPath(M_RuntimeConfig *runtimeConfig, const char *path, M_Status *status)¶

Sets the location where a runtime stores its artifacts.

These artifacts are required for compilation and execution of models. If no artifact path is set, the runtime defaults to using a temporary directory.

Parameters:

  • runtimeConfig – The runtime config.

  • path – The artifacts path.

  • status – The status object for reporting errors.

void M_setNumThreads(M_RuntimeConfig *config, size_t numThreads)¶

Sets the number of threads in a runtime’s threadpool.

Parameters:

  • config – The runtime config.

  • numThreads – The number of threads.

size_t M_getNumThreads(M_RuntimeConfig *config)¶

Gets the number of threads in a runtime’s threadpool.

Parameters:

config – The runtime config.

Returns:

The number of threads in the the runtime’s threadpool. Otherwise, 0 if M_setNumThreads() has not been called.

void M_enableCrashLog(M_RuntimeConfig *config, const char *crashDir)¶

Enables crash logging and sets the location where crash dumps are stored. Note that this will install signal handlers to do so; ensure that this method is called last to unwind to previously registered handlers.

Parameters:

crashDir – The crash dump directory.

void M_freeRuntimeConfig(M_RuntimeConfig *config)¶

Deallocates the memory for a runtime config. No-op if config is NULL.

Parameters:

config – The runtime config.

M_RuntimeContext *M_newRuntimeContext(const M_RuntimeConfig *config, M_Status *status)¶

Creates a runtime context.

The context is an application-level object that sets up various resources such as threadpool and allocators during inference. You need this before you can call M_compileModel().

It’s expected that there’s only one runtime context active in an inference session at a time. We recommended you create one context and use it throughout your application.

For example:

M_Status *status = M_newStatus();
M_RuntimeConfig *runtimeConfig = M_newRuntimeConfig();
M_RuntimeContext *context = M_newRuntimeContext(runtimeConfig, status);
if (M_isError(status)) {
  logError(M_getError(status));
  return EXIT_FAILURE;
}

Parameters:

  • config – The runtime config, from M_newRuntimeConfig().

  • status – The status object for reporting errors. It is filled with an error message if construction of the runtime context fails.

Returns:

A pointer to the runtime context object. On success, this is a valid pointer. On failure, this is a NULL pointer with an error message in the status. You are responsible for the memory associated with the pointer returned. You can deallocate the memory by calling M_freeRuntimeContext().

void M_freeRuntimeContext(M_RuntimeContext *context)¶

Deallocates the memory for a runtime context. No-op if context is NULL.

Parameters:

context – The runtime context.