Tensor

Preview

MAX Engine is coming in Q1 2024. Sign up for updates.

#include "modular/c/tensor.h"

Functions

M_TensorSpec *M_newTensorSpec(const int64_t *shape, size_t rankSize, M_Dtype dtype, const char *tensorName)¶

Creates a tensor specification.

You need this before you set the input tensors with M_borrowTensorInto().

When storing tensor data in memory, we always use a diminishing stride size. That is, earlier dimensions in the shape have larger strides than later dimensions. For example, a C array declared as int arr[1][2][3] would have a shape specified as {1, 2, 3}.

Parameters:
  • shape – The shape of the tensor.

  • rankSize – The rank size of the tensor.

  • dtype – The datatype for the tensor.

  • tensorName – The name for the tensor. This string gets copied as part of the operation of M_newTensorSpec, so your original string need not remain valid after the completion of this call.

Returns:

A pointer to the tensor spec. You are responsible for the memory associated with the pointer returned. The memory can be deallocated by calling M_freeTensorSpec().

int64_t M_getDimAt(const M_TensorSpec *spec, size_t axis)¶

Gets the element at a particular axis.

Parameters:
  • spec – The tensor spec.

  • axis – The requested axis

Returns:

The dimension at requested axis if the spec and axis are valid. Otherwise, 0. A dimension of -1 indicates dynamic dimension e.g. batch-size of a model expecting a batched tensor.

size_t M_getRank(const M_TensorSpec *spec)¶

Gets the rank from the tensor spec.

Parameters:

spec – The tensor spec.

Returns:

The number of dimensions in the tensor spec if the spec is valid. Otherwise, 0.

M_Dtype M_getDtype(const M_TensorSpec *spec)¶

Gets the datatype from the tensor spec.

Parameters:

spec – The tensor spec.

Returns:

The element type from the tensor spec if the tensor spec is valid. Otherwise, M_UNKNOWN.

const char *M_getName(M_TensorSpec *spec)¶

Gets the name of the tensor from the tensor spec.

Parameters:

spec – The tensor spec.

Returns:

A null-terminated string containing the name of the tensor if the spec is valid. Otherwise, NULL. The memory associated with the returned string is owned by spec.

M_AsyncTensorMap *M_newAsyncTensorMap(const M_RuntimeContext *context)¶

Creates a map of tensor names to async tensors.

Parameters:
  • context – The runtime context.

  • size – The size of the collection.

Returns:

A pointer to the tensor map. You are responsible for the memory associated with the pointer returned. The memory can be deallocated by calling M_freeAsyncTensorMap().

size_t M_getTensorMapSize(const M_AsyncTensorMap *tensorMap, M_Status *status)¶

Gets the size of the tensor map.

Parameters:
  • tensorMap – The tensor map.

  • status – The status object for reporting errors.

Returns:

The size of the tensor map if the tensor map is valid. Otherwise, 0 and the status parameter contains an error message.

void M_borrowTensorInto(M_AsyncTensorMap *tensors, const void *input, const M_TensorSpec *tensorSpec, M_Status *status)¶

Adds a tensor to the tensor map.

You are responsible for the lifetime of the input tensor data. Its data gets “borrowed” into the Tensor Map.

Parameters:
  • tensors – The tensor map, from M_newAsyncTensorMap().

  • input – The input tensor data.

  • tensorSpec – The tensor spec, from M_newTensorSpec(). This gets copied as part of the operation of M_borrowTensorInto, so your original tensorSpec need not exist through the lifetime of the tensor map.

  • status – The status object for reporting errors.

M_AsyncValue *M_createBorrowedTensor(const void *data, const M_TensorSpec *tensorSpec, M_RuntimeContext *context)¶

Creates a borrowed tensor wrapped in an AsyncValue.

Parameters:
  • data – The tensor data.

  • tensorSpec – The tensor spec, from M_newTensorSpec(). This gets copied as part of the operation of M_createBorrowedTensor, so your original tensorSpec need not exist through the lifetime of the tensor.

  • context – The runtime context.

Returns:

A pointer to the value. You are responsible for the memory associated with the pointer returned. The memory can be deallocated by calling M_freeValue(). The held value inside the return value is owned by the AsyncValue, however the tensor data is borrowed and must outlive the returned M_AsyncValue.

M_AsyncTensor *M_getTensorByNameFrom(M_AsyncTensorMap *tensorMap, const char *name, M_Status *status)¶

Gets a tensor from the tensor map by name.

Parameters:
  • tensorMap – The tensor map.

  • tensorName – The name of the tensor.

  • status – The status object for reporting errors.

Returns:

A pointer to the tensor. You are responsible for the memory associated with the pointer returned. The memory can be deallocated by calling M_freeTensor(). The held tensor inside the return value is simply borrowed from the corresponding input M_AsyncTensorMap. If the tensor map or name are invalid, a NULL pointer is returned and the status parameter contains an error message.

M_AsyncTensor *M_getTensorFromValue(M_AsyncValue *value)¶

Gets a tensor from the async value.

Parameters:

value – The async value.

Returns:

A pointer to the tensor. You are responsible for the memory associated with the pointer returned. The memory can be deallocated by calling M_freeTensor(). The held tensor inside the return value is simply borrowed from the M_AsyncValue. Note that the tensor name is not available through this method (unlike M_getTensorByNameFrom). If the value is invalid or not a tensor, a NULL pointer is returned.

size_t M_getTensorNumElements(const M_AsyncTensor *tensor)¶

Gets the number of elements for the tensor.

Parameters:

tensor – The tensor which must not be NULL.

Returns:

The number of elements for the given tensor.

M_Dtype M_getTensorType(const M_AsyncTensor *tensor)¶

Gets the corresponding M_Dtype for the tensor.

Parameters:

tensor – The tensor which must not be NULL.

Returns:

The corresponding M_Dtype for the tensor.

const void *M_getTensorData(const M_AsyncTensor *tensor)¶

Gets a pointer to underlying data of the tensor.

Parameters:

tensor – The tensor which must not be NULL.

Returns:

A pointer to the underlying data of the tensor. This pointer is valid for the lifetime of the underlying tensor.

M_TensorSpec *M_getTensorSpec(const M_AsyncTensor *tensor)¶

Gets a Tensor Spec for the tensor.

Parameters:

tensor – The tensor.

Returns:

The tensor spec for the tensor if the tensor is valid. Otherwise, NULL.

M_TensorMapIterator *M_getTensorMapIterator(M_AsyncTensorMap *tensorMap, M_Status *status)¶

Gets a tensor map iterator for the tensor map.

Parameters:
  • tensorMap – The tensor map.

  • status – The status object for reporting errors.

Returns:

A pointer to the tensor map iterator. You are responsible for the memory associated with the pointer returned. The memory can be deallocated by calling M_freeTensorMapIterator(). If the tensor map is invalid, a NULL pointer is returned and the status parameter contains an error message.

void M_advanceTensorMapIterator(M_TensorMapIterator *iterator)¶

Advances the tensor map iterator by one entry.

Parameters:

iterator – The tensor map iterator.

const char *M_getNameFromMapIterator(M_TensorMapIterator *iterator)¶

Gets the name of the tensor from the tensor map iterator.

Parameters:

iterator – The tensor map iterator.

Returns:

A null-terminated string containing the name of the tensor if the iterator is valid. Otherwise, NULL. The memory associated with the returned string is owned by spec.

M_AsyncTensor *M_getTensorFromMapIterator(M_TensorMapIterator *iterator)¶

Gets the tensor from the tensor map iterator.

Parameters:

iterator – The tensor map iterator.

Returns:

A pointer to the tensor. You are responsible for the memory associated with the pointer returned. The memory can be deallocated by calling M_freeTensor(). The held tensor inside the return value is simply borrowed from the corresponding input M_AsyncTensorMap. If the tensor map iterator is invalid, a NULL pointer is returned.

bool M_isEndOfTensorMap(M_AsyncTensorMap *tensorMap, M_TensorMapIterator *iterator)¶

Checks if the iterator has reached the end of the tensor map.

Parameters:
  • tensorMap – The tensor map.

  • iterator – The tensor map iterator.

Returns:

True if the iterator points to the end of the map, false otherwise. Also returns true if either the tensorMap or iterator are invalid.

void M_freeTensor(M_AsyncTensor *tensor)¶

Deallocates the memory for the tensor. No-op if tensor is NULL.

Parameters:

tensor – The tensor to deallocate.

void M_freeTensorNameArray(M_TensorNameArray *names)¶

Deallocates the memory for the array of tensor names. No-op if names is NULL.

Parameters:

names – The tensor names to deallocate.

void M_freeTensorSpec(M_TensorSpec *spec)¶

Deallocates the memory for the tensor spec. No-op if spec is NULL.

Parameters:

spec – The tensor spec to deallocate.

void M_freeAsyncTensorMap(M_AsyncTensorMap *tensorMap)¶

Deallocates the memory for the tensor map. No-op if tensorMap is NULL.

Parameters:

tensorMap – The tensor map to deallocate.

void M_freeTensorMapIterator(M_TensorMapIterator *iterator)¶

Deallocates the memory for the tensor map iterator. No-op if iterator is NULL.

Parameters:

iterator – The tensor map iterator to deallocate.