Mojo struct
Model
struct Model
Represents a model that's loaded and ready for execution.
Do not instantiate this object directly. Instead, create it with
InferenceSession.load().
For example:
var session = engine.InferenceSession()
var model = session.load("bert-base-uncased")
var session = engine.InferenceSession()
var model = session.load("bert-base-uncased")
Then you can run inference by passing your inputs to execute()
as a NumPy array, a
TensorMap, or one
of the other tensor types.
Implemented traits
AnyType,
Copyable,
Movable,
UnknownDestructibility
Methods
__moveinit__
__moveinit__(out self, owned existing: Self)
Move initializer for model.
Args:
- existing (
Self): Model to move.
__del__
__del__(owned self)
Destructor for Model.
execute
execute(self, inputs: TensorMap) -> TensorMap
Execute model with given inputs.
Args:
- inputs (
TensorMap): A tensor map with input names as keys and inputs as values.
Returns:
A TensorMap with output names as keys.
execute(self, inputs: PythonObject) -> TensorMap
Execute model with given inputs.
Args:
- inputs (
PythonObject): Inputs as a Python object, which must be a dictionary with string keys (matching input names) and NumPy array values.
Returns:
A TensorMap with output names as keys.
execute(self, *inputs: NamedTensor) -> TensorMap
Execute model with given inputs.
Args:
- *inputs (
NamedTensor): A variadic list of NamedTensor values.
Returns:
A TensorMap with output names as keys.
execute(self, *inputs: Tuple[StringLiteral, EngineNumpyView]) -> TensorMap
Execute model with given inputs.
Args:
- *inputs (
Tuple[StringLiteral, EngineNumpyView]): A variadic list of tuples with first element of tuple is input name and second element is non owning view of a Numpy array.
Returns:
A TensorMap with output names as keys.
execute[type: DType](self, name: String, input: Tensor[type]) -> TensorMap
Execute model with given input.
Parameters:
- type (
DType): DType of input tensor.
Args:
- name (
String): Name of the input tensor. - input (
Tensor[type]): Input tensor to the model.
Returns:
A TensorMap with output names as keys.
execute(self, name: String, mut input: PythonObject) -> TensorMap
Execute model with given input.
Args:
- name (
String): Name of the input tensor. - input (
PythonObject): Input to the model as numpy array.
Returns:
A TensorMap with output names as keys.
execute[type1: DType, type2: DType](self, name1: String, input1: Tensor[type1], name2: String, input2: Tensor[type2]) -> TensorMap
Execute model with given inputs.
Parameters:
- type1 (
DType): DType of first input tensor. - type2 (
DType): DType of second input tensor.
Args:
- name1 (
String): Name of the first input tensor. - input1 (
Tensor[type1]): First Input tensor to the model. - name2 (
String): Name of the second input tensor. - input2 (
Tensor[type2]): Second Input tensor to the model.
Returns:
A TensorMap with output names as keys.
execute(self, name1: String, mut input1: PythonObject, name2: String, mut input2: PythonObject) -> TensorMap
Execute model with given inputs.
Args:
- name1 (
String): Name of the first input tensor. - input1 (
PythonObject): First Input to the model as numpy array. - name2 (
String): Name of the second input tensor. - input2 (
PythonObject): Second Input to the model as numpy array.
Returns:
A TensorMap with output names as keys.
execute[type1: DType, type2: DType, type3: DType](self, name1: String, input1: Tensor[type1], name2: String, input2: Tensor[type2], name3: String, input3: Tensor[type3]) -> TensorMap
Execute model with given inputs.
Parameters:
- type1 (
DType): DType of first input tensor. - type2 (
DType): DType of second input tensor. - type3 (
DType): DType of third input tensor.
Args:
- name1 (
String): Name of the first input tensor. - input1 (
Tensor[type1]): First Input tensor to the model. - name2 (
String): Name of the second input tensor. - input2 (
Tensor[type2]): Second Input tensor to the model. - name3 (
String): Name of the third input tensor. - input3 (
Tensor[type3]): Third Input tensor to the model.
Returns:
A TensorMap with output names as keys.
execute(self, name1: String, mut input1: PythonObject, name2: String, mut input2: PythonObject, name3: String, mut input3: PythonObject) -> TensorMap
Execute model with given inputs.
Args:
- name1 (
String): Name of the first input tensor. - input1 (
PythonObject): First Input to the model as numpy array. - name2 (
String): Name of the second input tensor. - input2 (
PythonObject): Second Input to the model as numpy array. - name3 (
String): Name of the third input tensor. - input3 (
PythonObject): Third Input to the model as numpy array.
Returns:
A TensorMap with output names as keys.
execute(self, owned *inputs: AnyMemory) -> List[AnyMemory]
Execute model with the given inputs.
Arguments: inputs: Inputs can either be tensors or mojo objects for opaque types defined in the graph, which may be located on any Device. If there are both tensor and opaque inputs, all tensor inputs should come before opaque inputs in graph. If inputs are tensors, API will automatically copy the input tensors to the Device set in the InferenceSession's SessionConfig.
Returns:
A list of outputs which can either be output tensors, which are located on the Device set in the InferenceSession's SessionConfig, or opaque mojo types as defined in the graph.
num_model_inputs
num_model_inputs(self) -> Int
Gets the number of inputs of the model.
Returns:
Number of inputs of model.
get_model_input_names
get_model_input_names(self) -> List[String]
Gets the names of model inputs.
Returns:
Input names of the model.
num_model_outputs
num_model_outputs(self) -> Int
Gets the number of outputs of the model.
Returns:
Number of model outputs.
get_model_output_names
get_model_output_names(self) -> List[String]
Gets the names of model outputs.
Returns:
Output names of the model.
get_model_input_metadata
get_model_input_metadata(self) -> List[EngineTensorSpec]
Get metadata about the model's input tensors, as a list of EngineTensorSpec objects.
Returns:
Metadata list of the model's input tensors.
get_model_output_metadata
get_model_output_metadata(self) -> List[EngineTensorSpec]
Get metadata about the model's output tensors, as a list of EngineTensorSpec objects.
Returns:
Metadata list of the model's output tensors.
export_compiled_model
export_compiled_model(self, path: String)
Exports a compiled model as a MEF to a given path. Args: path: The path of the MEF file to export.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!