Mojo struct
Model
Represents a model that's loaded and ready for execution.
Do not instantiate this object directly. Instead, create it with
InferenceSession.load()
.
For example:
var session = engine.InferenceSession()
var model = session.load("bert-base-uncased")
Then you can run inference by passing your inputs to execute()
as a NumPy array, a
TensorMap
, or one
of the other tensor types.
Implemented traitsβ
AnyType
,
CollectionElement
,
Copyable
,
Movable
Methodsβ
__moveinit__
β
__moveinit__(inout self: Self, owned existing: Self)
Move initializer for model.
Args:
- βexisting (
Self
): Model to move.
__del__
β
__del__(owned self: Self)
Destructor for Model.
execute
β
execute(self: Self, inputs: TensorMap) -> TensorMap
Execute model with given inputs.
Args:
- βinputs (
TensorMap
): A tensor map with input names as keys and inputs as values.
Returns:
A TensorMap with output names as keys.
execute(self: Self, inputs: PythonObject) -> TensorMap
Execute model with given inputs.
Args:
- βinputs (
PythonObject
): Inputs as a Python object, which must be a dictionary with string keys (matching input names) and NumPy array values.
Returns:
A TensorMap with output names as keys.
execute(self: Self, *inputs: NamedTensor) -> TensorMap
Execute model with given inputs.
Args:
- β*inputs (
NamedTensor
): A variadic list of NamedTensor values.
Returns:
A TensorMap with output names as keys.
execute(self: Self, *inputs: Tuple[StringLiteral, EngineNumpyView]) -> TensorMap
Execute model with given inputs.
Args:
- β*inputs (
Tuple[StringLiteral, EngineNumpyView]
): A variadic list of tuples with first element of tuple is input name and second element is non owning view of a Numpy array.
Returns:
A TensorMap with output names as keys.
execute[type: DType](self: Self, name: String, input: Tensor[type]) -> TensorMap
Execute model with given input.
Parameters:
- βtype (
DType
): DType of input tensor.
Args:
- βname (
String
): Name of the input tensor. - βinput (
Tensor[type]
): Input tensor to the model.
Returns:
A TensorMap with output names as keys.
execute(self: Self, name: String, inout input: PythonObject) -> TensorMap
Execute model with given input.
Args:
- βname (
String
): Name of the input tensor. - βinput (
PythonObject
): Input to the model as numpy array.
Returns:
A TensorMap with output names as keys.
execute[type1: DType, type2: DType](self: Self, name1: String, input1: Tensor[type1], name2: String, input2: Tensor[type2]) -> TensorMap
Execute model with given inputs.
Parameters:
- βtype1 (
DType
): DType of first input tensor. - βtype2 (
DType
): DType of second input tensor.
Args:
- βname1 (
String
): Name of the first input tensor. - βinput1 (
Tensor[type1]
): First Input tensor to the model. - βname2 (
String
): Name of the second input tensor. - βinput2 (
Tensor[type2]
): Second Input tensor to the model.
Returns:
A TensorMap with output names as keys.
execute(self: Self, name1: String, inout input1: PythonObject, name2: String, inout input2: PythonObject) -> TensorMap
Execute model with given inputs.
Args:
- βname1 (
String
): Name of the first input tensor. - βinput1 (
PythonObject
): First Input to the model as numpy array. - βname2 (
String
): Name of the second input tensor. - βinput2 (
PythonObject
): Second Input to the model as numpy array.
Returns:
A TensorMap with output names as keys.
execute[type1: DType, type2: DType, type3: DType](self: Self, name1: String, input1: Tensor[type1], name2: String, input2: Tensor[type2], name3: String, input3: Tensor[type3]) -> TensorMap
Execute model with given inputs.
Parameters:
- βtype1 (
DType
): DType of first input tensor. - βtype2 (
DType
): DType of second input tensor. - βtype3 (
DType
): DType of third input tensor.
Args:
- βname1 (
String
): Name of the first input tensor. - βinput1 (
Tensor[type1]
): First Input tensor to the model. - βname2 (
String
): Name of the second input tensor. - βinput2 (
Tensor[type2]
): Second Input tensor to the model. - βname3 (
String
): Name of the third input tensor. - βinput3 (
Tensor[type3]
): Third Input tensor to the model.
Returns:
A TensorMap with output names as keys.
execute(self: Self, name1: String, inout input1: PythonObject, name2: String, inout input2: PythonObject, name3: String, inout input3: PythonObject) -> TensorMap
Execute model with given inputs.
Args:
- βname1 (
String
): Name of the first input tensor. - βinput1 (
PythonObject
): First Input to the model as numpy array. - βname2 (
String
): Name of the second input tensor. - βinput2 (
PythonObject
): Second Input to the model as numpy array. - βname3 (
String
): Name of the third input tensor. - βinput3 (
PythonObject
): Third Input to the model as numpy array.
Returns:
A TensorMap with output names as keys.
execute(self: Self, owned *inputs: AnyMemory) -> List[AnyMemory, 0]
Execute model with the given inputs.
Arguments: inputs: Inputs can either be tensors or mojo objects for opaque types defined in the graph, which may be located on any Device. If there are both tensor and opaque inputs, all tensor inputs should come before opaque inputs in graph. If inputs are tensors, API will automatically copy the input tensors to the Device set in the InferenceSession's SessionConfig.
Returns:
A list of outputs which can either be output tensors, which are located on the Device set in the InferenceSession's SessionConfig, or opaque mojo types as defined in the graph.
num_model_inputs
β
num_model_inputs(self: Self) -> Int
Gets the number of inputs of the model.
Returns:
Number of inputs of model.
get_model_input_names
β
get_model_input_names(self: Self) -> List[String, 0]
Gets the names of model inputs.
Returns:
Input names of the model.
num_model_outputs
β
num_model_outputs(self: Self) -> Int
Gets the number of outputs of the model.
Returns:
Number of model outputs.
get_model_output_names
β
get_model_output_names(self: Self) -> List[String, 0]
Gets the names of model outputs.
Returns:
Output names of the model.
get_model_input_metadata
β
get_model_input_metadata(self: Self) -> List[EngineTensorSpec, 0]
Get metadata about the model's input tensors, as a list of EngineTensorSpec
objects.
Returns:
Metadata list of the model's input tensors.
get_model_output_metadata
β
get_model_output_metadata(self: Self) -> List[EngineTensorSpec, 0]
Get metadata about the model's output tensors, as a list of EngineTensorSpec
objects.
Returns:
Metadata list of the model's output tensors.
export_compiled_model
β
export_compiled_model(self: Self, path: String)
Exports a compiled model as a MEF to a given path. Args: path: The path of the MEF file to export.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!
If you'd like to share more information, please report an issue on GitHub
π What went wrong?