Get started in Python

A walkthrough of the Python API, showing how to load and run a trained model.

MAX Engine is coming in Q1 2024. Sign up for updates.

The Python API for MAX Engine makes it easy to instantly upgrade your model’s inference performance. With just a few lines of code, you can run any TensorFlow or PyTorch model with reduced latency and compute cost.

This page shows you how to load a trained TensorFlow model and execute it with MAX Engine. (It’s just as easy with a PyTorch model.) If you’d like to see performance benchmarks with other models, see our performance dashboard.

We also offer a C API and our Mojo API is coming soon.

Import Python modules

import numpy as np
from modular import engine
from pathlib import Path

Load The Model

First, we need to create an InferenceSession and load the model. In this case, we’re using a BERT model from Hugging Face.

session = engine.InferenceSession()
model_path = Path('bert-base-uncased')
model = session.load(model_path)

This compiles the model into the MAX Engine format that’s ready for inference.

Run an inference

Before running the model, let’s check the input tensor name, shape, and data type (all three of which we need to know for the next step):

for tensor in model.input_metadata:
    print(f'name: {tensor.name}, shape: {tensor.shape}, dtype: {tensor.dtype}')
name: attention_mask, shape: [None, None], dtype: DType.int32
name: input_ids, shape: [None, None], dtype: DType.int32
name: token_type_ids, shape: [None, None], dtype: DType.int32

This indicates that the model has three input tensors, each with a unique name, but they all have the same shape and data type. When you see None as a dimension size, it means that size is unknown/dynamic.

Just to demonstrate how to use the MAX Engine API, let’s run an inference with meaningless input that matches the input shape. Notice that execute() requires you to specify the tensor name for your input as a keyword argument (which we printed in the previous code block):

inputs = {
    'attention_mask': np.zeros((1, 1), dtype=np.int32),
    'input_ids': np.zeros((1, 1), dtype=np.int32),
    'token_type_ids': np.zeros((1, 1), dtype=np.int32),
}

model.execute(**inputs)
{'pooler_output': array([[ 0.33634973,  0.4382758 , -0.880655  ,  0.5247563 ,  0.6226673 ,
          0.15791656, -0.1529078 , -0.09287227, -0.93732643,  0.8328007 ,
         -0.96353513,  0.9775384 , -0.6381251 ,  0.71126765, -0.23176268,
         -0.2377114 , -0.7837848 ,  0.27580774,  0.04403401,  0.6924035 ,
          0.07992269,  0.99178046, -0.7603975 ,  0.02017736, -0.27105588,
          0.9931121 , -0.52566427, -0.36698568, -0.32346362, -0.49057052,
         -0.19889057, -0.03442122, -0.03643664,  0.28650644, -0.92579997,
          0.63242227,  0.09924106,  0.42935947,  0.2194969 , -0.09932356,
         -0.03639645, -0.3432125 ,  0.4792686 ,  0.31321752,  0.46784794,
          0.25716123, -0.36892334, -0.06144615,  0.46011817,  0.9667667 ,
          0.80092657,  0.99057263, -0.11803398, -0.07922235, -0.02071325,
         -0.47276574, -0.14470139, -0.15124999,  0.32586476,  0.38809848,
          0.41079137, -0.4369291 , -0.8521993 ,  0.4440624 ,  0.98745507,
          0.9051021 , -0.1254215 ,  0.08910038,  0.19977519, -0.61087567,
          0.05059442, -0.19777554, -0.03385994,  0.04260049,  0.8408216 ,
         -0.20924443,  0.0989235 ,  0.68454915, -0.20256889,  0.28830904,
          0.9000486 ,  0.6900545 ,  0.06723185, -0.70728856,  0.69794244,
         -0.45646688, -0.08781175,  0.14618637,  0.34736162,  0.16225158,
          0.33332053,  0.04038048,  0.78631204,  0.00386609, -0.16937475,
         -0.2744523 , -0.12023388, -0.9194039 ,  0.04038318, -0.4393532 ,
         -0.09997462, -0.03953328,  0.11911136,  0.25096354,  0.0455033 ,
         -0.00542437,  0.4298188 ,  0.58886486,  0.33776587, -0.08027262,
         -0.05857795,  0.31116313, -0.25107923,  0.4144062 , -0.35520673,
          0.4781246 ,  0.13623935, -0.05205302,  0.53087944,  0.11776772,
         -0.42846337, -0.11736962, -0.65163755, -0.75944734, -0.03253395,
          0.29470742, -0.98139095, -0.78192395, -0.699058  , -0.4551184 ,
         -0.54358864, -0.03663593,  0.5516501 ,  0.2691304 , -0.4202187 ,
         -0.64051163, -0.08844347, -0.08354909, -0.24360047, -0.21451889,
         -0.89439756, -0.67664534,  0.94589114,  0.37256432,  0.817609  ,
          0.12155091,  0.19336265, -0.43896228, -0.57239467,  0.53162605,
         -0.8044149 ,  0.1482865 , -0.6339047 ,  0.5582202 ,  0.5150729 ,
         -0.0582227 , -0.73252225, -0.958113  , -0.9055014 ,  0.17088602,
         -0.24039227, -0.5345361 ,  0.7518124 , -0.53460693, -0.22541489,
         -0.4863761 ,  0.0617635 , -0.08663265,  0.5743458 , -0.8710392 ,
         -0.19398159, -0.12628348, -0.685362  , -0.5151056 , -0.03090599,
          0.09663671, -0.24636324, -0.38570905, -0.3937442 ,  0.9091646 ,
          0.324638  ,  0.6440926 ,  0.18234618, -0.21269724, -0.31115347,
          0.11122026, -0.6909473 ,  0.42006528,  0.21257406,  0.05033445,
          0.02804736, -0.7721633 ,  0.26802832,  0.01489835,  0.53264475,
          0.20849915, -0.9075251 ,  0.36240834,  0.27836612,  0.29188892,
          0.41129354, -0.29022446, -0.14925365, -0.2577127 ,  0.9282866 ,
          0.40887934,  0.6042675 ,  0.130684  , -0.31065765, -0.36881638,
          0.01610462, -0.15613987, -0.87557656,  0.27737865,  0.5602491 ,
          0.20738426, -0.46073356,  0.48145217,  0.08712011, -0.08988769,
          0.08646591, -0.7669825 ,  0.6001699 , -0.23401767,  0.7644532 ,
          0.2772029 ,  0.05628882,  0.3068098 ,  0.1820888 ,  0.4314455 ,
          0.68910956,  0.8012675 ,  0.20536554, -0.5642948 ,  0.08323602,
         -0.9155044 , -0.17850898, -0.49384674,  0.39770177, -0.28166476,
          0.24613939, -0.608928  ,  0.9533511 ,  0.8280281 ,  0.320014  ,
         -0.52401334,  0.7373039 , -0.30611423,  0.0640998 ,  0.82966655,
          0.8700948 , -0.01664729,  0.14347807, -0.34683093, -0.37705228,
         -0.89525414, -0.13888755,  0.34153894, -0.69852275, -0.25919223,
          0.6919279 ,  0.7983652 , -0.10081065,  0.9392834 ,  0.19298026,
         -0.02501265,  0.18599309, -0.22130483, -0.17681496, -0.7059771 ,
          0.32163167,  0.11370608, -0.1526765 , -0.07259192, -0.32591075,
         -0.00977677, -0.8995849 ,  0.08034595,  0.5910528 , -0.49268386,
         -0.38615912,  0.12556842, -0.9350363 , -0.6715234 ,  0.98155487,
         -0.6500931 , -0.42339957, -0.65928143, -0.9833358 , -0.8261918 ,
         -0.838392  , -0.9936503 , -0.8895074 ,  0.4867895 , -0.3782954 ,
         -0.2636482 , -0.8154136 ,  0.4020777 ,  0.22955087,  0.5764215 ,
          0.8454584 , -0.32935095, -0.8722197 , -0.82595   , -0.44873226,
          0.00754006,  0.41334474,  0.995194  , -0.06481443, -0.25662875,
          0.55265373, -0.26050502, -0.06725232, -0.14310834, -0.88823986,
          0.4142961 , -0.9235339 , -0.9405952 ,  0.9878059 , -0.1694679 ,
          0.29403093,  0.7207535 , -0.27301314,  0.195608  , -0.10488584,
         -0.06074034,  0.18423937,  0.9212    ,  0.5622187 ,  0.08636817,
          0.00432227,  0.65768886,  0.17051172,  0.6427843 , -0.67228806,
         -0.1676179 ,  0.78112555,  0.3595076 , -0.55232877,  0.86165583,
         -0.6012837 ,  0.17242254, -0.4627754 , -0.02757289,  0.9298384 ,
          0.8634413 ,  0.8052199 ,  0.31630492,  0.6474879 , -0.06057842,
         -0.33921236,  0.3029645 ,  0.41186956,  0.3117967 ,  0.21846788,
         -0.44689095,  0.2720084 , -0.15022373, -0.6550195 ,  0.4535245 ,
         -0.89818394, -0.909651  , -0.40910497,  0.6494213 ,  0.9879839 ,
          0.6859109 ,  0.3130218 , -0.5200646 , -0.26151237, -0.66935176,
          0.05911548, -0.16520773, -0.54868037, -0.24090023,  0.52449393,
          0.7413315 ,  0.23653533,  0.24434794, -0.48326525,  0.6400021 ,
          0.27857855, -0.23418446, -0.8538576 , -0.79252446,  0.8094041 ,
          0.07890143,  0.14444003, -0.04106639,  0.18055302, -0.2148896 ,
          0.5172356 ,  0.81428343,  0.864497  , -0.5708772 ,  0.9625277 ,
         -0.20074284,  0.18814279,  0.1132905 , -0.2773666 ,  0.03611476,
          0.08928124,  0.17913088,  0.8869471 , -0.1674644 ,  0.23089492,
         -0.28758898, -0.29669172,  0.40644294,  0.32292652,  0.28170916,
         -0.14623263,  0.6325908 ,  0.9956666 , -0.11793867, -0.20797001,
          0.9749485 , -0.22325923,  0.4123714 , -0.14609633, -0.24394032,
         -0.885814  ,  0.24081172,  0.32782194, -0.2378022 ,  0.5186626 ,
          0.46668583,  0.22077742, -0.42205545,  0.9576045 , -0.04076983,
         -0.10706503,  0.9108269 ,  0.8888355 , -0.96719116, -0.24185887,
          0.23620321, -0.7236194 , -0.987024  ,  0.32022288,  0.25947988,
          0.2964504 , -0.86658764,  0.08260533, -0.3121776 ,  0.31276184,
          0.9544305 ,  0.814349  ,  0.33598503,  0.6280616 , -0.78299356,
          0.65332663,  0.17075908, -0.94998395, -0.6441684 ,  0.5436886 ,
          0.7816253 ,  0.21552005,  0.19109654, -0.9535237 ,  0.02033621,
          0.02532255,  0.26298505, -0.11785381,  0.9131552 , -0.25250039,
         -0.88257664, -0.6318578 ,  0.10264025, -0.40519255,  0.0787983 ,
          0.7646762 , -0.9745462 ,  0.17277467,  0.6238525 ,  0.40705773,
          0.5315559 , -0.30413678, -0.39788696,  0.17600118, -0.2701695 ,
          0.8556776 , -0.0453406 , -0.6670514 , -0.98054487,  0.11606318,
          0.03907799, -0.22928193,  0.7061613 ,  0.75548255, -0.3848831 ,
          0.71481764,  0.12366836, -0.05044064, -0.40190345, -0.9239353 ,
          0.36062372, -0.0726086 ,  0.29713675,  0.03106394, -0.05300564,
         -0.20777689,  0.95885324, -0.12364084,  0.82722473,  0.43148172,
         -0.96751475, -0.06227767,  0.9909471 ,  0.19834334,  0.6033949 ,
          0.15706557,  0.6280709 ,  0.5672189 , -0.21609144,  0.21654314,
         -0.1056879 ,  0.15753616, -0.34205583, -0.53859776,  0.06965489,
         -0.05152934, -0.14888021, -0.26006377,  0.03405817,  0.9872242 ,
          0.24504972, -0.00331698,  0.37909335, -0.49331173, -0.8267014 ,
         -0.20420729,  0.10629248, -0.01956551,  0.25526962, -0.11268955,
         -0.0951492 , -0.05548569,  0.6107313 ,  0.98006386,  0.8965241 ,
         -0.9869701 , -0.0581628 , -0.38478655,  0.24196506,  0.40119156,
          0.66900975,  0.23268977,  0.8917936 , -0.2542645 , -0.06034516,
         -0.23486479, -0.767062  ,  0.35215703,  0.50672734,  0.19595486,
          0.23920481, -0.00602156,  0.3686828 , -0.7999747 , -0.27938306,
         -0.11407807,  0.26017648,  0.372448  ,  0.04732919,  0.05847509,
          0.17837268,  0.15731634,  0.276737  , -0.18426956,  0.25881308,
          0.61018324,  0.23662177,  0.06918693, -0.9851929 ,  0.17843312,
         -0.6918077 ,  0.7282228 ,  0.75399506,  0.28079405, -0.3257566 ,
          0.0741763 ,  0.7971313 ,  0.3615784 , -0.9554972 , -0.40646768,
         -0.47286633,  0.04450254, -0.8806803 ,  0.4224371 , -0.12540968,
          0.09113984, -0.14441568, -0.8496339 , -0.30955845,  0.00543724,
          0.5092119 , -0.1271866 , -0.71507424,  0.2949188 , -0.08226705,
          0.29091275,  0.98027474,  0.53195673,  0.04800846, -0.2117991 ,
         -0.326849  , -0.0422755 , -0.15559569, -0.15678278, -0.797286  ,
         -0.98556715, -0.529293  ,  0.15301251, -0.24851248,  0.36681545,
          0.22139   , -0.3238486 ,  0.032518  ,  0.95978093, -0.41477865,
          0.38036585, -0.8038362 , -0.8466853 , -0.051939  , -0.23846222,
         -0.40926054, -0.54235244, -0.27919865,  0.72026616, -0.03466899,
          0.5094433 , -0.2141696 ,  0.88317156,  0.273042  ,  0.7456856 ,
         -0.40016848,  0.0056108 ,  0.47364527, -0.06114733, -0.22960466,
         -0.94915867, -0.25643474, -0.94137096, -0.9375863 , -0.09925931,
         -0.55607975,  0.59748006,  0.60303676,  0.6612024 ,  0.28623286,
          0.17036872, -0.11351323, -0.5041499 , -0.10833682, -0.20048694,
         -0.927143  , -0.23719507, -0.42829823,  0.9194631 , -0.3539802 ,
          0.6704697 ,  0.33066177, -0.05752219,  0.42088094,  0.39100602,
          0.17481703,  0.3766574 , -0.04780364,  0.18703094, -0.45795926,
         -0.2910918 ,  0.10263997, -0.18213077, -0.927975  , -0.6848192 ,
          0.428496  ,  0.10197723, -0.8703106 ,  0.59603065, -0.40106347,
          0.4498196 ,  0.41199398, -0.6869667 , -0.88331467, -0.4727539 ,
         -0.3568111 , -0.18682222, -0.12640046, -0.09471475, -0.49729455,
         -0.4386251 , -0.6979312 , -0.7957786 , -0.223085  , -0.94872296,
         -0.1666007 ,  0.58617234,  0.5906526 ,  0.04163352,  0.7452141 ,
         -0.03302062, -0.46477315,  0.08495977,  0.7611173 ,  0.45579726,
          0.23555002,  0.53198206,  0.282265  , -0.13611242,  0.35181504,
          0.293087  ,  0.435594  , -0.63013345,  0.02405759,  0.11500448,
         -0.42590174,  0.9000811 ,  0.04299171,  0.14812674,  0.08838079,
         -0.6745266 ,  0.11514002, -0.7354059 ,  0.12494115, -0.5352946 ,
          0.9554149 , -0.48070928,  0.8871911 ,  0.49923268,  0.05406432,
         -0.09357029,  0.44152328, -0.46227756, -0.11459112,  0.26222312,
         -0.5401233 ,  0.07240213, -0.20145921,  0.991279  , -0.8326989 ,
         -0.94054514, -0.5549356 , -0.02815124, -0.3543411 , -0.5501085 ,
         -0.9785962 , -0.44103706, -0.8121113 ,  0.8386491 , -0.9405914 ,
          0.81872714, -0.8692777 ,  0.6617686 ,  0.09637178,  0.2352118 ,
          0.8849425 ,  0.25399473, -0.7176291 , -0.11014245, -0.9423573 ,
          0.99029136, -0.6857488 , -0.5696828 , -0.58251816, -0.02468232,
         -0.8538158 ,  0.1697502 , -0.49499127]], dtype=float32),
 'last_hidden_state': array([[[-0.13070619,  0.56097835, -0.32906005,  0.48060262,
           0.6118489 , -1.0306357 ,  0.8963195 ,  0.6179422 ,
          -0.17987405, -0.5308388 ,  0.02489086, -0.7941481 ,
          -0.65863836,  0.7507831 ,  0.05877931, -0.21967833,
           0.705258  , -0.05564936,  0.32577395,  0.14915857,
           0.7066365 , -0.9960112 , -0.2798843 ,  0.5891059 ,
           0.40155864, -0.14675494,  0.39346418,  0.49962792,
           0.16889544, -0.41312686,  0.31720483, -0.6966925 ,
           0.58417886,  0.09473374, -0.6508842 ,  0.14442143,
           0.14616063, -0.26926216, -1.0651721 ,  0.21444432,
          -1.1682739 , -0.3414799 , -0.40732875,  0.36571074,
           0.8547242 , -0.39251512,  0.04198571,  0.59213156,
           0.6390623 , -0.49417916, -1.192279  ,  0.08355719,
           0.08335092, -1.1903421 , -0.1838665 ,  1.15915   ,
          -1.0604228 , -0.04759855, -0.89778507, -0.5699103 ,
           0.46921715, -0.787195  ,  0.8996896 , -0.31775755,
           1.4546896 ,  0.1950307 ,  0.64556694,  0.3607471 ,
          -0.46158716, -0.6524823 , -0.45027173,  0.68374115,
          -0.06442644, -0.5214019 ,  0.11453422,  0.06585307,
          -0.21809451,  0.42394254, -0.48220205, -0.5924332 ,
          -0.36358798,  0.59559083, -0.34738058,  1.0918424 ,
          -0.06742915, -0.9581543 ,  0.29091635,  0.1453687 ,
           0.2764131 ,  0.23055223, -1.2097452 , -0.20437582,
          -0.3491998 , -1.3781356 ,  0.7685086 , -0.29232657,
           0.327684  ,  0.23722252, -0.68443996, -0.13461737,
           0.49594143, -0.22628468, -0.38584182, -0.49726185,
          -0.17716545,  0.42423505,  0.7464347 , -0.56362945,
           0.6386951 , -0.12130267, -0.22382918, -0.24958982,
          -0.473364  , -0.563211  , -0.38119677,  0.83049124,
           1.0771875 ,  0.10308248, -0.7701846 , -0.9506556 ,
           1.2429652 ,  0.03067552, -0.18202592,  0.9676093 ,
          -0.01748729,  0.98553383, -1.011939  ,  0.8643382 ,
          -0.62971157,  0.33480996,  0.6536365 ,  0.15744442,
           0.18352234,  0.61300075, -0.5683315 ,  0.11731039,
          -1.1450483 , -0.09883954, -0.34352437,  0.33869928,
          -0.24887194, -0.6524879 ,  0.4282187 ,  0.03476866,
           0.00370534, -0.1827907 ,  0.5481757 ,  0.5019692 ,
          -0.29213482, -0.30818295,  0.6249315 ,  0.2208839 ,
          -1.0242693 , -0.2878442 ,  1.0555813 ,  0.8276184 ,
           0.32440296, -0.22321635, -0.78868204, -0.0365466 ,
           0.68589205, -0.6670029 , -0.4400769 ,  0.19148067,
          -0.27178237,  0.88169247, -0.3487798 ,  1.56178   ,
           0.04470907,  0.20801564, -0.4361611 ,  0.07449178,
           0.72212845,  0.03190171,  0.5512577 ,  0.24432845,
          -0.05022848, -0.46128455, -0.4242184 , -0.17636691,
          -0.6326112 ,  0.24462402, -0.39085922, -0.03174145,
          -0.00242986, -0.6888315 ,  0.50026923, -0.48308945,
           0.40111348,  0.4193344 ,  0.05361585, -1.0117234 ,
          -0.21518369, -0.26869124,  0.1077061 ,  0.60052145,
           1.2707329 , -0.44914362, -0.7343387 , -0.48364905,
           0.62926245,  0.35575438, -0.40093362,  0.3542507 ,
          -1.4031749 , -1.1345705 ,  0.329091  ,  0.19254903,
           0.56586564, -0.53363967, -0.79022723,  0.38536552,
          -0.10097308,  0.6013102 ,  0.51846963, -0.6003553 ,
           0.20970125, -0.20465548,  0.1618816 , -0.28378808,
          -0.168573  ,  0.38219506, -1.5048846 , -0.5956948 ,
           0.5580945 ,  0.36405683, -1.0269406 , -0.23599753,
          -0.3552681 ,  0.75282145, -1.0173585 , -1.233976  ,
           1.4939828 , -0.16823877, -0.7078335 , -0.00704764,
           0.10064495, -0.66386956,  0.58724076, -1.5462672 ,
           0.28466442,  0.50153726,  0.04082686, -0.7145941 ,
           0.715618  , -0.23226747, -0.3260428 ,  0.11827889,
           1.1098704 , -0.09950624, -0.70716393,  0.5504059 ,
          -0.28363076, -0.99290115,  0.0411147 ,  0.02846446,
          -0.18537825, -0.5152807 , -0.10580173,  0.17291589,
           0.61984706, -0.36556485,  0.00486616,  0.2693878 ,
          -1.3599197 , -1.9409273 , -0.72550535,  0.5914056 ,
           0.74791867, -0.72604245, -0.03994111, -0.9326026 ,
          -0.9785843 ,  1.182484  , -0.5335201 , -0.10863065,
           0.5278269 , -0.68235475,  0.8714685 , -0.534378  ,
          -0.01968205,  0.82195896, -0.02339636, -0.81928885,
          -0.95943284,  0.2685683 , -1.152626  , -0.11575048,
           0.75500965, -0.8838433 ,  0.13063438, -0.8575024 ,
           0.15129848, -0.90684015, -0.4416466 , -0.07741572,
          -0.39743555,  0.7971681 ,  0.7112538 ,  0.8838111 ,
          -0.8998649 , -0.22221799, -0.27790585,  0.02170464,
           0.81562775,  0.2864136 ,  0.45526022, -0.4192294 ,
          -0.10821219, -0.01173865,  0.64296055,  0.08269848,
          -0.11293522,  0.68554634, -0.8301693 , -0.4327479 ,
          -1.0576909 ,  0.14193203,  0.06932534, -0.5195076 ,
           0.59102833,  0.54360783,  0.21455768, -0.37319353,
           0.13090962, -0.40314683,  0.22963008,  0.09903043,
           1.1161487 , -0.72996837, -0.6031664 ,  0.04990699,
          -0.09762714,  0.00903247, -0.2828101 , -0.5198949 ,
          -0.60348266, -0.6766751 ,  0.8413144 , -0.19436339,
          -1.4117619 ,  0.9457298 , -0.2981304 ,  0.38728854,
           0.28686443, -2.3321793 , -1.3453292 ,  0.20666274,
          -0.5443581 , -1.6220201 ,  0.23484229,  0.37728745,
           1.189149  ,  0.08249569,  0.6147379 , -0.60303545,
          -0.77022564,  0.39793923,  0.2570238 ,  0.27554822,
           0.5479777 , -0.9737311 , -0.67901045,  0.7233957 ,
           1.1048203 ,  0.3854454 , -0.26524433,  0.2140401 ,
           0.574837  ,  0.40410313, -0.20854959,  1.1095418 ,
          -0.56345534, -0.3375141 , -0.67238307,  0.46841094,
          -0.4674822 ,  0.12298109, -0.9446747 ,  0.5468424 ,
          -0.7965362 , -0.48967385, -0.8081539 , -0.89539266,
           0.89487106, -0.18034469, -0.14805181, -0.06285246,
           0.19352421,  0.4358164 ,  0.6472523 , -0.06244554,
          -0.7801361 ,  0.6939883 , -0.518143  ,  0.67748255,
          -0.18701209, -0.23456669, -0.38842577, -0.71358013,
           0.42467767,  0.01719525,  0.00361871, -0.2605585 ,
           1.317157  ,  1.5969329 ,  0.1676693 , -0.01457353,
          -0.6429688 ,  0.48833996,  0.07454808,  0.42695498,
           0.39698893,  0.41556072,  0.9391392 , -1.1077911 ,
           0.54680735,  0.47204286, -0.0060193 , -0.07285108,
          -0.39196125,  0.25913903,  0.13300426, -1.0794587 ,
           0.7493063 ,  0.80841917, -0.26729485, -0.3943474 ,
          -0.6534204 ,  0.2608106 , -0.47529066,  0.9670492 ,
          -0.3170671 ,  0.0138075 ,  0.31985512, -0.02042241,
           0.6785591 , -0.6166722 ,  0.6218157 , -0.14304695,
           0.45639703, -0.1008734 ,  0.54197073,  0.3151297 ,
           0.05306283, -1.2624941 , -0.8779191 , -0.15245746,
          -0.25547886,  1.0166787 , -0.5929626 ,  0.07251662,
           0.5577737 ,  0.38336763, -0.37024   ,  0.12942715,
           0.08602033,  0.21266347, -0.56360275,  0.43717054,
          -0.05164079, -1.2248243 ,  0.32070115,  0.3784562 ,
           0.5164251 , -0.02014172,  0.4315209 , -1.0380818 ,
           0.33220646,  0.30168092,  0.33311397, -0.2921508 ,
          -0.24783808,  0.38767207,  0.1342622 , -0.47082558,
          -0.00748836, -0.01956358, -0.23092298, -0.42161578,
           0.01787264, -0.00932573, -0.08945228,  1.1183335 ,
          -0.2635445 , -0.5009572 , -0.61574143,  0.46552953,
          -0.1808983 , -0.6943465 , -0.55508626, -0.3961518 ,
          -0.28220096, -0.139426  , -0.6626846 ,  0.0924177 ,
           0.5683468 ,  0.36154443, -0.27173975, -0.78231716,
          -1.4510425 , -0.32109642,  0.00911641,  0.2115351 ,
           0.20654476, -1.0521833 ,  0.3613393 , -0.63219416,
          -0.0781871 , -0.69445944,  1.40908   , -0.12779678,
           1.0228604 ,  0.34401464, -0.0319814 ,  0.32917175,
           0.36450374, -0.19675799, -0.63863415,  0.40534806,
          -0.2119247 , -0.2814421 ,  0.90881854,  0.95169586,
           0.18975805,  0.6358083 , -0.16188037, -0.17424636,
          -1.6459799 , -0.25374356,  0.95465386, -0.32829288,
          -0.6044571 , -0.9464598 , -0.14771427,  0.8498832 ,
          -0.31125507, -0.1227755 , -1.3208737 , -0.6577374 ,
           0.5650021 ,  0.10880428,  0.36928612,  0.46886718,
          -0.46674117, -0.4967023 ,  0.8338697 , -0.5070639 ,
          -0.8172393 , -0.37124208,  0.2626884 ,  0.79346454,
           0.54908127, -0.57489365,  0.02140209,  0.14203806,
           0.2094276 ,  1.0224231 , -0.2543878 , -0.57415134,
           0.37057814, -0.2854368 , -0.70745754, -0.5494478 ,
           0.5840061 , -0.27614033,  0.5971688 , -0.1229126 ,
          -0.9684051 ,  0.43697232, -0.8482494 ,  0.27393177,
           1.5629255 ,  0.6860556 ,  0.6928695 , -1.2773398 ,
           0.7712554 , -1.1218005 , -1.1269588 ,  0.13974695,
          -0.01901121,  0.05925995,  1.1428083 ,  1.2952515 ,
           0.21090265, -1.6537588 , -0.30215195,  0.20589031,
          -0.9080027 , -0.3349734 , -0.5914053 ,  0.64174354,
           0.57226133, -0.5045423 , -0.15689605, -0.2754728 ,
          -0.980956  , -0.504299  , -0.62797415, -0.4381451 ,
           0.04892979, -0.5060925 ,  0.08556013, -0.16442727,
          -0.11850684, -0.3642153 , -1.9185553 ,  0.45108473,
           1.2806623 , -0.5151494 ,  0.05267955,  0.04821999,
          -0.17906307,  0.31819203, -0.7849271 , -0.79304767,
           1.0654403 ,  0.3112794 , -0.4300018 , -0.8470579 ,
          -0.44891003,  0.5694208 , -1.6193643 , -0.39013588,
          -0.15061784,  0.6445516 ,  1.0701745 ,  0.4016454 ,
          -0.5101448 ,  0.6792764 ,  0.26808187, -0.143387  ,
           0.171201  ,  0.00970425,  1.1144677 , -1.140297  ,
           0.33604762,  1.1384777 , -0.18343508, -0.31613258,
           0.89570165,  0.60004616, -0.14598347, -0.5776477 ,
          -0.63589644,  0.9759306 ,  0.24864873,  0.575373  ,
          -0.09134383, -0.01411218,  0.34329557,  0.5277475 ,
          -0.13142121,  0.621231  ,  0.03124982, -0.14935152,
           0.01911194, -0.7359157 , -0.02892536,  1.3795599 ,
           0.02510865,  0.934321  ,  0.3668606 ,  0.03668516,
          -0.54722214, -1.0217199 , -0.4653033 ,  1.1937679 ,
           0.33779323,  0.1584824 ,  1.5445862 , -0.6568547 ,
           0.11253092,  0.1679722 , -0.05597013,  0.19083995,
          -0.37361452,  0.44019523,  0.18408822,  0.8437035 ,
          -1.0232302 ,  0.30196667,  0.6690449 ,  1.0930237 ,
           0.03833272,  0.04704016,  0.4525358 ,  0.15407525,
           0.4271382 ,  0.317348  ,  0.08353905, -0.59705514,
           1.5691302 ,  0.2868204 ,  0.5067944 , -0.1479588 ,
           0.07713384,  0.27432784, -0.24003467, -0.9230786 ,
           0.39977637,  0.357588  ,  0.6886464 , -0.78002554,
           0.54875964,  0.07926539,  0.2975999 ,  0.33668476,
           1.1851134 , -0.38585585,  0.68619084, -0.6292323 ,
          -0.32022542, -0.13575228,  0.6037366 ,  1.4577174 ,
          -0.85901105, -0.09229155, -0.05684938,  1.5015578 ,
           0.24948438,  0.3087986 , -0.40189296, -0.5135418 ,
           0.14051637,  0.4745335 , -0.3900953 , -0.40054792,
          -0.38887116,  0.73724246, -0.47079042,  0.5738027 ,
          -0.20209512,  1.1832703 , -0.20207691,  0.83604085,
           0.01479598,  0.64506805, -0.18430795, -1.2872366 ,
           0.09496463,  0.01046154,  0.30503735,  0.55138415,
           0.3144789 , -0.2784385 , -0.44463784, -0.56755364,
           1.0552932 , -0.46488783, -0.08018452, -0.6396279 ,
           0.21384002, -0.31687298,  0.04027668, -0.34859502,
          -0.14831465, -0.6796664 , -0.39032426, -0.34151927,
          -0.11555819, -0.44766372,  0.6413573 , -0.8301837 ,
          -0.68855345,  0.741763  ,  0.59531116,  0.37294528]]],
       dtype=float32)}

That’s it! execute() returns the output as a dictionary, in which each tensor is an ndarray that’s specified by its tensor name.

However, beware that this demo does not illustrate the MAX Engine performance—for real benchmark examples, see our performance dashboard.

For more details about the API, check out the Python API reference.

MAX Engine is coming in Q1 2024. Sign up for updates.