For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Python class
BatchProcessor
BatchProcessor
class max.pipelines.lib.interfaces.BatchProcessor(config, runtime)
Bases: ABC, Generic[ContextT, InputsT]
Batches pipeline contexts into model inputs and parses execution outputs.
-
Parameters:
-
- config (ArchConfig)
- runtime (BatchProcessorRuntime)
get_symbolic_inputs()
abstract get_symbolic_inputs(*, kv_params, device_refs)
Returns non-KV graph input types in execution order.
-
Parameters:
-
- kv_params (KVCacheParamInterface)
- device_refs (list[DeviceRef])
-
Return type:
prepare_initial_token_inputs()
abstract prepare_initial_token_inputs(replica_batches, kv_cache_inputs=None, return_n_logits=1)
Prepares inputs for the first execution step of a batch.
process_outputs()
abstract process_outputs(outputs)
Maps raw Model.execute buffers to ModelOutputs.
-
Parameters:
-
Return type:
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!