For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Python class

BatchProcessorRuntime

BatchProcessorRuntime

class max.pipelines.lib.interfaces.BatchProcessorRuntime(pipeline_config, devices, return_logits, return_hidden_states=ReturnHiddenStates.NONE, signal_buffers=(), lora_manager=None, pad_token_id=0, max_batch_size=None)

source

Bases: object

Runtime dependencies shared by batch processors.

Parameters:

• pipeline_config (PipelineConfig)
• devices (list[Device])
• return_logits (ReturnLogits)
• return_hidden_states (ReturnHiddenStates)
• signal_buffers (Sequence[Buffer])
• lora_manager (LoRAManager | None)
• pad_token_id (int)
• max_batch_size (int | None)

devices

devices: list[Device]

source

lora_manager

lora_manager: LoRAManager | None = None

source

max_batch_size

max_batch_size: int | None = None

source

pad_token_id

pad_token_id: int = 0

source

pipeline_config

pipeline_config: PipelineConfig

source

return_hidden_states

return_hidden_states: ReturnHiddenStates = 'none'

source

return_logits

return_logits: ReturnLogits

source

signal_buffers

signal_buffers: Sequence[Buffer] = ()

source

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit