For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Python class
BatchProcessorInputs
BatchProcessorInputs
class max.pipelines.context.BatchProcessorInputs(logits, logit_offsets, context_batch)
Bases: object
Arguments for a batch logits processor.
- logits: The model logits, a float32 tensor with shape
(N_batch, vocab_size).N_batchis the number of logits returned by the model for each sequence in the batch. - logit_offsets: If the model returns multiple logits, this is a tensor with
shape
(batch_size + 1, 1)that contains the offsets of each sequence in the batch. Otherwise, this isNone. - context_batch: The batch of contexts containing the inputs to the model.
-
Parameters:
-
- logits (md.Buffer)
- logit_offsets (md.Buffer | None)
- context_batch (Sequence[TextContext])
context_batch
context_batch: Sequence[TextContext]
The ordered sequence of generation contexts corresponding to each batch entry.
logit_offsets
logit_offsets: md.Buffer | None
Offsets tensor with shape (batch_size + 1, 1) for multi-logit models, or None.
logits
logits: md.Buffer
The model logits buffer with shape (N_batch, vocab_size) and float32 dtype.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!