For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Python function
apply_logits_processors
apply_logits_processors()
max.pipelines.sampling.apply_logits_processors(context_batch, batch_logits, batch_logit_offsets, batch_processors=None)
Applies logits processors to a batch of logits.
-
Parameters:
-
- context_batch (list[TextGenerationContextType]) – The batch of contexts containing the inputs to the model.
- batch_logits (Buffer) – The model logits, a float32 tensor with shape (N_batch, vocab_size).
- batch_logit_offsets (Buffer | None) – If the model returns multiple logits, this is a tensor with shape (batch_size + 1, 1) that contains the offsets of each sequence in the batch. Otherwise, this is None.
- logits_processors – List of logits processors to apply to the logits for each context in the batch. The length of this list must match the number of contexts in the batch.
- batch_processors (list[Callable[[BatchProcessorInputs], None]] | None) – List of batch processors to apply to the batch logits. These are applied in order after the individual context-level processors.
-
Return type:
-
None
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!