Python module
context
Standardized context object for Pipeline Inference.
InputContext
class max.pipelines.context.InputContext(*args, **kwargs)
A base class for model contexts, represent model inputs for TokenGenerators.
cache_seq_id
property cache_seq_id*: int*
current_length
property current_length*: int*
The current length of the sequence, including completed and active tokens.
is_done()
Returns true if token gen for this context completed, else false.
log_probabilities
property log_probabilities*: int*
When > 0, returns the log probabilities for the top N tokens for each element token in the sequence.
log_probabilities_echo
property log_probabilities_echo*: bool*
When True, the input tokens are added to the returned logprobs.
max_length
property max_length*: int*
The maximum length of this sequence.
seq_len
property seq_len*: int*
num tokens input this iteration.
This will be the prompt size for context encoding, and simply 1 for token generation.
-
Type:
Current sequence length
update()
Updates the next_tokens and extens existing tokens to include all generated tokens.
TextAndVisionContext
class max.pipelines.context.TextAndVisionContext(cache_seq_id: int, prompt: str, max_length: int, next_tokens: ndarray, pixel_values: ndarray | None = None, log_probabilities: int = 0, log_probabilities_echo: bool = False)
A base class for model context, specifically for Text model variants.
is_done()
Returns true if token gen for this context completed, else false.
next_tokens
property next_tokens*: ndarray*
seq_len
property seq_len*: int*
num tokens input this iteration.
This will be the prompt size for context encoding, and simply 1 for token generation.
-
Type:
Current sequence length
update()
Updates the next_tokens attribute, and extends current_length if needed, based on the provided num_steps.
TextContext
class max.pipelines.context.TextContext(cache_seq_id: int, prompt: str, max_length: int, next_tokens: ndarray, log_probabilities: int = 0, log_probabilities_echo: bool = False)
A base class for model context, specifically for Text model variants.
is_done()
Returns true if token gen for this context completed, else false.
next_tokens
property next_tokens*: ndarray*
seq_len
property seq_len*: int*
num tokens input this iteration.
This will be the prompt size for context encoding, and simply 1 for token generation.
-
Type:
Current sequence length
update()
Updates the next_tokens and extens existing tokens to include all generated tokens.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!