Python class
EOSTracker
EOSTracker
class max.pipelines.modeling.types.EOSTracker(*, eos_token_ids=<factory>, eos_sequences=<factory>, eos_stop_strings=<factory>)
Bases: BaseModel
Centralized EOS tracking: single-ID, sequence-ID, and stop-sequence checks.
Used by Context and sampling to decide when generation stops and which token IDs to mask during min_tokens. Built once (e.g. by tokenizer from request params) and passed into context, so eos fields are immutable; Server and Context both use this type.
Three kinds of checks:
- EOS Single ID: token in eos_token_ids
- EOS Sequence ID: suffix of generated_tokens matches any eos_sequences
- EOS String Stop Sequence: decoded text contains a stop string.
-
Parameters:
eos_sequences
eos_sequences: Sequence[Sequence[int]]
eos_stop_strings
eos_stop_strings: Sequence[str]
eos_token_ids
is_eos_from_string()
is_eos_from_string(next_token_decoded)
Register an incremental decoded str into the continuation buffer.
If a stop sequence is detected, return the matched sequence. Else, return None.
is_eos_from_tokens()
is_eos_from_tokens(generated_tokens)
Checks for end-of-sequence conditions.
This function performs two checks:
- Whether the newly generated token is in the set of eos_token_ids.
- Whether appending the new token results in a sequence that matches any per-request stop sequence. Note: If called with a span of multiple newly generated tokens, EOS occurring before the final token in that span may not be detected by this method.
model_config
model_config: ClassVar[ConfigDict] = {}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
model_post_init()
model_post_init(_EOSTracker__context)
Post-initialization hook to set the maximum stop length and continuation tail.
-
Parameters:
-
_EOSTracker__context (Any)
-
Return type:
-
None
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!