Python class
SamplingParamsInput
SamplingParamsInput
class max.interfaces.SamplingParamsInput(top_k=None, top_p=None, min_p=None, temperature=None, frequency_penalty=None, presence_penalty=None, repetition_penalty=None, max_new_tokens=None, min_new_tokens=None, ignore_eos=None, stop=None, stop_token_ids=None, detokenize=None, seed=None, logits_processors=None)
Bases: object
Input dataclass for creating SamplingParams instances.
All fields are optional, allowing partial specification with None values
indicating “use default”. This enables static type checking while maintaining
the flexibility to specify only the parameters you want to override.
-
Parameters:
-
- top_k (int | None)
- top_p (float | None)
- min_p (float | None)
- temperature (float | None)
- frequency_penalty (float | None)
- presence_penalty (float | None)
- repetition_penalty (float | None)
- max_new_tokens (int | None)
- min_new_tokens (int | None)
- ignore_eos (bool | None)
- stop (list[str] | None)
- stop_token_ids (list[int] | None)
- detokenize (bool | None)
- seed (int | None)
- logits_processors (Sequence[Callable[[ProcessorInputs], None]] | None)
detokenize
Whether to convert output token IDs back to text. Defaults to None (use class default).
frequency_penalty
The penalty applied proportionally to token frequency in the generated text. Defaults to None (use class default).
ignore_eos
Whether to continue generating past end-of-sequence tokens. Defaults to None (use class default).
logits_processors
logits_processors: Sequence[Callable[[ProcessorInputs], None]] | None = None
Callables applied to model logits before sampling. Defaults to None (use class default).
max_new_tokens
The maximum number of tokens to generate. Defaults to None (use class default).
min_new_tokens
The minimum number of tokens to generate before stopping. Defaults to None (use class default).
min_p
The minimum probability threshold for a token relative to the most likely token. Defaults to None (use class default).
presence_penalty
The flat penalty applied to tokens that have appeared at least once. Defaults to None (use class default).
repetition_penalty
The factor by which logits of repeated tokens are divided. Defaults to None (use class default).
seed
The random seed for reproducible sampling. Defaults to None (use class default).
stop
A list of strings that, when generated, will stop the generation. Defaults to None (use class default).
stop_token_ids
A list of token IDs that, when generated, will stop the generation. Defaults to None (use class default).
temperature
The temperature for controlling output randomness. Defaults to None (use class default).
top_k
The number of most probable tokens to keep when sampling. Defaults to None (use class default).
top_p
The cumulative probability threshold for nucleus sampling. Defaults to None (use class default).
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!