Python class
SamplingParamsGenerationConfigDefaults
SamplingParamsGenerationConfigDefaults
class max.interfaces.SamplingParamsGenerationConfigDefaults(temperature=None, top_p=None, top_k=None, repetition_penalty=None, max_new_tokens=None, min_new_tokens=None, do_sample=None, eos_token_id=None)
Bases: object
Default sampling parameter values extracted from a model’s GenerationConfig.
This class encapsulates sampling parameter defaults that come from a HuggingFace model’s GenerationConfig. These defaults have middle priority when creating SamplingParams instances:
Priority order (highest to lowest):
- User-provided values (SamplingParamsInput)
- Model’s GenerationConfig values (this class)
- SamplingParams class defaults
All fields default to None, indicating that the model’s GenerationConfig does not explicitly set that parameter. When None, SamplingParams will fall back to its own class defaults.
defaults = SamplingParamsGenerationConfigDefaults(
temperature=0.7,
top_k=50,
max_new_tokens=512,
)
params = SamplingParams.from_input_and_generation_config(
SamplingParamsInput(),
sampling_params_defaults=defaults,
)-
Parameters:
do_sample
If False, uses greedy sampling.
eos_token_id
EOS token ID from the model’s GenerationConfig, if explicitly set.
max_new_tokens
Maximum number of new tokens from the model’s GenerationConfig, if explicitly set.
min_new_tokens
Minimum number of new tokens from the model’s GenerationConfig, if explicitly set.
repetition_penalty
Repetition penalty value from the model’s GenerationConfig, if explicitly set.
temperature
Temperature value from the model’s GenerationConfig, if explicitly set.
top_k
Top-k sampling value from the model’s GenerationConfig, if explicitly set.
top_p
Top-p (nucleus sampling) value from the model’s GenerationConfig, if explicitly set.
values_to_update
property values_to_update: dict[str, float | int | list[int]]
Non-None field values as a dictionary.
-
Returns:
-
A dictionary mapping field names to their values, excluding any fields that are
None. This dictionary can be used to updateSamplingParamsdefault values.
defaults = SamplingParamsGenerationConfigDefaults(
temperature=0.7,
top_k=50,
)
defaults.values_to_updateWas this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!