For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Python class
SamplingConfig
SamplingConfig
class max.pipelines.SamplingConfig(*, config_file=None, section_name=None, in_dtype=float32, out_dtype=float32, enable_structured_output=False, enable_variable_logits=False, enable_penalties=False, enable_min_tokens=False)
Bases: ConfigFileModel
Configuration for the sampling stage of token generation.
-
Parameters:
-
- config_file (str | None)
- section_name (str | None)
- in_dtype (Annotated[DType, BeforeValidator(func=~max.pipelines.sampling.sampling_config._coerce_dtype, json_schema_input_type=PydanticUndefined)])
- out_dtype (Annotated[DType, BeforeValidator(func=~max.pipelines.sampling.sampling_config._coerce_dtype, json_schema_input_type=PydanticUndefined)])
- enable_structured_output (bool)
- enable_variable_logits (bool)
- enable_penalties (bool)
- enable_min_tokens (bool)
enable_min_tokens
enable_min_tokens: bool
enable_penalties
enable_penalties: bool
enable_structured_output
enable_structured_output: bool
enable_variable_logits
enable_variable_logits: bool
from_generation_config_sampling_defaults()
classmethod from_generation_config_sampling_defaults(sampling_params_defaults, **kwargs)
Creates a SamplingConfig from generation config defaults and kwargs.
Inspects the provided defaults to determine if penalty-related or min-tokens-related fields are set to non-default values; if so, enables the corresponding flags in the result unless already set in kwargs.
-
Parameters:
-
- sampling_params_defaults (SamplingParamsGenerationConfigDefaults) – The generation config defaults containing explicit values for sampling parameters.
- **kwargs – Additional keyword arguments to override or supplement the config.
-
Returns:
-
A new SamplingConfig instance with the appropriate fields set.
-
Return type:
in_dtype
in_dtype: _CoercedDType
model_config
model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'strict': False}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
model_post_init()
model_post_init(context, /)
This function is meant to behave like a BaseModel method to initialise private attributes.
It takes context as an argument since that’s what pydantic-core passes when calling it.
-
Parameters:
-
- self (BaseModel) – The BaseModel instance.
- context (Any) – The context.
-
Return type:
-
None
out_dtype
out_dtype: _CoercedDType
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!