For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Python class
TokenSampler
TokenSampler
class max.pipelines.sampling.TokenSampler(session, sampling_config, device, return_logits=False)
Bases: object
Samples tokens from the logits.
-
Parameters:
-
- session (InferenceSession)
- sampling_config (SamplingConfig)
- device (DeviceRef)
- return_logits (bool)
sample_logits()
sample_logits(logits, sampler_inputs, penalty_inputs=None)
Samples tokens from the logits.
-
Parameters:
-
- logits (Buffer)
- sampler_inputs (SamplerInputs)
- penalty_inputs (PenaltyInputs | None)
-
Return type:
sample_logits_with_prev()
sample_logits_with_prev(logits, prev_tokens, prev_logits, sampler_inputs, penalty_inputs=None)
Samples tokens from the logits with previous logits.
-
Parameters:
-
- logits (Buffer)
- prev_tokens (Buffer)
- prev_logits (Buffer)
- sampler_inputs (SamplerInputs)
- penalty_inputs (PenaltyInputs | None)
-
Return type:
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!