For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Python module

max.pipelines.sampling

Configuration

SamplingConfig	Configuration for the sampling stage of token generation.

Processors

FrequencyData	Container for token frequency data in CSR format.
FusedSamplingProcessor	Applies sampling parameters to logits and stores the chosen tokens.
PenaltyInputs	Container for penalty inputs.
SamplerInputs	Container for sampler inputs.

Samplers

RejectionRunner	Interface for rejection sampling runners.
SyntheticRunner	Synthetic acceptance sampler for benchmarking.
TokenSampler	Samples tokens from the logits.

rejection_runner_registry	Given a rejection runner strategy, returns the type of RejectionRunner.
rejection_sampler	Builds a graph that implements speculative decoding rejection sampling.
rejection_sampler_with_residuals	Builds a rejection sampler with residual sampling for speculative decoding.
token_sampler	Builds a sampling graph that samples tokens from logits.

Logits processing

apply_logits_processors	Applies logits processors to a batch of logits.
build_greedy_acceptance_sampler_graph	Builds a graph that implements strict greedy acceptance for MTP.
build_stochastic_acceptance_sampler_graph	Builds a target-only stochastic rejection sampler for speculative decoding.
build_synthetic_acceptance_sampler_graph	Builds a graph that implements synthetic acceptance sampling.

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit