For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo module

toppminp_gpu

comptime values

DEBUG_FILE

comptime DEBUG_FILE = False

SEED

comptime SEED = 42

Structs

• ​DoubleBuffer:

Functions

• ​min_p_sampling_gpu: GPU implementation of Min-P sampling for token selection. This function applies temperature scaling, softmax, a radix sort, and then samples tokens based on the calculated probability threshold (Min-P).
• ​normalize:
• ​normalize_u32:
• ​radix_sort_pairs_kernel: Radix pair sort kernel for (default) descending order.
• ​reinterpret:
• ​run_radix_sort_pairs_gpu:
• ​top_p_sampling_gpu: GPU implementation of Top-P sampling for token selection. This function applies temperature scaling, softmax, a radix sort, and then samples tokens based on the cumulative probability mass (Top-P).
• ​topk_wrapper: Copy of Kernels/mojo/nn/topk.mojo:_topk_stage1 with the addition of max_vals and p_threshold arguments to determine if sorting is needed for top-p/min-p sampling.
• ​topk_wrapper_no_shmem: Shared-memory-free variant of topk_wrapper for Apple GPUs.
• ​topp_minp_sampling_kernel: Top P-Min P sampling kernel.

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit