Python class

RejectionSampler

RejectionSampler

class max.nn.RejectionSampler(device, top_k=1, top_p=1, temperature=1.0, seed=0, eps=1e-05)

source

Bases: Module

Rejection sampler for speculative decoding verification.

Accepts a draft token when the draft logit for that token does not exceed the target logit by more than eps. Returns (first_rejected_idx, sampled_target_token) - a single recovered token at the first rejected position.

Parameters:

• device (DeviceRef)
• top_k (int)
• top_p (float)
• temperature (float)
• seed (int)
• eps (float)

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit