For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).
Mojo struct
Struct_fused_token_sampling
struct Struct_fused_token_sampling
Registers the sampler.fused_token_sampling graph op with the graph compiler.
Implemented traits
Methods
execute
static def execute[dtype: DType, rank: Int, out_idx_type: DType, target: StringSlice[ImmStaticOrigin], _trace_name: StringSlice[ImmStaticOrigin]](out_idxs: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=out_idxs.static_spec], K: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=K.static_spec], max_k: Scalar, temperature: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=temperature.static_spec], top_p: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=top_p.static_spec], min_top_p: Float32, min_p: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=min_p.static_spec], seed: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=seed.static_spec], input: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=input.static_spec], ctx: DeviceContext)