Skip to main content

/

Docs

Nightly

v24.6

Log in

Python module

hf_pipeline

Generalized Token Generation Pipeline

HFTextGenerationPipeline​

class max.pipelines.hf_pipeline.HFTextGenerationPipeline(pipeline_config: PipelineConfig, torch_device_type: str)

HuggingFace text token generator pipeline.

next_token()​

next_token(batch: dict[str, max.pipelines.context.TextContext], num_steps: int = 1) → list[dict[str, max.pipelines.response.TextResponse]]

Provided a batch, process batch inputs, execute the graph for num_steps in a multi-step scenario, then decode the tokens holistically and return the list of decoded tokens.

release()​

release(context: TextContext) → None

Releases resources associated with this context.

  • Parameters:

    context (TokenGeneratorContext) – Finished context.