Skip to main content

/

Docs

Nightly

v25.1

Log in

Python module

hf_pipeline

Generalized Token Generation Pipeline

HFEmbeddingsPipeline​

class max.pipelines.hf_pipeline.HFEmbeddingsPipeline(pipeline_config: PipelineConfig, torch_device_type: str)

Generalized token generator pipeline.

encode()​

encode(batch: dict[str, max.pipelines.context.TextContext]) → dict[str, max.pipelines.response.EmbeddingsResponse]

Encodes a batch of text inputs.

prepare_initial_token_inputs()​

prepare_initial_token_inputs(context_batch: list[max.pipelines.context.TextContext]) → tuple[torch.Tensor, torch.Tensor]

HFTextGenerationPipeline​

class max.pipelines.hf_pipeline.HFTextGenerationPipeline(pipeline_config: PipelineConfig, torch_device_type: str)

HuggingFace text token generator pipeline.

next_token()​

next_token(batch: dict[str, max.pipelines.context.TextContext], num_steps: int) → list[dict[str, max.pipelines.response.TextResponse]]

Provided a batch, process batch inputs, execute the graph for num_steps in a multi-step scenario, then decode the tokens holistically and return the list of decoded tokens.

release()​

release(context: TextContext) → None

Releases resources associated with this context.

  • Parameters:

    context (TokenGeneratorContext) – Finished context.