Python module

hf_pipeline

Generalized Token Generation Pipeline

`HFEmbeddingsPipeline`

class max.pipelines.hf_pipeline.HFEmbeddingsPipeline(pipeline_config: PipelineConfig, torch_device_type: str)

Generalized token generator pipeline.

`encode()`

encode(batch: dict[str, max.pipelines.context.context.TextContext]) → dict[str, max.pipelines.interfaces.response.EmbeddingsResponse]

Encodes a batch of text inputs.

`prepare_initial_token_inputs()`

prepare_initial_token_inputs(context_batch: list[max.pipelines.context.context.TextContext]) → tuple[torch.Tensor, torch.Tensor]

`HFTextGenerationPipeline`

class max.pipelines.hf_pipeline.HFTextGenerationPipeline(pipeline_config: PipelineConfig, torch_device_type: str)

HuggingFace text token generator pipeline.

`next_token()`

next_token(batch: dict[str, max.pipelines.context.context.TextContext], num_steps: int) → dict[str, max.pipelines.interfaces.response.TextGenerationResponse]

Provided a batch, process batch inputs, execute the graph for num_steps in a multi-step scenario, then decode the tokens holistically and return the list of decoded tokens.

`release()`

release(context: TextContext) → None

Releases resources associated with this context.

Parameters:

context (TokenGeneratorContext) – Finished context.

HFEmbeddingsPipeline​

encode()​

prepare_initial_token_inputs()​

HFTextGenerationPipeline​

next_token()​

release()​