Python class

GenerateMixin

`GenerateMixin`

class max.pipelines.GenerateMixin(*args, **kwargs)

source

Bases: Protocol[TextGenerationContextType, RequestType]

Protocol for pipelines that support text generation.

execute(inputs)

source

Executes the pipeline for the given inputs.

Parameters:: inputs (TextGenerationInputs[TextGenerationContextType])
Return type:: dict[RequestID, TextGenerationOutput]

generate(prompts)

source

Generates outputs for the given prompts.

async generate_async(prompts)

source

Generates outputs asynchronously for the given prompts.

property kv_manager: PagedKVCacheManager

source

Returns the KV cache managers for this pipeline.

property pipeline_config: PipelineConfig

source

Returns the pipeline configuration.

release(request_id)

source

Releases resources for the given request.

property tokenizer: PipelineTokenizer[TextGenerationContextType, ndarray[tuple[Any, ...], dtype[integer[Any]]], RequestType]

source

Returns the tokenizer for this pipeline.