For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Python class

GenerateMixin

`GenerateMixin`

class max.pipelines.GenerateMixin(*args, **kwargs)

source

Bases: Protocol[TextGenerationContextType, RequestType]

Protocol for pipelines that support text generation.

`execute()`

execute(inputs)

source

Executes the pipeline for the given inputs.

Parameters:: inputs (TextGenerationInputs[TextGenerationContextType])
Return type:: dict[RequestID, TextGenerationOutput]

`generate()`

generate(prompts)

source

Generates outputs for the given prompts.

Parameters:: prompts (RequestType | list[RequestType])
Return type:: list[TextGenerationOutput]

`generate_async()`

async generate_async(prompts)

source

Generates outputs asynchronously for the given prompts.

Parameters:: prompts (RequestType | list[RequestType])
Return type:: Any

`kv_manager`

property kv_manager: PagedKVCacheManager

source

Returns the KV cache managers for this pipeline.

`pipeline_config`

property pipeline_config: PipelineConfig

source

Returns the pipeline configuration.

`release()`

release(request_id)

source

Releases resources for the given request.

Parameters:: request_id (RequestID)
Return type:: None

`tokenizer`

property tokenizer: PipelineTokenizer[TextGenerationContextType, ndarray[tuple[Any, ...], dtype[integer[Any]]], RequestType]

source

Returns the tokenizer for this pipeline.

GenerateMixin​

execute()​

generate()​

generate_async()​

kv_manager​

pipeline_config​

release()​

tokenizer​