Python module

max.pipelines.architectures.qwen3_embedding_modulev3

Qwen3 architecture for embeddings generation.

Qwen3EmbeddingConfig

class max.pipelines.architectures.qwen3_embedding_modulev3.Qwen3EmbeddingConfig(*, pipeline_config)

source

Bases: ArchConfig

Qwen3 embedding model configuration.

Parameters:

pipeline_config (PipelineConfig)

calculate_max_seq_len()

static calculate_max_seq_len(pipeline_config, huggingface_config)

source

Parameters:

• pipeline_config (PipelineConfig)
• huggingface_config (AutoConfig)

Return type:

int

get_max_seq_len()

get_max_seq_len()

source

Returns the default maximum sequence length for the model.

Subclasses should determine whether this value can be overridden by setting the --max-length (pipeline_config.model.max_length) flag.

Return type:

int

initialize()

classmethod initialize(pipeline_config, model_config=None)

source

Initialize the config from a PipelineConfig.

Parameters:

• pipeline_config (PipelineConfig) – The pipeline configuration.
• model_config (MAXModelConfig | None) – The model configuration to read from. When None (the default), pipeline_config.model is used. Pass an explicit config (e.g. pipeline_config.draft_model) to initialize the arch config for a different model.

Return type:

Self

pipeline_config

pipeline_config: PipelineConfig

source

Qwen3EmbeddingInputs

class max.pipelines.architectures.qwen3_embedding_modulev3.Qwen3EmbeddingInputs(tokens, input_row_offsets, return_n_logits, *, kv_cache_inputs=None, lora_ids=None, lora_ranks=None, hidden_states=None)

source

Bases: ModelInputs

Input structure for Qwen3 embedding models.

Parameters:

• tokens (Buffer)
• input_row_offsets (Buffer)
• return_n_logits (Buffer)
• kv_cache_inputs (KVCacheInputs[Buffer, Buffer] | None)
• lora_ids (Buffer | None)
• lora_ranks (Buffer | None)
• hidden_states (Buffer | list[Buffer] | None)

input_row_offsets

input_row_offsets: Buffer

source

Row offsets for ragged tensors [batch_size + 1]

return_n_logits

return_n_logits: Buffer

source

Number of logits to return (kept for interface compatibility)

tokens

tokens: Buffer

source

Input token IDs [total_seq_len]

Qwen3EmbeddingModel

class max.pipelines.architectures.qwen3_embedding_modulev3.Qwen3EmbeddingModel(pipeline_config, session, devices, kv_cache_config, weights, adapter=None, return_logits=ReturnLogits.ALL)

source

Bases: PipelineModel[TextContext]

Qwen3 embedding pipeline model without KV caching (V3 eager API).

Optimized for embedding generation with:

• No KV cache overhead
• Single-pass forward computation
• Flash attention without cache operations
• Last token pooling with L2 normalization

Parameters:

• pipeline_config (PipelineConfig)
• session (InferenceSession)
• devices (list[Device])
• kv_cache_config (KVCacheConfig)
• weights (Weights)
• adapter (WeightsAdapter | None)
• return_logits (ReturnLogits)

calculate_max_seq_len()

static calculate_max_seq_len(pipeline_config, huggingface_config)

source

Calculates the optimal max sequence length for the model.

Models are expected to implement this method. The following example shows how to implement it for a Mistral model:

class MistralModel(PipelineModel):
    @classmethod
    def calculate_max_seq_len(cls, pipeline_config, huggingface_config) -> int:
        try:
            return upper_bounded_default(
                upper_bound=huggingface_config.max_seq_len,
                default=pipeline_config.model.max_length,
            )
        except ValueError as e:
            raise ValueError(
                "Unable to infer max_length for Mistral, the provided "
                f"max_length ({pipeline_config.model.max_length}) exceeds the "
                f"model's max_seq_len ({huggingface_config.max_seq_len})."
            ) from e

Parameters:

• pipeline_config (PipelineConfig) – Configuration for the pipeline.
• huggingface_config (AutoConfig) – Hugging Face model configuration.

Returns:

The maximum sequence length to use.

Return type:

int

execute()

execute(model_inputs)

source

Execute the model.

Parameters:

model_inputs (ModelInputs)

Return type:

ModelOutputs

load_model()

load_model()

source

Build and compile the embedding model using V3 eager API.

Return type:

Callable[[…], Any]

model

model: Callable[..., Any]

source

Compiled model callable.

prepare_initial_token_inputs()

prepare_initial_token_inputs(replica_batches, kv_cache_inputs=None, return_n_logits=1)

source

Prepares the initial inputs to be passed to execute().

The inputs and functionality can vary per model. For example, model inputs could include encoded tensors, unique IDs per tensor when using a KV cache manager, and kv_cache_inputs (or None if the model does not use KV cache). This method typically batches encoded tensors, claims a KV cache slot if needed, and returns the inputs and caches.

Parameters:

• replica_batches (Sequence[Sequence[TextContext]])
• kv_cache_inputs (KVCacheInputs[Buffer, Buffer] | None)
• return_n_logits (int)

Return type:

Qwen3EmbeddingInputs

prepare_next_token_inputs()

prepare_next_token_inputs(next_tokens, prev_model_inputs)

source

Prepares the secondary inputs to be passed to execute().

While prepare_initial_token_inputs is responsible for managing the initial inputs. This function is responsible for updating the inputs, for each step in a multi-step execution pattern.

Parameters:

• next_tokens (Buffer)
• prev_model_inputs (ModelInputs)

Return type:

Qwen3EmbeddingInputs

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit