IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /max/get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Python module

max.pipelines.architectures.internvl

InternVL vision-language architecture for multimodal text generation.

InternVLConfig​

class max.pipelines.architectures.internvl.InternVLConfig(*, devices, downsample_ratio, num_image_token, vision_config, llm_config)

source

Bases: ArchConfigWithKVCache

Configuration for InternVL models.

Parameters:

calculate_max_seq_len()​

static calculate_max_seq_len(pipeline_config, huggingface_config)

source

Calculate maximum sequence length for InternVL.

Parameters:

Return type:

int

construct_kv_params()​

static construct_kv_params(huggingface_config, pipeline_config, devices, kv_cache_config, cache_dtype)

source

Parameters:

Return type:

KVCacheParams

devices​

devices: list[DeviceRef]

source

Devices that the InternVL model is parallelized over.

downsample_ratio​

downsample_ratio: float

source

Downsample ratio for vision features.

finalize()​

finalize(huggingface_config, llm_state_dict, vision_state_dict, dtype, return_logits, norm_method='rms_norm')

source

Finalize the InternVLConfig instance with state_dict dependent fields.

Parameters:

  • huggingface_config (AutoConfig) – HuggingFace model configuration.
  • llm_state_dict (dict[str, WeightData]) – Language model weights dictionary.
  • vision_state_dict (dict[str, WeightData]) – Vision encoder weights dictionary.
  • dtype (DType) – Data type for model parameters.
  • return_logits (ReturnLogits) – Return logits configuration.
  • norm_method (Literal['rms_norm', 'layer_norm']) – Normalization method.

Return type:

None

get_kv_params()​

get_kv_params()

source

Returns the KV cache parameters from the embedded LLM config.

Return type:

KVCacheParams

get_max_seq_len()​

get_max_seq_len()

source

Returns the maximum sequence length from the embedded LLM config.

Return type:

int

get_num_layers()​

static get_num_layers(huggingface_config)

source

Parameters:

huggingface_config (AutoConfig)

Return type:

int

initialize()​

classmethod initialize(pipeline_config, model_config=None)

source

Initializes an InternVLConfig instance from pipeline configuration.

Parameters:

Returns:

An InternVLConfig instance with fields initialized from config.

Return type:

Self

initialize_from_config()​

classmethod initialize_from_config(pipeline_config, huggingface_config)

source

Initializes an InternVLConfig from pipeline and HuggingFace configs.

This method creates a config instance with all fields that can be determined from the pipeline and HuggingFace configurations, without needing the state_dict. Fields that depend on the state_dict should be set via the finalize() method.

Parameters:

  • pipeline_config (PipelineConfig) – The MAX Engine pipeline configuration.
  • huggingface_config (AutoConfig) – HuggingFace model configuration.

Returns:

An InternVLConfig instance ready for finalization.

Return type:

Self

llm_config​

llm_config: Llama3Config | Qwen3Config

source

Language model configuration (Qwen2 or Qwen3).

num_image_token​

num_image_token: int

source

Number of image tokens per patch.

vision_config​

vision_config: VisionConfig

source

Vision encoder configuration.

InternVLInputs​

class max.pipelines.architectures.internvl.InternVLInputs(tokens, input_row_offsets, signal_buffers, return_n_logits, pixel_values=None, image_token_indices=None, *, kv_cache_inputs=None, lora=None, hidden_states=None)

source

Bases: ModelInputs

A class representing inputs for the InternVL model.

Parameters:

has_vision_inputs​

property has_vision_inputs: bool

source

Check if this input contains vision data.

image_token_indices​

image_token_indices: list[Buffer] | None = None

source

Per-device pre-computed indices of image tokens in the input sequence.

input_row_offsets​

input_row_offsets: list[Buffer]

source

Per-device tensors containing the offsets for each row in the ragged input sequence.

pixel_values​

pixel_values: list[Buffer] | None = None

source

Pixel values for vision inputs.

return_n_logits​

return_n_logits: Buffer

source

Number of logits to return, used by speculative decoding for example.

signal_buffers​

signal_buffers: list[Buffer]

source

Device buffers used for synchronization in communication collectives.

tokens​

tokens: Buffer

source

Tensor containing the input token IDs.

InternVLModel​

class max.pipelines.architectures.internvl.InternVLModel(pipeline_config, session, devices, kv_cache_config, weights, adapter=None, return_logits=ReturnLogits.LAST_TOKEN)

source

Bases: AlwaysSignalBuffersMixin, PipelineModelWithKVCache[TextAndVisionContext]

An InternVL pipeline model for multimodal text generation.

Parameters:

calculate_max_seq_len()​

classmethod calculate_max_seq_len(pipeline_config, huggingface_config)

source

Uses max_length when set, else llm_config.max_position_embeddings (config bounds).

Parameters:

Return type:

int

execute()​

execute(model_inputs)

source

Executes the InternVL model with the prepared inputs.

Parameters:

model_inputs (ModelInputs)

Return type:

ModelOutputs

language_model​

language_model: Model

source

The compiled language model for text generation.

load_model()​

load_model(session)

source

Loads the compiled InternVL models into the MAX Engine session.

Returns:

A tuple of (vision_model, language_model).

Parameters:

session (InferenceSession)

Return type:

tuple[Model, Model]

model_config_cls​

model_config_cls

source

alias of InternVLConfig

prepare_initial_token_inputs()​

prepare_initial_token_inputs(replica_batches, kv_cache_inputs=None, return_n_logits=1)

source

Prepares the initial inputs for the first execution pass of the InternVL model.

Parameters:

Return type:

ModelInputs

vision_model​

vision_model: Model

source

The compiled vision model for processing images.

VisionConfig​

class max.pipelines.architectures.internvl.VisionConfig(hidden_size, intermediate_size, norm_type, image_size, patch_size, num_attention_heads, head_dim, layer_norm_eps, qk_normalization, qkv_bias, num_hidden_layers, dtype=bfloat16, o_proj_bias=False)

source

Bases: object

Base configuration for InternVL models with required fields.

Parameters:

  • hidden_size (int)
  • intermediate_size (int)
  • norm_type (Literal['rms_norm', 'layer_norm'])
  • image_size (int)
  • patch_size (int)
  • num_attention_heads (int)
  • head_dim (int)
  • layer_norm_eps (float)
  • qk_normalization (bool)
  • qkv_bias (bool)
  • num_hidden_layers (int)
  • dtype (DType)
  • o_proj_bias (bool)

dtype​

dtype: DType = 80

source

DType of the InternVL vision model weights.

finalize()​

finalize(dtype, state_dict)

source

Finalize VisionConfig with state_dict dependent fields.

Parameters:

Return type:

None

head_dim​

head_dim: int

source

Dimension of each attention head.

hidden_size​

hidden_size: int

source

Hidden size of the vision encoder.

image_size​

image_size: int

source

Input image size.

initialize_from_config()​

classmethod initialize_from_config(hf_vision_config)

source

Initialize VisionConfig from HuggingFace vision config.

Note: dtype and o_proj_bias fields will be set to defaults and should be updated via finalize() once state_dict is available.

Parameters:

hf_vision_config (AutoConfig)

Return type:

VisionConfig

intermediate_size​

intermediate_size: int

source

Intermediate size in the vision encoder’s feed-forward layers.

layer_norm_eps​

layer_norm_eps: float

source

Epsilon for layer normalization.

norm_type​

norm_type: Literal['rms_norm', 'layer_norm']

source

Type of normalization used in the vision encoder.

num_attention_heads​

num_attention_heads: int

source

Number of attention heads in the vision encoder.

num_hidden_layers​

num_hidden_layers: int

source

Number of hidden layers in the vision encoder.

o_proj_bias​

o_proj_bias: bool = False

source

Whether to use bias in the out projection.

patch_size​

patch_size: int

source

Vision transformer patch size.

qk_normalization​

qk_normalization: bool

source

Whether to use QK normalization in attention.

qkv_bias​

qkv_bias: bool

source

False.

Type:

Whether to use bias in the QKV projection. Default

Models

Cookbook

Puzzles

Blog

Community

Contact