For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Python module
max.pipelines.architectures.eagle3_deepseekV3
DeepseekV3 + Eagle3 speculator pipeline.
Eagle3DeepseekV3
class max.pipelines.architectures.eagle3_deepseekV3.Eagle3DeepseekV3(config)
Bases: Eagle3MLADraft
Eagle3 draft paired with a DeepseekV3 target.
See Eagle3MLADraft for the implementation.
-
Parameters:
-
config (DeepseekV3Config)
Eagle3DeepseekV3Inputs
class max.pipelines.architectures.eagle3_deepseekV3.Eagle3DeepseekV3Inputs(tokens, input_row_offsets, signal_buffers, host_input_row_offsets, batch_context_lengths, *, kv_cache_inputs=None, lora=None, vision_embeddings=<factory>, vision_scatter_indices=<factory>, hidden_states=None, return_n_logits, data_parallel_splits, ep_inputs=(), draft_tokens=None, seed=None, temperature=None, top_k=None, max_k=None, top_p=None, min_top_p=None, in_thinking_phase=None, pinned_bitmask=None, wait_payload=None, device_bitmask_scratch=None, structured_output=False)
Bases: UnifiedSpecDecodeInputs, DeepseekV3Inputs
Inputs for the Eagle3 + DeepseekV3 unified model.
Target-prefix fields come from DeepseekV3Inputs; the spec-decode
fields and trailing buffer packing come from
UnifiedSpecDecodeInputs. The eagle3_deepseekV3 graph does not bind
in_thinking_phase.
-
Parameters:
-
- tokens (Buffer)
- input_row_offsets (Buffer)
- signal_buffers (list[Buffer])
- host_input_row_offsets (Buffer)
- batch_context_lengths (list[Buffer])
- kv_cache_inputs (KVCacheInputsInterface[Buffer, Buffer] | None)
- lora (LoRAInputs | None)
- vision_embeddings (list[Buffer])
- vision_scatter_indices (list[Buffer])
- hidden_states (Buffer | list[Buffer] | None)
- return_n_logits (Buffer)
- data_parallel_splits (Buffer)
- ep_inputs (tuple[Buffer, ...])
- draft_tokens (Buffer | None)
- seed (Buffer | None)
- temperature (Buffer | None)
- top_k (Buffer | None)
- max_k (Buffer | None)
- top_p (Buffer | None)
- min_top_p (Buffer | None)
- in_thinking_phase (Buffer | None)
- pinned_bitmask (Buffer | None)
- wait_payload (Buffer | None)
- device_bitmask_scratch (Buffer | None)
- structured_output (bool)
buffers
Returns positional Buffer inputs for model ABI calls.
Eagle3DeepseekV3Model
class max.pipelines.architectures.eagle3_deepseekV3.Eagle3DeepseekV3Model(pipeline_config, session, devices, kv_cache_config, weights, adapter=None, return_logits=ReturnLogits.ALL, return_hidden_states=ReturnHiddenStates.NONE, max_batch_size=1)
Bases: _UnifiedSpecDecodeModelMixin, DeepseekV3Model
Eagle3 + DeepseekV3: target + draft in one compiled graph.
Loads target weights from a DeepseekV3-shaped main checkpoint and draft
weights from a separate Eagle3 checkpoint
(pipeline_config.draft_model).
-
Parameters:
-
- pipeline_config (PipelineConfig)
- session (InferenceSession)
- devices (list[Device])
- kv_cache_config (KVCacheConfig)
- weights (Weights)
- adapter (WeightsAdapter | None)
- return_logits (ReturnLogits)
- return_hidden_states (ReturnHiddenStates)
- max_batch_size (int)
batch_processor_cls
batch_processor_cls
alias of Eagle3DeepseekV3BatchProcessor
Eagle3DeepseekV3Unified
class max.pipelines.architectures.eagle3_deepseekV3.Eagle3DeepseekV3Unified(config, draft_config=None, speculative_config=None, enable_structured_output=False)
Bases: Module
Fused nn.Module: merge + target forward + greedy rejection + shift.
The target model returns concatenated hidden states from 3 intermediate
layers (first, middle, last). The draft model fuses these via fc and
generates the next speculative token.
-
Parameters:
-
- config (DeepseekV3Config)
- draft_config (DeepseekV3Config | None)
- speculative_config (SpeculativeConfig | None)
- enable_structured_output (bool)
input_types()
input_types(kv_params)
Input types for the Eagle3 unified graph.
Distributed (DP + signals + EP) MLA-draft graph that appends the
structured-output bitmask triple when enabled. See
build_spec_decode_input_types() for the canonical ordering.
-
Parameters:
-
kv_params (MultiKVCacheParams)
-
Return type:
-
tuple[TensorType | BufferType, …]
Eagle3MHADeepseekV3Inputs
class max.pipelines.architectures.eagle3_deepseekV3.Eagle3MHADeepseekV3Inputs(tokens, input_row_offsets, signal_buffers, host_input_row_offsets, batch_context_lengths, *, kv_cache_inputs=None, lora=None, vision_embeddings=<factory>, vision_scatter_indices=<factory>, hidden_states=None, return_n_logits, data_parallel_splits, ep_inputs=(), draft_tokens=None, seed=None, temperature=None, top_k=None, max_k=None, top_p=None, min_top_p=None, in_thinking_phase=None, pinned_bitmask=None, wait_payload=None, device_bitmask_scratch=None, structured_output=False)
Bases: UnifiedSpecDecodeInputs, DeepseekV3Inputs
Inputs for the Eagle3 MHA-draft + DeepseekV3 unified model.
The draft owns its own MHA KVCacheInputs (separate from the target’s
MLA cache) as the "draft" leaf of the unified {"target", "draft"}
KV tree, so MHA dispatch metadata is plumbed at both prefill
(q_max_seq_len = 1 + num_speculative_tokens) and decode
(q_max_seq_len = 1) widths. The spec-decode fields and trailing buffer
packing come from UnifiedSpecDecodeInputs; the graph binds the
per-row in_thinking_phase flag and the structured-output bitmask
triple.
-
Parameters:
-
- tokens (Buffer)
- input_row_offsets (Buffer)
- signal_buffers (list[Buffer])
- host_input_row_offsets (Buffer)
- batch_context_lengths (list[Buffer])
- kv_cache_inputs (KVCacheInputsInterface[Buffer, Buffer] | None)
- lora (LoRAInputs | None)
- vision_embeddings (list[Buffer])
- vision_scatter_indices (list[Buffer])
- hidden_states (Buffer | list[Buffer] | None)
- return_n_logits (Buffer)
- data_parallel_splits (Buffer)
- ep_inputs (tuple[Buffer, ...])
- draft_tokens (Buffer | None)
- seed (Buffer | None)
- temperature (Buffer | None)
- top_k (Buffer | None)
- max_k (Buffer | None)
- top_p (Buffer | None)
- min_top_p (Buffer | None)
- in_thinking_phase (Buffer | None)
- pinned_bitmask (Buffer | None)
- wait_payload (Buffer | None)
- device_bitmask_scratch (Buffer | None)
- structured_output (bool)
buffers
Returns positional Buffer inputs for model ABI calls.
Eagle3MHADeepseekV3Model
class max.pipelines.architectures.eagle3_deepseekV3.Eagle3MHADeepseekV3Model(pipeline_config, session, devices, kv_cache_config, weights, adapter=None, return_logits=ReturnLogits.ALL, return_hidden_states=ReturnHiddenStates.NONE, max_batch_size=1)
Bases: _UnifiedSpecDecodeModelMixin, DeepseekV3Model
Eagle3 MHA-draft + DeepseekV3: target + draft in one compiled graph.
-
Parameters:
-
- pipeline_config (PipelineConfig)
- session (InferenceSession)
- devices (list[Device])
- kv_cache_config (KVCacheConfig)
- weights (Weights)
- adapter (WeightsAdapter | None)
- return_logits (ReturnLogits)
- return_hidden_states (ReturnHiddenStates)
- max_batch_size (int)
batch_processor_cls
batch_processor_cls
alias of Eagle3MHADeepseekV3BatchProcessor
convert_eagle3_draft_state_dict()
max.pipelines.architectures.eagle3_deepseekV3.convert_eagle3_draft_state_dict(state_dict, huggingface_config=None, pipeline_config=None)
Convert an Eagle3 draft checkpoint to Eagle3DeepseekV3 module keys.
The Eagle3 checkpoint (nvidia/Kimi-K2.5-Thinking-Eagle3) has keys:
fc.*, layers.0.*, norm.*, lm_head.*.
All weights are loaded; norm and lm_head are kept independent
from the target model. Only embed_tokens is shared from the target.
-
Parameters:
-
- state_dict (dict[str, Weights]) – Raw Eagle3 checkpoint.
- huggingface_config (PreTrainedConfig | None)
- pipeline_config (PipelineConfig | None)
-
Returns:
-
State dict with keys matching
Eagle3DeepseekV3module hierarchy. -
Return type:
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!