Python class

LongRoPEScalingParams

`LongRoPEScalingParams`

class max.nn.LongRoPEScalingParams(short_factor, long_factor, original_max_position, max_position_embeddings)

source

Parameters for LongRoPE scaling as used in Phi-3.5 models.

Parameters:

long_factor: list[float]

source

Scaling factors for long sequences (can be much larger).

max_position_embeddings: int

source

Current max position embeddings after scaling.

original_max_position: int

source

Original max position embeddings the model was trained with.

short_factor: list[float]

source

Scaling factors for short sequences (typically close to 1.0).