For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
RMSNormResidualAdd
struct RMSNormResidualAdd
Fused single-norm residual-add + RMSNorm.
Computes intermediate = input + residual_input and
output = rms_norm(intermediate, gamma, epsilon, weight_offset) in a
single launch, returning both output and intermediate. This is the
canonical transformer/mamba pre-norm boundary rms_norm(residual + out)
where the pre-add value is carried forward as the next block's residual.
Reuses the state_space fused-residual kernel; numerically identical to
the unfused mo.add + mo.reduce.rms_norm pair.
Implemented traits
Methods
execute
static def execute[dtype: DType, rank: Int, target: StringSlice[ImmStaticOrigin], multiply_before_cast: Bool = True](output: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=output.static_spec], residual_output: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=residual_output.static_spec], input: ManagedTensorSlice[IOSpec[_, _].FusedInput, static_spec=input.static_spec], residual_input: ManagedTensorSlice[IOSpec[_, _].FusedInput, static_spec=residual_input.static_spec], gamma: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=gamma.static_spec], epsilon: Float32, weight_offset: Scalar[dtype], ctx: DeviceContext)
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!