Mojo module

normalization

Functions

• ​block_reduce:
• ​group_norm:
• ​group_norm_gpu:
• ​group_norm_gpu_block:
• ​group_norm_gpu_warp_tiling:
• ​group_norm_reshape: Reshapes an input buffer for group normalization by flattening all dimensions except the group dimension. Returns a 2D buffer of shape (num_groups * N, group_size), where group_size is the product of channels_per_group and spatial.
• ​group_norm_shape:
• ​layer_norm:
• ​layer_norm_cpu: Computes layernorm(elementwise_fn(x)) across the last dimension of x, where layernorm is defined as $(x-mean(x))/(sqrt(var(x)+eps)*gamma_fn + beta$.
• ​layer_norm_gpu:
• ​layer_norm_gpu_block:
• ​layer_norm_gpu_warp_tiling:
• ​layer_norm_reshape:
• ​layer_norm_shape: Compute the output shape of a layer_norm operation.
• ​rms_norm:
• ​rms_norm_cpu:
• ​rms_norm_fused_residual_add:
• ​rms_norm_fused_residual_add_cpu:
• ​rms_norm_fused_residual_add_gpu:
• ​rms_norm_fused_residual_add_gpu_block:
• ​rms_norm_fused_residual_add_gpu_warp_tiling:
• ​rms_norm_gpu:
• ​rms_norm_gpu_block:
• ​rms_norm_gpu_warp_tiling:
• ​rms_norm_gpu_warp_tiling_128:
• ​rms_norm_shape:
• ​welford_block_all_reduce:
• ​welford_combine:
• ​welford_update:
• ​welford_warp_all_reduce:
• ​welford_warp_reduce:

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit