For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo function

fused_silu_kernel

fused_silu_kernel[output_dtype: DType, input_dtype: DType, output_layout: TensorLayout, input_layout: TensorLayout, row_offsets_layout: TensorLayout, num_threads: Int, num_sms: Int](output_tensor: TileTensor[output_dtype, output_layout, MutExternalOrigin], input_tensor: TileTensor[input_dtype, input_layout, ImmutExternalOrigin], row_offsets: TileTensor[DType.uint32, row_offsets_layout, ImmutExternalOrigin])

This kernel performs the SILU operation for all the MLPs in the EP MoE module. We need to manually implement the kernel here is because after the EP dispatch phase, the actual number of received tokens is not known to the host. This kernel will read the row offsets to determine the actual number of received tokens in the input tensor.

Arguments: output_tensor: The output tensor to store the result. input_tensor: The input tensor to perform the SILU operation. row_offsets: The row offsets to determine the actual number of received tokens.

Was this page helpful?

Thank you! We'll create more content like this.

Thank you for helping us improve!

😔 What went wrong?

Some code doesn’t work

It includes inaccurate information

It's missing information I need

It was difficult to understand

Other

Submit