For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
Struct_ep_fused_silu
struct Struct_ep_fused_silu
Implemented traits
AnyType,
ImplicitlyDestructible
Methods
execute
static def execute[output_dtype: DType, input_dtype: DType, //, target: StringSlice[StaticConstantOrigin]](output: ManagedTensorSlice[Output, static_spec=output.static_spec], input: ManagedTensorSlice[Input, static_spec=input.static_spec], row_offsets: ManagedTensorSlice[Input, static_spec=row_offsets.static_spec], context: DeviceContext)
Execute the Expert Parallelism fused SILU kernel.
This function launches the fused_silu kernel to perform the SILU operation for all the MLPs in the EP MoE module. We need to manually implement the custom operation here is because after the EP dispatch phase, the actual number of received tokens is not known to the host.
This kernel will read the row offsets to determine the actual number of received tokens in the input tensor, and then only perform the SILU operation on the received tokens.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!