For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
Struct_ep_fused_silu_mxfp4
struct Struct_ep_fused_silu_mxfp4
Registers the ep.fused_silu.mxfp4 graph op with the graph compiler.
Implemented traits
Methods
execute
static def execute[fp4_dtype: DType, scales_dtype: DType, input_dtype: DType, target: StringSlice[ImmStaticOrigin], *, fuse_a_scale_preshuffle: Bool = False, max_padded_M: Int = Int(0), clamp_activation: Bool = False](output: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=output.static_spec], scales: ManagedTensorSlice[IOSpec[_, _].Output, static_spec=scales.static_spec], input: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=input.static_spec], row_offsets: ManagedTensorSlice[IOSpec[_, _].Input, static_spec=row_offsets.static_spec], alpha: Float32, limit: Float32, context: DeviceContext)
Execute the Expert Parallelism fused SILU kernel with MXFP4 quantization.
This function launches the fused_silu_mxfp4 kernel to perform the SILU operation for all the MLPs in the EP MoE module.
This kernel will read the row offsets to determine the actual number of received tokens in the input tensor, and then only perform the SILU operation on the received tokens. Once the SILU operation is performed, the output will be quantized to the MXFP4 format.
When fuse_a_scale_preshuffle=True (KS64 fusion), the kernel
writes the E8M0 scale directly into the grouped matmul's per-expert
fixed-stride scale_4d slot layout (slot stride max_padded_M * K_SCALES), so the standalone preshuffle_grouped_scale_4d_gpu can be
omitted from the critical path. The scales output tensor must then have
shape [n_local_experts * max_padded_M, K_SCALES].
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!