For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo function

enqueue_apple_fp8_gemv

def enqueue_apple_fp8_gemv[c_type: DType = DType.float32, elementwise_lambda_fn: Optional[def[dtype: DType, width: SIMDLength, *, alignment: Int = Int(1)](IndexList[Int(2)], SIMD[dtype, width]) capturing thin -> None] = None](c: TileTensor[c_type, Storage=c.Storage, address_space=c.address_space, linear_idx_type=c.linear_idx_type], a: TileTensor[DType.bfloat16, Storage=a.Storage, address_space=a.address_space, linear_idx_type=a.linear_idx_type], weight: TileTensor[DType.float8_e4m3fn, Storage=weight.Storage, address_space=weight.address_space, linear_idx_type=weight.linear_idx_type], n: Int, k: Int, ctx: DeviceContext)

Enqueue the M=1 W8A16 GEMV: out = a @ W_fp8^T (raw, unscaled).

One warp per output column N (ceil(N*WARP_SIZE / block_dim) threadgroups of block_dim threads). a is the bf16 activation [1, K], weight the FP8-E4M3 weight [N, K]. Assumes M == 1; the caller (enqueue_apple_fp8_matmul) gates the M == 1 route. Any K is correct (the kernel's width-1 tail covers K % TILE_K).

Parameters:

c_type (DType): Output element type (fp16, bf16, fp32). Accumulation is fp32.
elementwise_lambda_fn (Optional[def[dtype: DType, width: SIMDLength, *, alignment: Int = Int(1)](IndexList[Int(2)], SIMD[dtype, width]) capturing thin -> None]): Optional fused epilogue (AMD's (row, col) contract), applied on the width-1 store.