For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).
Mojo struct
DistributedAllGather
struct DistributedAllGather
Implemented traits
AnyType,
ImplicitlyDestructible
Methods
execute
static def execute[dtype: DType, rank: Int, target: StringSlice[StaticConstantOrigin], _trace_name: StringSlice[StaticConstantOrigin]](outputs: VariadicTensors[Output, static_specs=outputs.static_specs], inputs: VariadicTensors[Input, static_specs=inputs.static_specs], signal_buffers: VariadicTensors[MutableInput, static_specs=signal_buffers.static_specs], dev_ctxs_input: DeviceContextList)
Distributed allgather operation implementation.
Args:
- outputs (
VariadicTensors[Output, static_specs=outputs.static_specs]): Output tensors (one per GPU) to store gathered results. - inputs (
VariadicTensors[Input, static_specs=inputs.static_specs]): Input tensors (one per GPU) containing values to gather. - signal_buffers (
VariadicTensors[MutableInput, static_specs=signal_buffers.static_specs]): Device buffer values used for synchronization. - dev_ctxs_input (
DeviceContextList): Device contexts for participating GPUs.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!