IMPORTANT: To view this page as Markdown, append `.md` to the URL (e.g. /get-started.md). For the complete documentation index, see llms.txt.
Skip to main content
For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /get-started.md).

Mojo struct

HostBuffer

struct HostBuffer[dtype: DType]

Represents a block of host-resident storage. For GPU devices, a host buffer is allocated in the host's global memory.

To allocate a HostBuffer, use one of the methods provided by DeviceContext, such as enqueue_create_host_buffer().

Parameters​

  • ​dtype (DType): Data type to be stored in the buffer.

Implemented traits​

AnyType, Copyable, Deinitable, ImplicitlyCopyable, Movable, Sized, Writable

Methods​

__init__​

def __init__(out self, *, copy: Self)

Creates a copy of an existing host buffer by incrementing its reference count.

This copy constructor creates a new reference to the same underlying host buffer by incrementing the reference count of the native buffer object. Both the original and the copy will refer to the same memory on the device.

Args:

  • ​copy (Self): The host buffer to copy.

__deinit__​

def __deinit__(deinit self)

Releases resources associated with this host buffer.

This function schedules an owned buffer free using the stream in the device context. The actual deallocation may occur asynchronously after all operations using this buffer have completed.

__getitem__​

def __getitem__(self, idx: Int) -> Scalar[dtype]

Retrieves the element at the specified index from the host buffer.

This operator allows direct access to individual elements in the host buffer using array indexing syntax.

Args:

  • ​idx (Int): The index of the element to retrieve.

Returns:

Scalar[dtype]: The scalar value at the specified index.

__setitem__​

def __setitem__(self, idx: Int, val: Scalar[dtype])

Sets the element at the specified index in the host buffer.

This operator allows direct modification of individual elements in the host buffer using array indexing syntax.

Args:

  • ​idx (Int): The index of the element to modify.
  • ​val (Scalar[dtype]): The new value to store at the specified index.

__len__​

def __len__(self) -> Int

Returns the number of elements in this buffer.

This method calculates the number of elements by dividing the total byte size of the buffer by the size of each element.

Returns:

Int: The number of elements in the buffer.

create_sub_buffer​

def create_sub_buffer[view_type: DType](self, offset: Int, size: Int) -> HostBuffer[view_type]

Creates a sub-buffer view of this buffer with a different element dtype.

This method creates a new buffer that references a subset of the memory in this buffer, potentially with a different element dtype. The sub-buffer shares the underlying memory with the original buffer.

Parameters:

  • ​view_type (DType): The data type for elements in the new sub-buffer.

Args:

  • ​offset (Int): The starting offset in elements from the beginning of this buffer.
  • ​size (Int): The number of elements in the new sub-buffer.

Returns:

HostBuffer[view_type]: A new HostBuffer referencing the specified region with the specified element dtype.

Raises:

If the operation fails.

enqueue_copy_to​

def enqueue_copy_to(self, dst: Self)

Enqueues an asynchronous copy from this buffer to another host buffer.

This method schedules a memory copy operation from this buffer to the destination buffer. The operation is asynchronous and will be executed in the stream associated with this buffer's context.

Args:

  • ​dst (Self): The destination host buffer to copy data to.

Raises:

If the operation fails.

def enqueue_copy_to(self, dst: DeviceBuffer[dtype])

Enqueues an asynchronous copy from this buffer to a device buffer.

This method schedules a memory copy operation from this buffer to the destination buffer. The operation is asynchronous and will be executed in the stream associated with this buffer's context.

Args:

Raises:

If the operation fails.

def enqueue_copy_to(self, dst_ptr: Pointer[Scalar[dtype]])

Enqueues an asynchronous copy from this buffer to host memory.

This method schedules a memory copy operation from this device buffer to the specified host memory location. The operation is asynchronous and will be executed in the stream associated with this buffer's context.

Args:

Raises:

If the operation fails.

def enqueue_copy_to(self, dst: Span[Scalar[dtype]])

Enqueues an asynchronous copy from this buffer to a Span.

This method schedules a memory copy operation from this buffer to the destination span. The operation is asynchronous and will be executed in the stream associated with this buffer's context. The span must contain at least as many elements as this buffer; this invariant is checked via debug_assert.

Args:

  • ​dst (Span[Scalar[dtype]]): The destination span to copy data to. Must have at least as many elements as this buffer.

Raises:

If the operation fails.

enqueue_copy_from​

def enqueue_copy_from(self, src: Self)

Enqueues an asynchronous copy to this buffer from another host buffer.

This method schedules a memory copy operation to this buffer from the source buffer. The operation is asynchronous and will be executed in the stream associated with this buffer's context.

Args:

  • ​src (Self): The source host buffer to copy data from.

Raises:

If the operation fails.

def enqueue_copy_from(self, src: DeviceBuffer[dtype])

Enqueues an asynchronous copy to this buffer from a device buffer.

This method schedules a memory copy operation to this buffer from the source buffer. The operation is asynchronous and will be executed in the stream associated with this buffer's context.

Args:

Raises:

If the operation fails.

def enqueue_copy_from(self, src_ptr: Pointer[Scalar[dtype]])

Enqueues an asynchronous copy to this buffer from host memory.

This method schedules a memory copy operation to this device buffer from the specified host memory location. The operation is asynchronous and will be executed in the stream associated with this buffer's context.

Args:

Raises:

If the operation fails.

def enqueue_copy_from(self, src: Span[Scalar[dtype]])

Enqueues an asynchronous copy to this buffer from a Span.

This method schedules a memory copy operation to this buffer from the source span. The operation is asynchronous and will be executed in the stream associated with this buffer's context. The span must contain at least as many elements as this buffer; this invariant is checked via debug_assert.

Args:

  • ​src (Span[Scalar[dtype]]): The source span to copy data from. Must have at least as many elements as this buffer.

Raises:

If the operation fails.

enqueue_fill​

def enqueue_fill(self, val: Scalar[dtype])

Enqueues an operation to fill this buffer with a specified value.

This method schedules a memory set operation that fills the entire buffer with the specified value. The operation is asynchronous and will be executed in the stream associated with this buffer's context.

Args:

Raises:

If the operation fails.

reassign_ownership_to​

def reassign_ownership_to(self, ctx: DeviceContext)

Transfers ownership of this buffer to another device context.

This method changes the device context that owns this buffer. This can be useful when sharing buffers between different contexts or when migrating workloads between devices.

Args:

  • ​ctx (DeviceContext): The new device context to take ownership of this buffer.

Raises:

If the operation fails.

take_ptr​

def take_ptr(deinit self) -> Pointer[Scalar[dtype], MutUntrackedOrigin]

Takes ownership of the device pointer from this buffer.

This method releases the device pointer from the buffer's control and returns it to the caller. After this call, the buffer no longer owns the pointer, and the caller is responsible for managing its lifecycle.

Returns:

Pointer[Scalar[dtype], MutUntrackedOrigin]: The raw device pointer that was owned by this buffer.

unsafe_ptr​

def unsafe_ptr(self) -> Pointer[Scalar[dtype], MutUntrackedOrigin]

Returns the raw device pointer without transferring ownership.

This method provides direct access to the underlying device pointer for advanced use cases. The buffer retains ownership of the pointer.

Returns:

Pointer[Scalar[dtype], MutUntrackedOrigin]: The raw device pointer owned by this buffer.

context​

def context(self) -> DeviceContext

Returns the device context associated with this buffer.

This method retrieves the device context that owns this buffer and is responsible for managing its lifecycle and operations.

Returns:

DeviceContext: The device context associated with this buffer.

Raises:

If the operation fails.

write_to​

def write_to(self, mut writer: T)

Writes a string representation of this buffer to the provided writer.

This method formats the buffer's contents as a string and writes it to the specified writer. For large buffers, a compact representation is used.

Args:

  • ​writer (T): The writer to output the formatted string to.

as_span​

def as_span[origin: Origin[mut=origin.mut], //](ref[origin.mut] self) -> Span[Scalar[dtype], origin_of(origin["buffer"])]

Returns a Span pointing to the underlying memory of the HostBuffer.

Parameters:

Returns:

Span[Scalar[dtype], origin_of(origin["buffer"])]: A Span over the buffer's memory. The span carries an interior origin derived from self, so any subsequent mutation of the HostBuffer invalidates it at compile time.

Models

Cookbook

Puzzles

Blog

Community

Contact