For the complete documentation index, see llms.txt. Markdown versions of all pages are available by appending .md to any URL (e.g. /max/get-started.md).

Mojo struct

BF16TokenFormat

struct BF16TokenFormat[output_layout: TensorLayout, //, _hid_dim: Int, _top_k: Int, _alignment: Int = 0]

Fields

output_tokens (BF16TokenFormat[_hid_dim, _top_k, _alignment].TensorType):

Implemented traits

AnyType, Copyable, DevicePassable, ImplicitlyCopyable, ImplicitlyDestructible, Movable, RegisterPassable, TokenFormat, TrivialRegisterPassable

`comptime` members

`alignment`

comptime alignment = _alignment if _alignment.__bool__() else get_device_alignment()

`device_type`

comptime device_type = BF16TokenFormat[_hid_dim, _top_k, _alignment]

`dispatch_smem_size`

comptime dispatch_smem_size = 0

`dispatch_wait_tile_shape`

comptime dispatch_wait_tile_shape = Tuple(128, 1)

`hid_dim`

comptime hid_dim = _hid_dim

`TensorType`

comptime TensorType = TileTensor[DType.bfloat16, output_layout, MutExternalOrigin]

`top_k`

comptime top_k = _top_k

Methods

`init`

__init__(output_tokens: TileTensor[DType.bfloat16, output_layout, address_space=output_tokens.address_space, linear_idx_type=output_tokens.linear_idx_type, element_size=output_tokens.element_size]) -> Self

`get_type_name`

static get_type_name() -> String

Returns:

String

`token_size`

static token_size() -> Int

Returns:

Int

`copy_token_to_send_buf`

static copy_token_to_send_buf[src_type: DType, block_size: Int, buf_addr_space: AddressSpace = AddressSpace.GENERIC](buf_p: UnsafePointer[UInt8, address_space=buf_addr_space], src_p: UnsafePointer[Scalar[src_type], address_space=src_p.address_space], input_scale: Float32)

`copy_msg_to_output_tensor`

copy_msg_to_output_tensor[buf_addr_space: AddressSpace = AddressSpace.GENERIC](self, buf_p: UnsafePointer[UInt8, address_space=buf_addr_space], token_index: Int)

Fields​

Implemented traits​

comptime members​

alignment​

device_type​

dispatch_smem_size​

dispatch_wait_tile_shape​

hid_dim​

TensorType​

top_k​

Methods​

__init__​

get_type_name​

token_size​

copy_token_to_send_buf​

copy_msg_to_output_tensor​