benchmark

Module

Implements the Benchmark class for runtime benchmarking.

You can import these APIs from the benchmark package. For example:

from benchmark import Benchmark

Benchmark

A benchmark harness.

The class allows to benchmark a given function (passed as a parameter) and configure various benchmarking parameters, such as number of warmup iterations, maximum number of iterations, minimum and maximum elapsed time.

Fields:

  • num_warmup (Int): The number of warmup iterations to perform before the main benchmark loop.
  • max_iters (Int): The maximum number of iterations to perform during the main benchmark loop.
  • min_time_ns (Int): The minimum time (in ns) to spend within the main benchmark loop.
  • max_time_ns (Int): The maximum time (in ns) to spend within the main benchmark loop.

Functions:

__init__

__init__(inout self: Self, num_warmup: Int, max_iters: Int, min_time_ns: Int, max_time_ns: Int)

Constructs a new benchmark object.

Given a function the benchmark object will benchmark it until min_time_ns has elapsed and either max_time_ns OR max_iters is hit.

Args:

  • num_warmup (Int): Number of warmup iterations to run before starting benchmarking (default 2).
  • max_iters (Int): Max number of iterations to run (default 100_000).
  • min_time_ns (Int): Upper bound on benchmarking time in ns (default 500ms).
  • max_time_ns (Int): Lower bound on benchmarking time in ns (default 1s).

__copyinit__

__copyinit__(inout self: Self, existing: Self)

__moveinit__

__moveinit__(inout self: Self, owned existing: Self)

run

run[func: fn() capturing -> None](self: Self) -> Int

Benchmarks the given function.

Benchmarking continues until min_time_ns has elapsed and either max_time_ns or max_iters is achieved.

Parameters:

  • func (fn() capturing -> None): The function to benchmark.

Returns:

Average execution time of func in ns.

clobber_memory

clobber_memory()

Forces all pending memory writes to be flushed to memory.

This ensures that the compiler does not optimize away memory writes if it deems them to be not neccessary. In effect, this operation acts a barrier to memory reads and writes.

keep

keep(val: Bool)

Provides a hint to the compiler to not optimize the variable use away.

This is useful in benchmarking to avoid the compiler not deleting the code to be benchmarked because the variable is not used in a side-effecting manner.

Args:

  • val (Bool): The value to not optimize away.

keep(val: Int)

Provides a hint to the compiler to not optimize the variable use away.

This is useful in benchmarking to avoid the compiler not deleting the code to be benchmarked because the variable is not used in a side-effecting manner.

Args:

  • val (Int): The value to not optimize away.

keep[type: DType, simd_width: Int](val: SIMD[type, simd_width])

Provides a hint to the compiler to not optimize the variable use away.

This is useful in benchmarking to avoid the compiler not deleting the code to be benchmarked because the variable is not used in a side-effecting manner.

Parameters:

  • type (DType): The dtype of the input and output SIMD vector.
  • simd_width (Int): The width of the input and output SIMD vector.

Args:

  • val (SIMD[type, simd_width]): The value to not optimize away.

keep[type: DType](val: DTypePointer[type])

Provides a hint to the compiler to not optimize the variable use away.

This is useful in benchmarking to avoid the compiler not deleting the code to be benchmarked because the variable is not used in a side-effecting manner.

Parameters:

  • type (DType): The type of the input.

Args:

  • val (DTypePointer[type]): The value to not optimize away.

keep[type: AnyType](val: Pointer[*"type"])

Provides a hint to the compiler to not optimize the variable use away.

This is useful in benchmarking to avoid the compiler not deleting the code to be benchmarked because the variable is not used in a side-effecting manner.

Parameters:

  • type (AnyType): The type of the input.

Args:

  • val (Pointer[*"type"]): The value to not optimize away.

keep[type: AnyType](inout *val: "type")

Provides a hint to the compiler to not optimize the variable use away.

This is useful in benchmarking to avoid the compiler not deleting the code to be benchmarked because the variable is not used in a side-effecting manner.

Parameters:

  • type (AnyType): The type of the input.

Args:

  • val (*"type"): The value to not optimize away.