Atomics

atomic_cas(array::TileArray, index, expected, desired; memory_order, memory_scope) -> T

Atomic compare-and-swap. Atomically compares the value at index with expected, and if equal, replaces it with desired. Returns the original value. Index is 1-indexed.

Example

# Spin-lock acquisition
while ct.atomic_cas(locks, idx, Int32(0), Int32(1); memory_order=ct.MemoryOrder.Acquire) == Int32(1)
    # spin
end

atomic_xchg(array::TileArray, index, val; memory_order, memory_scope) -> T

Atomic exchange. Atomically replaces the value at index with val and returns the original value. Index is 1-indexed.

Example

# Spin-lock release
ct.atomic_xchg(locks, idx, Int32(0); memory_order=ct.MemoryOrder.Release)

atomic_add(array::TileArray, index, val; memory_order, memory_scope) -> T

Atomic addition. Atomically adds val to the value at index and returns the original value. Index is 1-indexed. BFloat16 requires Tile IR bytecode ≥ 13.3 and Hopper (sm_90) or newer.

Example

old_val = ct.atomic_add(counters, idx, Int32(1))

atomic_max(array::TileArray, index, val; memory_order, memory_scope) -> T

Atomic maximum. Atomically replaces the value at index with max(old, val) and returns the original value. Index is 1-indexed. The comparison is signed for Signed element types and unsigned for Unsigned ones.

atomic_min(array::TileArray, index, val; memory_order, memory_scope) -> T

Atomic minimum. Atomically replaces the value at index with min(old, val) and returns the original value. Index is 1-indexed. The comparison is signed for Signed element types and unsigned for Unsigned ones.

atomic_and(array::TileArray, index, val; memory_order, memory_scope) -> T

Atomic bitwise AND. Atomically replaces the value at index with old & val and returns the original value. Index is 1-indexed. val must already have the array's element type; bitwise atomics do not convert implicitly.

atomic_or(array::TileArray, index, val; memory_order, memory_scope) -> T

Atomic bitwise OR. Atomically replaces the value at index with old | val and returns the original value. Index is 1-indexed. val must already have the array's element type; bitwise atomics do not convert implicitly.

atomic_xor(array::TileArray, index, val; memory_order, memory_scope) -> T

Atomic bitwise XOR. Atomically replaces the value at index with old ⊻ val and returns the original value. Index is 1-indexed. val must already have the array's element type; bitwise atomics do not convert implicitly.

atomic_store_add(dst, index, update) -> Nothing

Reduce update into a tile of dst without returning its previous value. dst may be a TileArray or a TiledView from eachtile. Updates broadcast to the tile shape. The operation uses relaxed, device-wide ordering.

Addition supports Int32, Int64, UInt32, UInt64, Float16, BFloat16, Float32 and Float64.

Also available: atomic_store_max, atomic_store_min, atomic_store_or, atomic_store_and, and atomic_store_xor, which support the four integer types only. Bitwise updates must have the destination element type.

Requires Tile IR bytecode ≥ 13.3. BFloat16 addition requires Hopper (sm_90) or newer.

atomic_store_max(dst, index, update) -> Nothing

Reduce update into a tile of dst with max, without returning the previous value. See atomic_store_add for the shared semantics and requirements.

atomic_store_min(dst, index, update) -> Nothing

Reduce update into a tile of dst with min, without returning the previous value. See atomic_store_add for the shared semantics and requirements.

atomic_store_and(dst, index, update) -> Nothing

Reduce update into a tile of dst with and, without returning the previous value. See atomic_store_add for the shared semantics and requirements.

atomic_store_or(dst, index, update) -> Nothing

Reduce update into a tile of dst with or, without returning the previous value. See atomic_store_add for the shared semantics and requirements.

atomic_store_xor(dst, index, update) -> Nothing

Reduce update into a tile of dst with xor, without returning the previous value. See atomic_store_add for the shared semantics and requirements.

@atomic [order] expr

Atomic reductions over a TileArray, TiledView, or gather target.

Statement forms return nothing:

@atomic A[i] += v          # also -=, &=, |=, ⊻=
@atomic A[i] = max(A[i], v)  # op ∈ + - max min & | ⊻; RHS must reference A[i]

Value forms follow Base and return old => new:

pair = @atomic A[i] + v        # pair.first == old, pair.second == new

An optional leading ordering symbol maps to Tile IR memory orderings: :monotonic maps to Relaxed; :acquire, :release, and :acquire_release map directly. :sequentially_consistent is rejected (no Tile IR equivalent).

Statement forms default to :monotonic; value forms default to :acquire_release. Stronger statement orderings use atomic_* and discard the result.

Memory ordering constants. Atomic operations accept Relaxed, Acquire, Release, and AcqRel. Weak is reserved for non-atomic loads and stores and is rejected by the atomic APIs.

Memory scope for atomic operations.