cuTile.jl

Tile-based GPU programming in Julia.

cuTile.jl compiles Julia functions to Tile IR, NVIDIA's portable tile-based instruction set. Rather than writing kernels that describe what a single GPU thread does, you write kernels that operate on tiles — multi-dimensional array fragments — and leave it to the Tile IR compiler to map those onto the hardware, including tensor cores and the tensor memory accelerator.

Quick start

A vector addition kernel:

julia> function vadd(a, b, c, tile_size::Int)
           pid = ct.bid(1)
           tile_a = ct.load(a; index=pid, shape=(tile_size,))
           tile_b = ct.load(b; index=pid, shape=(tile_size,))
           ct.store(c; index=pid, tile=tile_a + tile_b)
           return
       end;

julia> vector_size = 2^20;

julia> tile_size = 16;

julia> blocks = cld(vector_size, tile_size);

julia> grid = (blocks, 1, 1);

julia> a, b = CUDA.rand(Float32, vector_size), CUDA.rand(Float32, vector_size);

julia> c = CUDA.zeros(Float32, vector_size);

julia> @cuda backend=cuTile blocks=grid vadd(a, b, c, ct.Constant(tile_size));

julia> @assert c == a .+ b

Kernels are ordinary Julia functions: no decorator or macro is needed, though they must return nothing. They take array arguments, use ct.load and ct.store to move data between global memory and tiles, and operate on those tiles with standard Julia syntax: +, sum, reshape, broadcasting, and so on.

cuTile.jl builds on CUDA.jl, which owns device selection, streams, CuArray allocation and the rest of the Julia CUDA environment. This manual focuses on the tile programming model and refers to CUDA.jl's documentation for that shared functionality.

Where to go next

• Installation — driver and hardware requirements.
• Vector Addition — the above kernel, explained line by line.
• Matrix Multiplication — a GEMM built up in four steps.
• Programming Model — grids, blocks, arrays and tiles.
• Writing Kernels — kernel definition, launching, control flow.
• Comparison with cuTile Python — if you are porting from cuda.tile, start here.
• Debugging — inspecting generated Tile IR.

The Tile IR specification is the authoritative reference for the underlying model; these docs cross-reference it rather than restate it.

Acknowledgments

cuTile.jl is inspired by cuTile-Python, licensed under Apache 2.0 by NVIDIA Corporation & Affiliates.