Migrating from NCCL.jl 0.1

NCCL.jl 1.0 changes how communicators are created and destroyed, and checks the arguments of operations more strictly. This page lists the changes that can break code written for 0.1.

Dependencies

NCCL.jl now depends on CUDACore, the core of CUDA.jl, instead of on CUDA.jl itself. Code that relied on using NCCL to load CUDA.jl has to load it, or CUDACore, itself:

using CUDA, NCCL       # or: using CUDACore, NCCL

Creating communicators

The UniqueID is now a required positional argument. In 0.1, it defaulted to a new identifier, which differs on every rank, so the default only worked for a single rank:

# 0.1comm = NCCL.Communicator(nranks, rank; unique_id=id)comm = NCCL.Communicator(1, 0)# 1.0comm = NCCL.Communicator(nranks, rank, id)comm = NCCL.Communicator(1, 0, NCCL.UniqueID())

See Getting started for ways to share the identifier between processes, including the new MPI integration.

NCCL.Communicators throws an ArgumentError for an empty list of devices, or when a device occurs more than once.

Destroying communicators

Communicators are no longer destroyed by the garbage collector, as that could deadlock (see Communicator lifetime). Destroy them explicitly, or use the do-block form:

# 0.1comm = NCCL.Communicator(nranks, rank; unique_id=id)# ... (destroyed by the garbage collector at some point)# 1.0comm = NCCL.Communicator(nranks, rank, id)# ...NCCL.destroy(comm)# orNCCL.Communicator(nranks, rank, id) do comm    # ...end

Communicators that one task drives have to be destroyed together:

# 0.1NCCL.destroy.(comms)# 1.0NCCL.destroy(comms)

NCCL.destroy now waits for outstanding operations and synchronizes with the other ranks (ncclCommFinalize) before freeing the communicator. After NCCL.destroy or NCCL.abort, using the communicator throws an InvalidStateException; NCCL.abort used to leave the object pointing to freed memory. Neither function can be called inside NCCL.group.

Renamed and removed functions

# 0.1NCCL.size(comm)# 1.0NCCL.nranks(comm)

NCCL.groupStart() and NCCL.groupEnd() are removed, as code between them was not protected against exceptions or task switches. Use NCCL.group instead:

# 0.1NCCL.groupStart()for (comm, buf) in zip(comms, bufs)    NCCL.Allreduce!(buf, +, comm)endNCCL.groupEnd()# 1.0NCCL.group() do    for (comm, buf) in zip(comms, bufs)        NCCL.Allreduce!(buf, +, comm)    endend

NCCL.default_device_stream(comm) is removed. Operations use the stream it returned when no stream is passed:

# 0.1NCCL.Allreduce!(buf, +, comm; stream=NCCL.default_device_stream(comm))# 1.0NCCL.Allreduce!(buf, +, comm)# or, to get that stream:s = CUDACore.device!(CUDACore.stream, NCCL.device(comm))

NCCL.rank, NCCL.nranks and NCCL.device now return values cached when the communicator was created.

Group semantics

NCCL.group now pins the calling task to its thread, permanently, and NCCL calls from other tasks on the same thread throw a ConcurrencyViolationError while the group is open. See Groups for the rules.

Operations

Operations now validate their arguments before calling NCCL. Code that worked by accident may now throw:

• a DimensionMismatch for buffers whose lengths don't fit the operation, which used to fail an @assert or go unnoticed;
• an ArgumentError for non-contiguous views (which used to send the wrong data), host arrays and other array types, mismatched element types, roots or peers out of range, partially overlapping buffers, streams of another device, and the per-thread default stream.

Reductions with an unsupported operator throw an ArgumentError instead of a MethodError. A raw ncclRedOp_t other than the built-in operators (e.g. created with ncclRedOpCreatePreMulSum) is rejected; use NCCL.PreMulSum instead.

NCCL.Recv! returns the receive buffer, instead of its internal storage:

# 0.1NCCL.Recv!(buf, comm; source)   # returned buf.data# 1.0NCCL.Recv!(buf, comm; source)   # returns buf

Errors

NCCLError now holds NCCL's error message in its msg field (a String, empty when NCCL didn't report a new one), instead of the description of the result code, and is displayed with showerror.

Environment

When NCCL_COMM_BLOCKING is not set, loading NCCL.jl sets it to 1, and creating a communicator throws when it is set to 0 (see Nonblocking communicators). Processes started from Julia inherit the variable.

Low-level bindings

The bindings in NCCL.LibNCCL have been regenerated against NCCL 2.32. Element counts are now Csize_t (they were wrongly Cint), ncclMemAlloc takes a Ptr{CuPtr{Cvoid}}, and all functions are called with @gcsafe_ccall.

New features

These don't break existing code: the Alltoall!, Gather! and Scatter! collectives; communicator configuration with NCCL.Config; NCCL.split; creating communicators from MPI communicators; NCCL.PreMulSum; support for complex numbers, Bool, BFloat16 and other isbits element types; nothing for buffers that only matter on the root; single-buffer in-place forms of Allgather! and ReduceScatter!; and do-block forms of the constructors.