Async

Dusk’s async line arrived across the 0.4.x releases: futures and the event loop in 0.4.0, the epoll reactor in 0.4.1, and the async func, await, and async_run keyword layer in 0.4.2. TCP networking and an awaitable channel receive followed in 0.4.3, and 0.4.4 made the reactor portable and hardened it against closed peers, interrupted syscalls, and descriptor exhaustion. It rides the same thread pool and monitor machinery the 0.3.x concurrency line built, so the pieces you already know from concurrency are underneath it. This guide walks the async pieces in the order you are likely to reach for them. The precise rules live in the async reference, and the modules themselves are documented under std.async.

One event loop thread

The whole async substrate runs on a single thread, the one that calls loop_init. That thread owns a FIFO ready queue: a task that becomes runnable joins the tail, and the loop runs one task to its next suspension or return before picking up the next. An await always costs exactly one scheduler turn, even against a future that is already complete, and it never resumes inline. So a loop-only program’s interleaving is exact and reproducible: two worker tasks each printing a label and a counter around await tick() produce a0 b0 a1 b1 a2 b2, not a race.

Anything that crosses a pool worker, a spawned thread, or the reactor funnels back through one future completion and one enqueue, so the loop thread’s own ordering is never in question. Only the moment a pool worker or a spawned thread finishes its work is externally timed.

Writing an async func

An async func compiles to a state machine over a heap frame. Calling one writes its arguments into a fresh frame, mints the task’s result Future<T>, and runs nothing until the loop cranks it.

hello_async.dusk

@import std.async.future
@import std.async.loop

async func amain() -> int32 {
    println("in")
    return 7
}

func main() -> int32 {
    le := loop_init()
    le.ignore()
    rc := async_run(amain())
    loop_free()
    println(rc)
    return 0
}

Calling amain() mints a task and a future and does no work. async_run then cranks the loop until that future completes and yields its value, so this prints in and then 7. The keywords async func, await, and async_run need no import and no paradigm; only the loop, future, time, and io modules are ordinary stdlib imports.

An async func’s shape is fixed at one layout, so it takes no type parameters, and neither a parameter nor a return may be a future, a slice, a closure, or an interface value, since each may view a frame the task outlives. A method cannot be async and main cannot be async. The async reference states each rule with its exact message.

await, in four positions

await is a statement, not an operator. It never appears mid-expression, and it is legal in exactly four shapes:

v := await f            // bind the value; the completer's error is discarded
v, e := await f         // bind the value and the completer's pending error
await f                 // void discard, legal only when f's element is void
return await f          // forward the awaited tuple whole to the caller

await only suspends directly inside an async func body, never inside a lambda literal created there (a lambda has no task frame to suspend) and never under defer (a defer runs at completion and cannot suspend). The error word of a two-bind await is a pending error like any other and falls under the ordinary must-handle rule, so inspect it with exists, handle it with check, or discard it with ignore (see Errors).

await composes with every statement shape a value can sit in. Inside a while, an if, a for over a named fixed array, or a match arm reading its payload, each survives the suspension because the loop counter, the array’s pointer, length, and index, and the match payload are all frame slots, reloaded on the resume edge rather than kept in a register the suspension bypassed. Ordinary rules keep applying underneath the keyword too: move(p) into an awaited call still kills the mover’s name, so touching p after v := await consume(move(p)) is use of a moved pointer exactly as it would be with no await in the way.

async_run, the only bridge

async_run is the one bridge from synchronous code into the loop, and it is main’s job. It takes a direct call of an async func, written right at the call site, cranks the loop until that call’s future completes, then hands back the value.

fetch_chain.dusk

@import std.async.future
@import std.async.loop

async func fetch(n: int64) -> int64 {
    return n * 2
}

async func amain() -> int32 {
    a := await fetch(10)
    b := await fetch(20)
    println(a + b)          // 60
    return 0
}

func main() -> int32 {
    le := loop_init()
    le.ignore()
    rc := async_run(amain())
    loop_free()
    return rc
}

async_run cannot be called from inside an async func, since the enclosing task can simply await the call instead. It also refuses a stored future: a future does not carry which async func minted it, so you must pass the call itself, not a Future<T> you saved earlier.

Futures and the event loop by hand

The keyword layer is built on the hand-rolled futures 0.4.0 shipped, and you can still drive them directly. loop_init() -> error in std.async.loop starts the loop on the thread that will consume futures, and loop_free() frees it after the last completer has finished. A freed loop may be initialized again on any thread, which then becomes the owner.

A Future<T> from std.async.future is a one-shot completion slot: minted pending, completed exactly once from any thread, and consumed exactly once by the loop thread.

• future_new() -> Future<T> mints a pending future, the element type pinned by the binding annotation like chan_new, so write f: Future<int64> = future_new().
• complete(f, v, e) -> error stores the value and the error together from any thread and wakes the loop. A second completion is refused with future already completed and its value is dropped.
• await(f) -> (T, error) parks until completion and consumes the future.
• await_timeout(f, ms) -> (T, error) parks at most ms milliseconds against the monotonic clock, then comes back with await timed out, the zero value, and the future still live, the recoverable escape hatch.
• try_poll(f) -> (T, error) never parks, reporting future is pending while unresolved and consuming the future once it is ready.
• future_free(f) releases a future that will never be consumed.

Consuming reads the pair and retires the record, so a future is awaited once the way a thread is joined once, and a second consume faults with use of a dead future.

sleep_async(ms) -> Future<int64> in std.async.time mints a future the loop’s timer heap completes with 0 at its deadline. Timers fire while any await or poll runs, and two timers sharing a deadline complete in creation order.

timer.dusk

@import std.async.future
@import std.async.loop
@import std.async.time

func main() -> int32 {
    le := loop_init()
    le.ignore()
    f := sleep_async(10)
    v, e := await(f)
    e.ignore()
    println(v)          // 0, the timer's completion value
    loop_free()
    return 0
}

An await that provably cannot finish is a deadlock, not a hang. When no timer is pending, no spawned thread is alive, no pool task is in flight, and no readiness watch is armed, nothing in the process can complete the future, and the wait aborts with the event loop is idle but work is still pending. Each gauge drops only after its work finishes, and every drop wakes the loop, so the gate never fires against a completion still in flight.

When the completer runs on another thread, a pool worker or a spawned thread, it never captures the typed Future<T> handle, since a future belongs to the loop thread. It carries the future’s raw words instead and completes through complete_raw. The async reference walks that crossing.

The reactor at a glance

The reactor, added in 0.4.1, is one C thread that turns file descriptor readiness into one-shot readiness futures on the event loop, behind std.async.io. readable(fd) and writable(fd) arm a one-shot watch and return a Future<int64> that completes with the readiness mask (1 readable, 2 writable, 4 hangup, 8 error, ORed together). pipe_new gives you a pipe to exercise it, and read_nb and write_nb move bytes through a caller-staged buffer without ever blocking.

p, pe := pipe_new()          // r and w fields, a blocking pipe
pe.ignore()
ne := fd_nonblock(p.r)       // set non-blocking before arming a watch
ne.ignore()
w := readable(p.r)           // a one-shot Future<int64>, the readiness mask
m, me := await(w)            // 1 readable, 2 writable, 4 hangup, 8 error
me.ignore()

Start the loop, then the reactor, before arming any watch, and stop the reactor before freeing the loop. An armed watch is a possible completer, so it holds off the idle deadlock the same way a live thread does. The full byte surface and its fault family are in std.async.io.

TCP networking

std.async.net, added in 0.4.3, puts TCP straight over the reactor’s readiness watches. A socket is an ordinary file descriptor the reactor already knows how to watch, so networking is a thin library layer with no new event machinery. Three calls are synchronous, tcp_listen, tcp_local_port, and tcp_close, and four are async funcs you await, tcp_accept, tcp_connect, tcp_read, and tcp_write. Each async call tries its non-blocking socket operation, and when the operation would block it awaits readable or writable on the descriptor and retries, so a server that accepts many connections and a client that connects both suspend and resume as tasks under async_run rather than pumping the loop from inside a task.

tcp_write sends every byte, looping over writability until the whole buffer is gone, so a short write never drops the tail. tcp_connect finishes the non-blocking handshake by awaiting writability and then reading the socket error, so a refused connection comes back as a clean error rather than a descriptor that fails on first use. Addresses are literal IPv4 dotted quads; there is no name resolution yet. A listener bound to port 0 gets an ephemeral port you read back with tcp_local_port. Awaiting a net future follows the same rule as any other await: it is legal only inside an async func, and awaiting one from a synchronous function is rejected with 'await' is only legal inside an async func.

Here a server task and a client task share one loop. serve is minted as a future and left in flight before the client runs, so both sit on the event loop at once, interleaved by the reactor’s readiness. Start the loop, then the reactor, before any socket call, and stop the reactor before freeing the loop, the same order the pipe surface follows.

echo.dusk

@import std.async.net
@import std.async.future
@import std.async.loop

async func serve(listen_fd: int64) -> (int64, error) {
    cfd, ae := await tcp_accept(listen_fd)
    if ae.exists() {
        return (1, ae)
    }
    buf: *raw char = alloc_bytes(64)
    n, re := await tcp_read(cfd, buf, 64)
    if re.exists() {
        free(buf)
        cc := tcp_close(cfd)
        cc.ignore()
        return (2, re)
    }
    w, we := await tcp_write(cfd, buf, n)
    we.ignore()
    w.ignore()
    free(buf)
    cc := tcp_close(cfd)
    cc.ignore()
    return (0, error {})
}

async func client(port: int64) -> (int64, error) {
    fd, ce := await tcp_connect("127.0.0.1", port)
    if ce.exists() {
        return (1, ce)
    }
    msg: *raw char = alloc_bytes(8)
    msg[0] = 112
    msg[1] = 105
    msg[2] = 110
    msg[3] = 103
    w, we := await tcp_write(fd, msg, 4)
    we.ignore()
    w.ignore()
    n, re := await tcp_read(fd, msg, 7)
    re.ignore()
    msg[n] = 0
    println(cstr(msg))
    free(msg)
    cc := tcp_close(fd)
    cc.ignore()
    return (0, error {})
}

async func amain() -> int32 {
    lfd, le := tcp_listen(0, 4)
    if le.exists() {
        printerr(le)
        return 1
    }
    port, pe := tcp_local_port(lfd)
    if pe.exists() {
        printerr(pe)
        lc := tcp_close(lfd)
        lc.ignore()
        return 1
    }

    sf := serve(lfd)
    cv, ce := await client(port)
    ce.ignore()
    cv.ignore()
    sv, se := await sf
    se.ignore()
    sv.ignore()

    lc := tcp_close(lfd)
    lc.ignore()
    return 0
}

func main() -> int32 {
    le := loop_init()
    le.ignore()
    se := reactor_start()
    se.ignore()
    rc := async_run(amain())
    reactor_stop()
    loop_free()
    return rc
}

The accept side is the same shape run in a loop: an async func that awaits tcp_accept, handles the connection, and comes back for the next one, all as one task the loop cranks. The full signature list is in std.async.net.

An awaitable channel receive

chan_recv_async(c) returns a Future<T>, also added in 0.4.3, so a channel receive has a home on the event loop. A plain chan_recv on the loop thread would block and stall every other task, so this mints a future and hands the blocking receive to a detached helper thread that completes it off the loop thread; you await the future like any other. A closed and drained channel completes it with receive on a closed, drained channel, the same message the blocking receive uses. See concurrency for the channels themselves.

Run to completion, defers, and errors

A task runs to completion once started; there is no way to cancel one mid-flight. That is exactly what makes a defer sound across a suspension: a resource acquired before an await and deferred for release is guaranteed to see that release, in reverse registration order, exactly once, whenever the task actually returns, never at a suspension partway through.

There is no rejection channel. A completer hands its value and its error through together, and the awaited tuple destructures through the same must-handle machinery every other fallible result uses. await is monadic bind performed by the compiler: it sequences a suspending computation, threads its result into the frame that continues, and forwards its error alongside the value rather than short-circuiting through an exception.

Where to go next

• Async reference: the precise signature rules, the four await positions with their exact messages, async_run, the frame and state machine model, the fault family, and the cost table.
• std.async: the API surface for future, loop, time, and io, plus the async keyword forms.
• Concurrency: the threads, channels, and thread pool the async line schedules onto.
• Errors: the must-handle rule every (T, error) return here follows.