Async Runtimes, The Reactor-Executor Pattern & Work-Stealing

Unlike Go or JavaScript, Rust does not bundle a runtime into its standard library. The standard library provides only the Future and Poll interfaces.

To execute asynchronous tasks, Rust applications rely on external runtimes like Tokio. Understanding how runtimes combine Reactors and Executors is critical for building enterprise-grade network systems.

1. The Reactor-Executor Architecture#

An async runtime is divided into two distinct components:

The Division of Labor#

1. The Reactor: Interfaces with kernel event notifications (epoll on Linux, kqueue on macOS, IOCP on Windows). It sleeps until sockets become readable or writable, waking associated tasks.
2. The Executor: Manages a pool of worker threads that continuously poll active futures to completion.

2. Work-Stealing Scheduling Algorithm#

Modern async runtimes use Work-Stealing to distribute load evenly across multi-core processors:

• Global Queue: A shared queue holding newly spawned tasks.
• Local Queues: Each worker thread maintains a private ring-buffer queue (typically capacity 256).

By processing tasks primarily from local thread queues, worker threads minimize cross-core CPU cache-coherency overhead.

3. The Cardinal Sin: Blocking in Async Code#

Async worker threads rely on tasks yielding cooperatively via .await. If you execute a synchronous, blocking operation (such as reading a file with std::fs, doing heavy image compression, or calling thread::sleep), the entire worker thread is frozen!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
// ANTIPATTERN: Freezes an executor worker thread!
async fn bad_handler() {
    // Blocks the worker thread for 5 seconds!
    // All other 1,000 tasks assigned to this worker thread are frozen!
    std::thread::sleep(std::time::Duration::from_secs(5));
}

// IDIOMATIC SOLUTION 1: Use non-blocking async primitives
async fn good_handler_async() {
    tokio::time::sleep(std::time::Duration::from_secs(5)).await;
}

// IDIOMATIC SOLUTION 2: Offload CPU-bound work to blocking pool
async fn good_handler_cpu() {
    let result = tokio::task::spawn_blocking(|| {
        // Runs on a dedicated thread pool designed for blocking tasks:
        compute_heavy_cryptography()
    }).await.unwrap();
}

4. Concept Verification#

Test your understanding of async runtimes and work-stealing schedulers: