Why is tokio multi threaded runtime so much slower than current thread?

[bennetthardwick]

I was working on a project that downloads large files with reqwest when I noticed that the multi threaded runtime was downloading files significantly slower than the current thread runtime. I understand that there is going to be some extra overhead with multiple threads - but how significant a drop in performance would be reasonable?

I've written some simple benchmarks and put the code up here: https://github.com/bennetthardwick/reqwest-benches.

With a local file server that serves a 16 GiB/s I get the following speeds:

bench	speed
reqwest_current_thread	3276 MiB/s
reqwest_multi_thread	680 MiB/s
go	2804 MiB/s

The code looks like this with the only difference between the two being flavor = "current_thread" set for one.

#[tokio::main]
async fn main() {
    println!("Starting reqwest_multi_thread...");

    let mut stream = reqwest::get("http://localhost:3000")
        .await
        .unwrap()
        .bytes_stream();

    let mut byte_count: usize = 0;
    let start = Instant::now();

    while let Some(next) = stream.next().await {
        let bytes = next.unwrap();
        byte_count += bytes.len();
        black_box(bytes);
    }

    // server will return 16GiB of ones
    assert_eq!(byte_count, 16 * 1024 * 1024 * 1024);

    println!(
        "reqwest_multi_thread ran at an average of {} MiB/s",
        (byte_count as f64) / 1024. / 1024. / start.elapsed().as_secs() as f64
    );
}

Is this kind of slowdown to be expected or am I doing something wrong somewhere? I would appreciate any help.

[Darksonn]

The main thread is not a worker thread when using the multi-thread runtime. Wrap the body in tokio::spawn to actually execute the code on a worker thread.

[barafael]

is this documented somewhere? Why does it make such a large performance difference?

[mox692]

If you spawn a task on a non-worker thread, the task is first added to the global queue. Then, one of the worker threads tries to pop the task from the global queue and store it in the local queue, which introduces extra overhead compared to spawning the task directly on a worker thread.

is this documented somewhere?

Probably not, as far as I know.