async-std: Deadlock with recursive task::block_on
When calling task::block_on
recursively, the executor seems to dead-lock even when the recursion depth is much smaller than num cpus.
Sample code (deadlocks on a 8-cpu machine):
#[async_std::test]
async fn test_async_deadlock() {
use std::future::Future;
use futures::FutureExt;
fn nth(n: usize) -> impl Future<Output=usize> + Send {
async move {
async_std::task::block_on(async move {
if n == 0 {
0
} else {
let fut = async_std::task::spawn(nth(n-1)).boxed();
fut.await + 1
}
})
}
}
let input = 2;
assert_eq!(nth(input).await, input);
}
It seems that the test should deadlock when input >= num_cpus, but even when input=2, on a 8-cpu machine this seems to deadlock. Is this expected behaviour? Interestingly, input = 1 (which does involve a recursive call) does not deadlock. If the block_on
is removed, the test indeed passes for large input values.
Aside: it would be great if the executor could detect block_on
called within the pool processor threads, and spawn more threads. The block_on
could be considered an explicit hint that the particular worker is probably going to block.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 1
- Comments: 16
@rmanoka on 1.6.4 it doesn’t panic anymore, but it does deadlock once again with block_on inside spawn, when i have 1 core.
smol on the other hand works perfectly on one cpu with same block_on/spawn/block_on recursion, even with 4 smol threads. perhaps it would lock up on deeper recursion with a bunch of heavy tasks, i dunno.
moreover, smol has an unbounded sync channel with try_send, so you don’t even need to call block_on inside other tasks, because you can just communicate from sync context into async normally.
i’m jumping async_std for smol, personally.
It’s not true anymore. More info: https://github.com/stjepang/smol/issues/177#issuecomment-649035605
Yes, exactly - you can think of
Executor<'a>
as a scope that is allowed to borrow anything with lifetime'a
. Note thatExecutor<'a>
is something you need to “drive” yourself withrun()
ortick()
- it won’t run tasks by itself. There are no threads involved unless you spawn your own threads and then callrun()
on them.This design makes it possible to do lots of interesting things - here’s a scoped executor with task priorities: https://github.com/stjepang/async-executor/blob/master/examples/priority.rs
@stjepang Absolutely! Works in the
new-scheduler
branch.I believe this should work with the new scheduler: https://github.com/async-rs/async-std/pull/631
Can you try running the test again with
async-std
from thenew-scheduler
branch?