async-std: Deadlock with recursive task::block_on

When calling task::block_on recursively, the executor seems to dead-lock even when the recursion depth is much smaller than num cpus.

Sample code (deadlocks on a 8-cpu machine):

#[async_std::test]
async fn test_async_deadlock() {
    use std::future::Future;
    use futures::FutureExt;
    fn nth(n: usize) -> impl Future<Output=usize> + Send {
        async move {
            async_std::task::block_on(async move {

                if n == 0 {
                    0
                } else {
                    let fut = async_std::task::spawn(nth(n-1)).boxed();
                    fut.await + 1
                }

            })
        }
    }
    let input = 2;
    assert_eq!(nth(input).await, input);
}

It seems that the test should deadlock when input >= num_cpus, but even when input=2, on a 8-cpu machine this seems to deadlock. Is this expected behaviour? Interestingly, input = 1 (which does involve a recursive call) does not deadlock. If the block_on is removed, the test indeed passes for large input values.

Aside: it would be great if the executor could detect block_on called within the pool processor threads, and spawn more threads. The block_on could be considered an explicit hint that the particular worker is probably going to block.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 1
  • Comments: 16

Most upvoted comments

@rmanoka on 1.6.4 it doesn’t panic anymore, but it does deadlock once again with block_on inside spawn, when i have 1 core.

smol on the other hand works perfectly on one cpu with same block_on/spawn/block_on recursion, even with 4 smol threads. perhaps it would lock up on deeper recursion with a bunch of heavy tasks, i dunno.

moreover, smol has an unbounded sync channel with try_send, so you don’t even need to call block_on inside other tasks, because you can just communicate from sync context into async normally.

i’m jumping async_std for smol, personally.

Is this still true?

It’s not true anymore. More info: https://github.com/stjepang/smol/issues/177#issuecomment-649035605

Yes, exactly - you can think of Executor<'a> as a scope that is allowed to borrow anything with lifetime 'a. Note that Executor<'a> is something you need to “drive” yourself with run() or tick() - it won’t run tasks by itself. There are no threads involved unless you spawn your own threads and then call run() on them.

This design makes it possible to do lots of interesting things - here’s a scoped executor with task priorities: https://github.com/stjepang/async-executor/blob/master/examples/priority.rs

@stjepang Absolutely! Works in the new-scheduler branch.

I believe this should work with the new scheduler: https://github.com/async-rs/async-std/pull/631

Can you try running the test again with async-std from the new-scheduler branch?