tokio: Cannot use Tokio Semaphore when spawning tasks

0.2.6

[dependencies.tokio]
version="0.2.6"
features=["time", "sync", "rt-threaded"]

Windows/Linux x64

Description

This may harken back to the removal of shutdown_on_idle but attempting to pass a permit to a spawned future results in the semaphore never living long enough. It would be great to be able to pass a permit to a spawned future that can drop the permit and free up other futures to acquire a permit.

As of now, it seems that the semaphore is practically single-threaded. No matter how I wait on a JoinHandle, the program fails. The program succeeds when using block_on instead of spawn.

Code

//! Semaphore test

pub use tokio;
use tokio::sync::{Semaphore, SemaphorePermit};

pub struct TestStruct{
    sema: Semaphore,
}

async fn acquire_permit(cl: &TestStruct) -> SemaphorePermit<'_>{
    cl.sema.acquire().await
}

fn main(){
    let mut rt = tokio::runtime::Builder::new()
        .threaded_scheduler()
        .core_threads(4)
        .thread_name("test")
        .thread_stack_size(3 * 1024 * 1024)
        .enable_time()
        .build()
        .unwrap();
    let client = TestStruct{
        sema: Semaphore::new(1)
    };
    let permit: SemaphorePermit = rt.block_on(async{
        println!("Acquiring 1");
        acquire_permit(&client).await
    });
    let h = rt.spawn(async move{
        drop(permit);
    });
    let h2 = rt.spawn(async{
        println!("Acquiring 2");
        acquire_permit(&client).await;
    });
    rt.block_on(async move {
        h.await;
        h2.await;
    });
}

Expected

The program will complete without error

Actual

The program fails, complaining that the client does not liove long enough.


error[E0597]: `client` does not live long enough
  --> src\main.rs:28:14
   |
26 |       let permit: SemaphorePermit = rt.block_on(async{
   |  ____________________________________________________-
27 | |         println!("Acquiring 1");
28 | |         acq(&client).await
   | |         -----^^^^^^-------
   | |         |    |
   | |         |    borrowed value does not live long enough
   | |         returning this value requires that `client` is borrowed for `'static`
29 | |     });
   | |_____- value captured here by generator
...
41 |   }
   |   - `client` dropped here while still borrowed

error[E0597]: `client` does not live long enough
  --> src\main.rs:35:14
   |
33 |        let h2 = rt.spawn(async{
   |   _______________________-____-
   |  |_______________________|
   | ||
34 | ||         println!("Acquiring 2");
35 | ||         acq(&client).await;
   | ||              ^^^^^^ borrowed value does not live long enough
36 | ||     });
   | ||     -
   | ||_____|
   | |______value captured here by generator
   |        argument requires that `client` is borrowed for `'static`
...
41 |    }
   |    - `client` dropped here while still borrowed

error[E0597]: `client` does not live long enough
  --> src\main.rs:28:14
   |
26 |       let permit: SemaphorePermit = rt.block_on(async{
   |  ____________________________________________________-
27 | |         println!("Acquiring 1");
28 | |         acq(&client).await
   | |         -----^^^^^^-------
   | |         |    |
   | |         |    borrowed value does not live long enough
   | |         returning this value requires that `client` is borrowed for `'static`
29 | |     });
   | |_____- value captured here by generator
...
41 |   }
   |   - `client` dropped here while still borrowed

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0597`.
error[E0597]: `client` does not live long enough
  --> src\main.rs:35:14
   |
33 |        let h2 = rt.spawn(async{
   |   _______________________-____-
   |  |_______________________|
   | ||
34 | ||         println!("Acquiring 2");
35 | ||         acq(&client).await;
   | ||              ^^^^^^ borrowed value does not live long enough
36 | ||     });
   | ||     -
   | ||_____|
   | |______value captured here by generator
   |        argument requires that `client` is borrowed for `'static`
...
41 |    }
   |    - `client` dropped here while still borrowed

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0597`.
error: could not compile `rust-test`.
warning: build failed, waiting for other jobs to finish...
error: could not compile `rust-test`.

To learn more, run the command again with --verbose.

Process finished with exit code 101

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 1
Comments: 15 (10 by maintainers)

Most upvoted comments

You can do something like that by transmuting the reference to the Semaphore to a &'static Semaphore and using acquire.

Darksonn on Sep 21, 2020

Thanks for the detailed issue. You are correct that the SemaphoreGuard cannot be passed into spawns. As of now, there is no provided guard API that is 'static. However, there is a work around you can use:

let semaphore = Arc::new(Semaphore::new(100));

let permit = semaphore.acquire().await;

// Do not release the permit back to the semaphore:
permit.forget();

let s2 = semaphore.clone();

tokio::spawn(async move {
    // return the permit to the semaphore
    s2.add_permits(1);
});

You can implement your own guard that contains Arc<Semaphore> to do this automatically.

I would be greatly appreciative if you could submit a doc PR to tokio that adds the above to the Semaphore API docs.

carllerche on Dec 20, 2019

I would like a friendlier API for this case, but I am not sure what specifically.

Seems like, since Arc<Semaphore> is a valid method receiver, we could have a method where the self type is Arc<Self> that returns an owned guard variant?

hawkw on Apr 20, 2020