tower: breaking change in tower design (0.6 or beyond): first class support for async fn traits

Hi, I would like to open this work to start the discussion and start to make progress towards getting tower ready for when async fn in traits and impl return values for traits are allowed.

As I really needed this kind of first class support I have a completely ported version of tower ready at https://github.com/plabayo/tower-async.

it is meant to unblock me and very opiniated;
it does not reflect my willingness to work with you on making an actual tower version with such support a reality, I am open to any direction or approach you want me. That might be close to the approach I took or something very different
tower-async can be serve as a starting point of discussion. And While it can reflect the design we take, I am also fine for it to be nothing more then a starter for this discussion and be completely ignored after that
Even if we choose to go for a new design, I am more then happy to release a new breaking release of towers (and as many versions as it takes) to try out the ideas. I am more then happy to use that repo as a playground, and I’m actually using it in production, so you would get actual use feedback in my rama project 😃

I took a couple of drastic changes:

first of all the signature changes of Service. But that is obvious as it’s the entire idea of this proposal. You can see that trait at: https://github.com/plabayo/tower-async/blob/6680bc9422083d893c42d5c5a0a28293bf10f281/tower-async-service/src/lib.rs#L196-L209
- besides the async fn support you’ll notice that:
  1. I dropped poll_ready: see the FAQ: https://github.com/plabayo/tower-async#faq, happy to discuss more. Also again, just my current POV, happy to change it if this is not shared with maintainers
  2. I changed from &mut self in &self. This is not a requirement and my first ported version did still have &mut, however:
    - I never had a real need for &mut, and making it & reflects that direction and also gives a simpler life
    - it makes it easier to interface with codebases like hyper (v1);
because of (1) of the previous point I also dropped stuff that rely on poll_ready. e.g. anything related to load balancing and the like. This is on purpose as I didn’t have an eed for it, and I think it’s out of scope. I have ideas on how we can support it by providing such code but making it that users would integrate it in either the MakeService stuff or as utility code that they would inject themselves in their call functions. Or have services that can pool other services etc etc. But again I didn’t have a need for it or desire, so honestly I didn’t push any of those ideas further and just dropped it.

How to play with my experimental tower-async version?

The port of tower and tower-http is completely done and can be used now using the tower-async and tower-async-http crates.

You can use tower-async-bridge crate to interface with classic Tower interfaces (to and from).

You can use tower-async-hyper to interface with the “low level” hyper (v1) library.

All crates are published on crates.io: https://crates.io/search?q=tower-async

How is my experience?

Great. It works and I’m using it in production. I’m also working on rama (https://github.com/plabayo/rama) where I’ll have a similar production-ready setting ready as open source. But that is still early days.

Seems that all the design works just fine and with the current stability plans it means that tower-async and tower-async-http will be ready for stable rust this year.

Open Problems

As my production use proves I have no issues/problems any longer that block me. There are however none the less still open problems.

Boxed Services is non trivial

Open issue: https://github.com/plabayo/tower-async/issues/10

This can be solved and I might add it as an experiment to tower-async. But the only way I see how for now is using stuff like call(): Send, which… requires nightly. Dyn trait objects with async is not in active progress and not even sure this will be something that in the 2024 edition will be.

I need exactly this nightly feature for my current tower-async-bridge and tower-async-hyper crates. This makes me also fearful and unsure if we can really provide stable support for hyper anytime soon by only adding code to hyper-util. Because in order to implement hyper::service::Service we will always require a boxed Future. This is only possible with the call(): Send nightly feature, which is not stable anytime soon.

The only hope is that someone can help me figure a way out to add support for async fn Tower trait in hyper-util by making use of the service_fn approach that hyper allows. If we can internally within hyper-util use that, then we do not need a boxed future and thus also not the call(): Send syntax. I have a feeling however that this is only possible once the next described problem (https://github.com/rust-lang/rust/issues/114142) is resolved.

Trait Resolving is not flawless (e.g. might not deduce the future is `Send`)

Open issue: https://github.com/rust-lang/rust/issues/114142

Example: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=df177519275726a7df18045cf90a59a9

You can come in situations where a higher order function cannot correctly deduce that a future of an async fn trait implementation is Send. A work around for this is turbofish notation to explicitly declare the type that is passed in. This is however not always possible and makes usage also more awkward when you run into this.

So far however I’ve not come into a situation where this work around does not work. If you need it I would suggest to hide it behind a central point as much as possible so that the rest of your code can be as ergonomic as possible.

This problem and its workaround were also documented since 0.1 of tower-async at: https://github.com/plabayo/tower-async#faq.

Can we run everything in Stable?

Yes. However, currently there is no async fn trait support in hyper or hyper-util. So in order to make something like tower-async-hyper work I currently need to be able to define a Future type and that requires me to Box it. This requires me to use the call((): Send notation (however you call it) and this is going to be a nightly only feature for much longer.

The good news is that this shouldn’t be needed for an official async-fn ready tower version as we can then just implement official tower support with async fn trait support directly into hyper-util which would allow us to drop the Boxed Future and thus also the need for Nightly Rust.

Open for feedback

To iterate some of the above.

These are all just proposals though. I am open to any feedback, a totally different direction. I can do more experimentation if you want a different direction (completely or partly). Honestly fully open about it all. I just already needed it as for my purposes (where I do deal with plenty of async fn usage in my middlewares it was becoming a pain in the ass to hand roll these futures myself. If everything in Rust was manual futures it would be a lot easier, and I don’t mind the work. But given that some stuff (e.g. plenty of tokio utility code) works with async fn that became very awkward and painful very soon.

tower-async is not meant to be permanent or me wanting it this way or no way. It was just unblock myself. A realisation made in the 900th iteration of rama. After struggling a lot with classic tower in many iterations before that.

Prior work

@LucioFranco had apparently already added a case study for Tower in the context of the async fn trait RFC.

Link: https://rust-lang.github.io/async-fundamentals-initiative/evaluation/case-studies/tower.html

I didn’t know of its existence until I already have version 0.1 of tower-async. I did know about it prior to implementing version 0.2. As far as I can see it came to very similar conclusions as I have.

It also proposes some potential intermediate solutions, but so far I have not had a need for it.

Next Steps

Most important I think is that the maintainers align on a vision of how they see tower. tower-async can be an inspiration for this, and its design can even be taken as is. But even if none of its designs are taken at all, at least it might hopefully teach some lessons on how to to it different in that case.

Either way, this is is how I see it progress after initial discussions and alignments:

do small experiments with Rust play grounds of core concepts
once experiments and more discussions lead to an agreed upon vision and “core” design, I can work on version 0.3 of tower-async to actually implement these ideas and also test them in production use.

This can be seen as a typical design iteration loop, where we cycle through as much as needed and in any desired order. tower-async can be seen as a playground for “final” (hopeful) designs that we think are worth testing in production.

The final step would then be porting tower-async back into tower and tower-http.

Once that is complete the ecosystem can also start adopting it, which we could kickstart by providing first class support for it in hyper-util. I would obviously myself also start using tower once again (instead of tower-async) into rama. Other maintainers, such as the ones of axum can also help by migrating to this.

This requires no change of hyper and neither breaks any 1.0 promises in such systems, as we can provide bridge functionality for this new tower design by adding it to hyper-util, similar to https://github.com/hyperium/hyper-util/pull/46

Timeline

A proposed timeline would be get a version of tower out of the door with this in the next months. By then Rust has already stable support for all required features.

About this issue

Original URL
State: open
Created 7 months ago
Reactions: 8
Comments: 20 (7 by maintainers)

Most upvoted comments

I took a couple of drastic changes:

first of all the signature changes of Service. But that is obvious as it’s the entire idea of this proposal. You can see that trait at: https://github.com/plabayo/tower-async/blob/6680bc9422083d893c42d5c5a0a28293bf10f281/tower-async-service/src/lib.rs#L196-L209

besides the async fn support you’ll notice that:

I dropped poll_ready: see the FAQ: https://github.com/plabayo/tower-async#faq, happy to discuss more. Also again, just my current POV, happy to change it if this is not shared with maintainers

I changed from &mut self in &self. This is not a requirement and my first ported version did still have &mut, however:

I never had a real need for &mut, and making it & reflects that direction and also gives a simpler life

it makes it easier to interface with codebases like hyper (v1);

because of (1) of the previous point I also dropped stuff that rely on poll_ready. e.g. anything related to load balancing and the like. This is on purpose as I didn’t have an eed for it, and I think it’s out of scope. I have ideas on how we can support it by providing such code but making it that users would integrate it in either the MakeService stuff or as utility code that they would inject themselves in their call functions. Or have services that can pool other services etc etc. But again I didn’t have a need for it or desire, so honestly I didn’t push any of those ideas further and just dropped it.

I think cutting scope for the sake of experimentation makes sense, but I’d like to push back on this a little bit.

Removing poll_ready and changing the Service::call receiver to &selfis a substantial change from the existing design, and one that makes tower substantially less expressive. In many ways, tower’s primary purpose is to provide a shared abstraction in the form of the Service trait that allows a variety of libraries and codebases to interact. I think it’s important to maintain the ability to serve as an integration point, and that means that it’s important for tower’s central abstraction to be able to abstract over as wide a range of functionality as possible.

With the current design, functionality like pooling, routing, and load balancing can all be represented with the Service trait abstraction. A change that makes these things more difficult or impossible to abstract over using tower is kind of a substantial regression in what tower can be used for, and I think we should avoid that. Many users do make use of the code that had to be removed in order to make these changes, and I want to ensure that these users don’t have to give up on tower in the future.

This is not to say that I’m not open to making drastic changes to the Service trait, such as removing poll_ready and/or changing the receiver for call to &self. However, I think that if we’re going to make those changes, we need to have a clear story for how stuff that tower can currently represent will be represented in the future. In order to convince me that major changes to the Service trait like this are a good idea, I would want to see a proposal that includes an implementation of tower::balance or similar existing code with the new API. Proving that patterns that can currently be represented using tower are still possible with a new design would demonstrate that a potential change isn’t just making tower less useful for the projects that currently rely on it.

hawkw on Nov 21, 2023

The fact that there is (currently) no way to introduce bounds like Send, Sync, 'static, or Unpin on the futures generated by async trait functions is a significant limitation of async fns in traits, which, I think, would make a great deal of existing tower code challenging to represent. If users want to tokio::spawn a call() future, for example, they need to require that it’s Send + 'static.

It’s worth noting that the current design, with an associated future type, can be used with unboxed async blocks using #[feature(type_alias_in_trait)], which i would certainly hope will stabilize sooner than call(): Send or similar return type bounding. For example, we can implement a Service with the current Service trait using an unboxed async block:

#![feature(type_alias_in_trait)]

impl Service<Req> for MyService {
    type Response = Response;
    type Error = Error;
    // using the `type-alias-in-trait` feature, we can return an `async` block future
    // without boxing it.
    type Future = impl Future<Output = Result<Self::Response, Self::Error>>;
   
    fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
        // ...
    }

    fn call(&mut self, request: Req) -> Self::Future {
        // we can still use async-await to implement `call`!
         async move { 
             // do stuff...
         }
    }
}

Downstream code can depend on this Service implementation and add S::Future: Send + 'static or similar trait bounds on the associated Future type freely, and we can add Send, 'static, or other traits to the associated type like:

    type Future = impl Future<Output = Result<Self::Response, Self::Error>> + Send + 'static;

This allows code to introduce additional trait bounds on a Service’s call future type freely (such as Send + 'static to make a future spawnable), but doesn’t require tower to add those bounds at the trait level and require them for all implementations of Service. This is something that’s currently not possible at all using async fn in traits…

hawkw on Nov 23, 2023