starlette: Background tasks don't work with middleware that subclasses `BaseHTTPMiddleware`

Any updates on this issue ?

Hi all, I had a PR to fix this behavior (#1017 ) but it relied on forcing the response to remain unevaluated, which further confused those users who were relying on having a fully-evaluated response inside their middleware functions (#1022 ). In other words, there are contradictory goals here.

After looking at this BaseHTTPMiddleware class over the last week, I have come to the following conclusions:

• We should discourage the use of BaseHTTPMiddleware in favor of raw ASGI middleware which includes a method def __call__(self, scope, receive, send):, and

• If people still want to use this middleware class, it should never be used with StreamingResponse/FileResponse endpoints.

Unfortunately, while the BaseHTTPMiddleware class offers an easier interface because it provides a request argument and promises something like a response, this middleware class also encourages users to think about the asgi app functionality in a “complete” or “finished” way. This means this class will either load the entirety of streaming requests into memory (#1012 ) and run the background before returning the response (this issue), or if we fix those problems, that it will then encourage users to leave resources in a pending or open state, an arguably worse result. In short, it’s problematic.

Again, these problems should be absent if you avoid subclassing BaseHTTPMiddleware.

Lastly, I think it’s worth leaving this issue open so that other users who have the same problem can see it and see that it remains an open issue.

@ariefprabowo You can use asyncio.create_task(coro)

If anyone finds this and is trying to add headers to responses this is how i ended up doing it using the information from the above discussions.

from starlette.types import ASGIApp, Message, Receive, Scope, Send
from starlette.datastructures import MutableHeaders

# Custom class required to add headers in-order to not block background tasks
# https://github.com/encode/starlette/issues/919


class SecurityHeadersMiddleware:
    def __init__(
        self,
        app: ASGIApp,
    ) -> None:
        self.app = app

    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
        async def send_wrapper(message: Message) -> None:
            if message["type"] == "http.response.start":
                headers = MutableHeaders(scope=message)
                headers[
                    "Strict-Transport-Security"
                ] = "max-age=63072000; includeSubDomains; preload"
            await send(message)

        await self.app(scope, receive, send_wrapper)

I got this bug after updating fastapi 0.67.0 -> 0.70.0

This code worked on my project with few midllewares, but after migrating to newer version, “im alive” is not printing in my logs

async def task():
    logger.info("im a task")
    await sleep(1)
    logger.info("im alive")


@app.get("/")
async def root(background_tasks: BackgroundTasks):
    background_tasks.add_task(task)
    return {"status": "ok"}

It happens even i use only one dummy middleware:

class DM(BaseHTTPMiddleware):
    async def dispatch(self, request, call_next):
        return await call_next(request)

After removing all middlewares this code works well as expected

I tried to reproduce this on new pure project, but i could not.

Finally found a workaround by exploiting run_in_executor:

from asyncio import get_event_loop
from concurrent.futures import ThreadPoolExecutor
from functools import partial

def run_sync_code(task, *args, **kwargs):
    executor = ThreadPoolExecutor()
    loop = get_event_loop()
    loop.run_in_executor(executor, partial(task, *args, **kwargs))

Then in your favorite FastAPI endpoint:

@app.post("/debug/background_task", tags=["debug"])
async def __debug_background_task():
    def sleep_task(*args, **kwargs):
        print("start sleep....", args, kwargs)
        time.sleep(5)
        print("end sleep....")

    run_sync_code(sleep_task, 1, 2, a="a", b="b")

i have same issue and if i add

await asyncio.sleep(1) inside any method that is triggered by task, it is working fine. I don’t understand how it can work but it is working. even i put time.sleep(30) after asyncio.sleep, i get result immediately but process finish 30sec later which is correct.

Yes, it is a broader issue with background tasks but that also implies that something should be done differently with how concurrency is being managed in the app. @retnikt see this comment for further discussion.

IMO this isn’t really (just) an issue with BaseHTTPMiddleware, rather with how background tasks sure implemented; this should be fixed by making them either:

• detached from a Response by using asyncio.create_task or similar, so they actually run in the background
• or, renamed to something more appropriate to reflect what they’re actually for (i.e. for cleaning up after a request is handled)

I propose that this issue be closed - as far as I can tell, the issues reported here are stale, invalid, or duplicates of other issues:

• The original issue reported by @retnikt seems to be an issue regarding the way that background tasks interfere with the processing of new requests.
  • Despite my best efforts, I have been unable to reproduce the issue, even after installing starlette==0.13.2+uvicorn==0.11.4;
  • In Starlette 0.15.0, the adoption of anyio has completely changed the behavior of BaseHTTPMiddleware regarding that kind of concurrency and cancellation issues, meaning that I suspect that the issue is stale;
• @erewok issue from July 2020, like the previous report, may also be stale, as it dates from before Starlette 0.15.0 (released in June 2021);
• @jacopofar’s issue with GZipMiddleware cannot have anything to do with BaseHTTPMiddleware as GZipMiddleware is not a subclass of BaseHTTPMiddleware - maybe there is another issue being reported there?
• @zoola969’s issue looks to me like a cancellation issue, similar to #1438;
• @maksimt’s issue is clearly a duplicate of #1438;

Here is how I think we should proceed on this issue:

• If you have issues around cancellation when a BaseHTTPMiddleware is in use, then you may want to check #1438 (and my proposed fix in #1715);
• If you have other issues with current versions of Starlette, please open new issues;

I discovered same issue when trying to upgrade my app from fastapi 0.68 to 0.70. Interesting fact is that background tasks work fine on my workstation Ubuntu 20.04 but fail on server with CentOS 7 (same code, python 3.9 & modules versions).

It looks that best solution is to deprecate BaseHTTPMiddleware like mentioned in https://github.com/encode/starlette/issues/1678

Yes, I can reproduce it with the latest version 0.70.0.

If I enable the Gzip middleware:

from fastapi.middleware.gzip import GZipMiddleware
app.add_middleware(GZipMiddleware, minimum_size=1000)

then running a BackgroundTasks blocks the responses until the background task finishes. If I remove the middleware the background task works as expected and HTTP responses are returned immediately without waiting for it

Passing BackgroundTask to TemplateResponse for async functions has been broken for me the last few releases, so I’ve been resorting to simply using: asyncio.create_task(email(...))

Perhaps tests/test_background.py should add asyncio.sleep() to emulate work being done if this is breaking?

@tomchristie Any insight on this issue?

I’ve found a way how to start long computations and do not block API, check my answer.

@Kruszylo When working with async you need to use asyncio.sleep instead of time.sleep because otherwise it will block the whole process

even await asyncio.sleep(0.1) would do the job. the key here is await switching asyncio context.

Shame for me: never thought about using asyncio.create_task.

But I’m still wondering: running the same testcase under hypercorn solves this issue, but why? Is it a hypercorn bug?

What versions of Python, Starlette, and Uvicorn are you using? I’m not able to recreate with Python 3.8.2, Starlette 0.13.3, and Uvicorn 0.11.5.