httpx: PoolTimeout when num tasks in asyncio.gather() exceeds client max_connections

Have confirmed that the given example now works in httpx 0.21 (Fixed due to the substantial reworking in the latest httpcore.)

I have verified that https://github.com/encode/httpcore/pull/880 resolves this issue.

Using server example at: https://github.com/encode/httpx/issues/1171#issuecomment-1850923234 And client example at: https://github.com/encode/httpx/issues/1171#issuecomment-1864841152

Until this is resolved, is there any reasonable way to work around this? Maybe we use our own asyncio.Semaphore like this:

async def main() -> None:
    url = "https://www.example.com"
    max_connections = 2
    timeout = httpx.Timeout(5.0, pool=2.0)
    limits = httpx.Limits(max_connections=max_connections)
    client = httpx.AsyncClient(timeout=timeout, pool_limits=limits)
    semaphore = asyncio.Semaphore(max_connections)
    
    async def aw_task(aw):
        async with semaphore:
            return await aw

    async with client:
        tasks = []
        for _ in range(max_connections + 1):
            tasks.append(aw_task(client.get(url)))
        await asyncio.gather(*tasks)

I’m planning at getting stuck into this one pretty soon yup. It’s a bit of an involved one, but I know what we need to do to resolve it.

I’ve a reproducer:

Run this HTTP server script (a simple HTTP server that takes long to respond):

  import asyncio
  from hypercorn.asyncio import serve
  from hypercorn.config import Config
  from starlette.applications import Starlette
  from starlette.responses import JSONResponse
  from starlette.routing import Route

  async def homepage(request):
      await asyncio.sleep(10)
      return JSONResponse({})

  app = Starlette(
      routes=[
          Route("/", homepage),
      ],
  )

  config = Config.from_mapping({})
  config.bind = ["127.0.0.1:8001"]
  asyncio.run(serve(app, config))

Then run this client code:

  from anyio import create_task_group
  import asyncio
  import httpx

  async def main() -> None:
      async with httpx.AsyncClient(
          limits=httpx.Limits(max_connections=2),
          verify=False,
      ) as client:

          async def do_one_request() -> None:
              await client.get("http://localhost:8001/")

          # First, create many requests, then cancel while they are in progress.
          async with create_task_group() as tg:
              for i in range(5):
                  tg.start_soon(do_one_request)
              await asyncio.sleep(0.5)
              tg.cancel_scope.cancel()

          # Starting another request will now fail with a `PoolTimeout`.
          await do_one_request()

  asyncio.run(main())

Looks like the slots in the connection pool are not released during cancellation.

This happens for me on both httpx 0.25.0 + httpcore 0.18.0 as well as on httpx 0.25.2 + httpcore 1.0.2. Shielding the get() call or stream() call from cancellation is a workaround that works for us.

this was the fix:

- httpcore==1.0.1
- httpx==0.25.1
+ httpcore==0.18.0
+ httpx==0.25.0

Faced this error in 0.25.1. Fallback to 0.25.0 fixed the problem @tomchristie

Is this still a problem in 0.19 or 1.0.0? I tried running @tomchristie 's code sample but couldn’t replicate the behavior on 0.18.x or 0.19.

We’ve held off updating beyond 0.17.1 due to this, but would really like to get back onto the latest.

Hello everyone,

just want to make sure that this it what I’m looking for. The server I want to send request to have a limited number of allowed connections. Currently I limit the number of async task by using Semaphore. But the pool_limits parameter for AsyncClient looks like this is intended for my use case. Am I right here? If so, any idea when this issue here will be resolved?

Thanks a lot!

fin swimmer

@pssolanki111 Sure is, yup… https://www.python-httpx.org/advanced/#pool-limit-configuration