procrastinate: v2 beta: Django connector seems broken

This is about the v2 beta testing.

From discord, from @paulzakin:

error connecting in 'pool-1': connection failed: could not receive data from server: Connection refused
error connecting in 'pool-1': connection failed: could not receive data from server: Connection refused
error connecting in 'pool-1': connection failed: could not receive data from server: Connection refused
error connecting in 'pool-1': connection failed: could not receive data from server: Connection refused
Traceback (most recent call last):
  File "/Users/code/server/worker.py", line 39, in <module>
    main()
  File "/Users/code/server/worker.py", line 35, in main
    x.run_worker()  # pyright: ignore [reportUnknownMemberType]
    ^^^^^^^^^^^^^^
  File "/Users/code/.venv/lib/python3.11/site-packages/procrastinate/app.py", line 270, in run_worker
    asyncio.run(f())
  File "/Users/foo/.local/share/mise/installs/python/3.11.7/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/Users/foo/.local/share/mise/installs/python/3.11.7/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/Users/foo/.local/share/mise/installs/python/3.11.7/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/code/.venv/lib/python3.11/site-packages/procrastinate/app.py", line 267, in f
    async with self.open_async():
  File "/Users/code/.venv/lib/python3.11/site-packages/procrastinate/utils.py", line 194, in __aenter__
    await self._open_coro()
  File "/Users/code/.venv/lib/python3.11/site-packages/procrastinate/psycopg_connector.py", line 170, in open_async
    await self._async_pool.open(wait=True)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/code/.venv/lib/python3.11/site-packages/psycopg_pool/pool_async.py", line 382, in open
    await self.wait(timeout=timeout)
  File "/Users/code/.venv/lib/python3.11/site-packages/psycopg_pool/pool_async.py", line 169, in wait
    raise PoolTimeout(f"pool initialization incomplete after {timeout} sec")
psycopg_pool.PoolTimeout: pool initialization incomplete after 30.0 sec

I invoke the entry with poetry run python server/worker.py And the entry looks like this

import django

django.setup()

import procrastinate
from procrastinate.contrib.django import app

from server.env import EnvService
from server.services.db import ProcrastinateService

ENV = EnvService.get_env()
env = EnvService.get_env_variables(ENV)

connector = procrastinate.PsycopgConnector(
    kwargs={
        "host": env["POSTGRES_HOST"],
        "user": env["POSTGRES_USER"],
        "password": env["POSTGRES_PASSWORD"],
        "port": env["POSTGRES_PORT"],
        "dbname": env["POSTGRES_DB"],
        **EnvService.get_ssl(),
    }
)


def main() -> None:
    x = app.with_connector(connector)
    blueprints = ProcrastinateService.get_blueprints(show_warnings=True)
    for blueprint, namespace in blueprints:
        x.add_tasks_from(blueprint, namespace=namespace)
    x.run_worker()  # pyright: ignore [reportUnknownMemberType]


if __name__ == "__main__":
    main()

And the way I used to run it was poetry run procrastinate --app=server.worker.app worker And I used to define app = procrastinate.App(connector=connector) in the settings.py basically and then import that into worker.py Even more interesting: when all I do is change the version to v2 and leave all the code as it is (no django stuff changed), app defined in settings.py and then invoked in worker.py, I get that same error pool initialization incomplete after 30.0 sec.

So (my guess) when I invoke poetry run procrastinate --app=server.worker.app worker --concurrency=1 with the imported app on the new version, something changed.

About this issue

  • Original URL
  • State: closed
  • Created 5 months ago
  • Comments: 59 (57 by maintainers)

Most upvoted comments

Well… It was not stated on the title at the time of your comment 😃

image

@hyusetiawan your comment makes me think that you’re probably using the last stable release v1.1.2, while this issue is (sorry, unclearly) about the v2 beta.

As far as I can tell, the docs are configured to still point to 1.1.2, so maybe you’re looking at the docs on the main branch ?

That being said, you’re more than welcome to try and integrate v2.0.0b2 (it’s worth re-reading the Django docs completely) and report any problem here. The v2 had the django integration completely re-done, and in theory it should be much smoother. That said, @paulzakin can attest that so far we’re still experiencing road bumps.

Are you expected to install aiopg to get this to work? I thought we were switching to psycopg3 because it has a good async interface? I’m happy to install aiopg, just wanted to check.