pytest-asyncio: RuntimeError: There is no current event loop in thread 'MainThread'.

@lindycoder I do have some tests which do this with one more level of isolation (where a test calls a sync function, but under the hood that sync function ends up running async code). In my particular case the reason is highly involved and boils down to using an async postgresql context inside alembic, which is sync. (The involved thing is why we decided to this in the first place…). Another example would be testing something like asyncio.run() (e.g. testing an async repl).

Of course if the fixture / test is under one’s control one can simply make it async, but if the sync -> async boundary is under the hood it’s harder.

I don’t know why @qci-amos needs it though, but it’s definitely a valid (if weird) use-case.

We have the same issue as @qci-amos reported (out setup.cfg contains [tool.pytest.ini_options] asyncio_mode = "auto").

pytest assumes that tests are synchronous functions and gives a warning when you try to write async def tests. pytest-asyncio wraps async tests to in synchronous functions to satisfy pytest. In v.0.21.0, this synchronization wrapper had an explicit reference to the event loop in which the test should run. This reference is no longer present in v0.23.0a1 (see https://github.com/pytest-dev/pytest-asyncio/commit/36b226936e17232535e88ca34f9707cdf211776b) and the synchronization wrappers run in the loop returned by asyncio.get_event_loop().

I can see that the order in which fixtures are evaluated has changed from v0.21.0 to v0.23.0a1.

v0.21.0:

$ pytest --asyncio-mode=auto --setup-show
===== test session starts =====
platform linux -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/tst
plugins: asyncio-0.21.0
asyncio: mode=Mode.AUTO
collected 4 items                                                                                                                                                                                                                                                            

test_a.py 
        SETUP    F event_loop
        SETUP    F a1[True]
        SETUP    F a2[True]
        SETUP    F nested_async
        test_a.py::test_repro[True-True] (fixtures used: a1, a2, event_loop, nested_async).
        TEARDOWN F nested_async
        TEARDOWN F a2[True]
        TEARDOWN F a1[True]
        TEARDOWN F event_loop
        SETUP    F event_loop
        SETUP    F a1[False]
        SETUP    F a2[True]
        SETUP    F nested_async
        test_a.py::test_repro[True-False] (fixtures used: a1, a2, event_loop, nested_async).
        TEARDOWN F nested_async
        TEARDOWN F a2[True]
        TEARDOWN F a1[False]
        TEARDOWN F event_loop
        SETUP    F event_loop
        SETUP    F a1[True]
        SETUP    F a2[False]
        SETUP    F nested_async
        test_a.py::test_repro[False-True] (fixtures used: a1, a2, event_loop, nested_async).
        TEARDOWN F nested_async
        TEARDOWN F a2[False]
        TEARDOWN F a1[True]
        TEARDOWN F event_loop
        SETUP    F event_loop
        SETUP    F a1[False]
        SETUP    F a2[False]
        SETUP    F nested_async
        test_a.py::test_repro[False-False] (fixtures used: a1, a2, event_loop, nested_async).
        TEARDOWN F nested_async
        TEARDOWN F a2[False]
        TEARDOWN F a1[False]
        TEARDOWN F event_loop

===== 4 passed in 0.01s ====

$ pytest --asyncio-mode=auto --setup-show
===== test session starts =====
platform linux -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /tmp/tst
plugins: asyncio-0.23.0a1
asyncio: mode=Mode.AUTO
collected 4 items                                                                                                                                                                                                                                                            

test_a.py 
SETUP    S event_loop_policy
        SETUP    F a1[True]
        SETUP    F a2[True]
        SETUP    F nested_async
        SETUP    F event_loop
        test_a.py::test_repro[True-True] (fixtures used: a1, a2, event_loop, event_loop_policy, nested_async).
        TEARDOWN F event_loop
        TEARDOWN F nested_async
        TEARDOWN F a2[True]
        TEARDOWN F a1[True]
        SETUP    F event_loop
        SETUP    F a1[False]
        SETUP    F a2[True]
        SETUP    F nested_async
        test_a.py::test_repro[True-False] (fixtures used: a1, a2, event_loop, event_loop_policy, nested_async)F
        TEARDOWN F nested_async
        TEARDOWN F a2[True]
        TEARDOWN F a1[False]
        TEARDOWN F event_loop
        SETUP    F event_loop
        SETUP    F a1[True]
        SETUP    F a2[False]
        SETUP    F nested_async
        test_a.py::test_repro[False-True] (fixtures used: a1, a2, event_loop, event_loop_policy, nested_async)F
        TEARDOWN F nested_async
        TEARDOWN F a2[False]
        TEARDOWN F a1[True]
        TEARDOWN F event_loop
        SETUP    F event_loop
        SETUP    F a1[False]
        SETUP    F a2[False]
        SETUP    F nested_async
        test_a.py::test_repro[False-False] (fixtures used: a1, a2, event_loop, event_loop_policy, nested_async)F
        TEARDOWN F nested_async
        TEARDOWN F a2[False]
        TEARDOWN F a1[False]
        TEARDOWN F event_loop
TEARDOWN S event_loop_policy

===== FAILURES ====
_____test_repro[True-False] _____

self = <Coroutine test_repro[True-False]>

    def runtest(self) -> None:
        if self.get_closest_marker("asyncio"):
            self.obj = wrap_in_sync(
                # https://github.com/pytest-dev/pytest-asyncio/issues/596
                self.obj,  # type: ignore[has-type]
            )
>       super().runtest()

venv/lib/python3.11/site-packages/pytest_asyncio/plugin.py:427: 
_ _ _
venv/lib/python3.11/site-packages/pytest_asyncio/plugin.py:856: in inner
    _loop = asyncio.get_event_loop()
_ _ _

self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x7fb546759850>

    def get_event_loop(self):
        """Get the event loop for the current context.
    
        Returns an instance of EventLoop or raises an exception.
        """
        if (self._local._loop is None and
                not self._local._set_called and
                threading.current_thread() is threading.main_thread()):
            self.set_event_loop(self.new_event_loop())
    
        if self._local._loop is None:
>           raise RuntimeError('There is no current event loop in thread %r.'
                               % threading.current_thread().name)
E           RuntimeError: There is no current event loop in thread 'MainThread'.

/usr/lib/python3.11/asyncio/events.py:677: RuntimeError
----- Captured stdout setup -----
here!
_____test_repro[False-True] _____

self = <Coroutine test_repro[False-True]>

    def runtest(self) -> None:
        if self.get_closest_marker("asyncio"):
            self.obj = wrap_in_sync(
                # https://github.com/pytest-dev/pytest-asyncio/issues/596
                self.obj,  # type: ignore[has-type]
            )
>       super().runtest()

venv/lib/python3.11/site-packages/pytest_asyncio/plugin.py:427: 
_ _ _
venv/lib/python3.11/site-packages/pytest_asyncio/plugin.py:856: in inner
    _loop = asyncio.get_event_loop()
_ _ _

self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x7fb546759850>

    def get_event_loop(self):
        """Get the event loop for the current context.
    
        Returns an instance of EventLoop or raises an exception.
        """
        if (self._local._loop is None and
                not self._local._set_called and
                threading.current_thread() is threading.main_thread()):
            self.set_event_loop(self.new_event_loop())
    
        if self._local._loop is None:
>           raise RuntimeError('There is no current event loop in thread %r.'
                               % threading.current_thread().name)
E           RuntimeError: There is no current event loop in thread 'MainThread'.

/usr/lib/python3.11/asyncio/events.py:677: RuntimeError
----- Captured stdout setup -----
here!
_____test_repro[False-False] _____

self = <Coroutine test_repro[False-False]>

    def runtest(self) -> None:
        if self.get_closest_marker("asyncio"):
            self.obj = wrap_in_sync(
                # https://github.com/pytest-dev/pytest-asyncio/issues/596
                self.obj,  # type: ignore[has-type]
            )
>       super().runtest()

venv/lib/python3.11/site-packages/pytest_asyncio/plugin.py:427: 
_ _ _
venv/lib/python3.11/site-packages/pytest_asyncio/plugin.py:856: in inner
    _loop = asyncio.get_event_loop()
_ _ _

self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x7fb546759850>

    def get_event_loop(self):
        """Get the event loop for the current context.
    
        Returns an instance of EventLoop or raises an exception.
        """
        if (self._local._loop is None and
                not self._local._set_called and
                threading.current_thread() is threading.main_thread()):
            self.set_event_loop(self.new_event_loop())
    
        if self._local._loop is None:
>           raise RuntimeError('There is no current event loop in thread %r.'
                               % threading.current_thread().name)
E           RuntimeError: There is no current event loop in thread 'MainThread'.

/usr/lib/python3.11/asyncio/events.py:677: RuntimeError
----- Captured stdout setup -----
here!
===== short test summary info =====
FAILED test_a.py::test_repro[True-False] - RuntimeError: There is no current event loop in thread 'MainThread'.
FAILED test_a.py::test_repro[False-True] - RuntimeError: There is no current event loop in thread 'MainThread'.
FAILED test_a.py::test_repro[False-False] - RuntimeError: There is no current event loop in thread 'MainThread'.
===== 3 failed, 1 passed in 0.08s =====```

The logs show that tests fail when the nested_async fixture is setup after event_loop. Note that nested_async calls asyncio.run(). As 2e0byo already mentioned, asyncio.run sets the current event loop to None when finishing. This causes the synchronization wrappers to fail, because the current event loop is None.

I don’t know why the fixtures are evaluated in different order for different parametrizations, but it don’t think it’s relevant for the case.

Although I agree that it’s preferable to use

@pytest_asyncio.fixture
async def nested_async():
    return await my_async_method()

over asyncio.run(), the test fails for a non-obvious reason and the error message doesn’t really help.

@qci-amos calling asyncio.run anywhere except the top-level entrypoint is generally frowned upon. The docs say

This function should be used as a main entry point for asyncio programs, and should ideally only be called once.

Calling run cleans up and stops the executor, which is probably why. Thus it’s not valid to run any async code afterwards (without provisioning a new loop). This is doubtless a surprise to pytest-asyncio, since the executor is already stopped when it goes to clean up.

FWIW I regard asyncio.run() as ‘surprising’ code: personally I’d definitely use the async fixture here to avoid the side effect. But there are times when you can’t.

(The postgresql case is more fun although I haven’t looked into it: sync_engine returns something something greenlets something something.)

Ok, here’s a new repro:

pip install pytest-asyncio==0.23.0a1

import pytest
import asyncio


async def my_async_method():
    print("here!")


@pytest.fixture
def nested_async():
    return asyncio.run(my_async_method())


@pytest.mark.parametrize("a1", [True, False])
@pytest.mark.parametrize("a2", [True, False])
async def test_repro(a1, a2, nested_async):
    print("in test", a1, a2)

======================================================================== short test summary info =========================================================================
FAILED test_repro.py::test_repro[True-False] - RuntimeError: There is no current event loop in thread 'MainThread'.
FAILED test_repro.py::test_repro[False-True] - RuntimeError: There is no current event loop in thread 'MainThread'.
FAILED test_repro.py::test_repro[False-False] - RuntimeError: There is no current event loop in thread 'MainThread'.
================================================================ 3 failed, 1 passed, 23 warnings in 0.12s ================================================================

Thanks, this version fixed the test I created the repro for, but the other test (which looks the same in this regard to my eye) is still failing. I’ll see if I can make another repro today.

It’s not letting me reopen this @seifertm , but I find my error was indeed separate.

In conftest.py:

async def my_async_method():
    print("here!")


@pytest.fixture
def nested_async():
    return asyncio.run(my_async_method())

In test_repro.py:

async def test_repro(nested_async):
    print("in test")

gives me:

======================================================================== FAILURES =========================================================================
_______________________________________________________________________ test_repro ________________________________________________________________________
self = <Coroutine test_repro>

    def runtest(self) -> None:
        if self.get_closest_marker("asyncio"):
            self.obj = wrap_in_sync(
                # https://github.com/pytest-dev/pytest-asyncio/issues/596
                self.obj,  # type: ignore[has-type]
            )
>       super().runtest()

../../../miniconda3/envs/py311/lib/python3.11/site-packages/pytest_asyncio/plugin.py:426:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../miniconda3/envs/py311/lib/python3.11/site-packages/pytest_asyncio/plugin.py:847: in inner
    _loop = asyncio.get_event_loop()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x7f3b3a640810>

    def get_event_loop(self):
        """Get the event loop for the current context.

        Returns an instance of EventLoop or raises an exception.
        """
        if (self._local._loop is None and
                not self._local._set_called and
                threading.current_thread() is threading.main_thread()):
            self.set_event_loop(self.new_event_loop())

        if self._local._loop is None:
>           raise RuntimeError('There is no current event loop in thread %r.'
                               % threading.current_thread().name)
E           RuntimeError: There is no current event loop in thread 'MainThread'.

../../../miniconda3/envs/py311/lib/python3.11/asyncio/events.py:677: RuntimeError
------------------------------------------------------------------ Captured stdout setup ------------------------------------------------------------------
here!

Thanks @seifertm!

@gabrielmbmb I filed a PR to mark pytest-asyncio v0.22.0 as broken on conda-forge: https://github.com/conda-forge/admin-requests/pull/857

Hey guys, just wanted to mention that pytest-asyncio==0.22.0 hasn’t been yanked in conda-forge https://anaconda.org/conda-forge/pytest-asyncio

@lindycoder larger scopes are being discussed over here https://github.com/pytest-dev/pytest-asyncio/issues/657

Thank you for fixing by removing 0.22.0.

I would like to share something regarding this warning:

PytestDeprecationWarning: xxxxxxxx is asynchronous and explicitly requests the "event_loop" fixture. Asynchronous fixtures and test functions should use "asyncio.get_running_loop()" instead.

We have session scoped async fixtures so we have this central fixture

@pytest.fixture(scope="session")
def event_loop() -> Generator[AbstractEventLoop, None, None]:
    """Make the loop session scope to use session async fixtures."""
    policy = asyncio.get_event_loop_policy()
    loop = policy.new_event_loop()
    yield loop
    loop.close()

And every single AsyncGenerator fixtures we have actually require event_loop to make sure the teardown is done before the loop is closed.

Sharing this use-case hoping to help further development.