anyio: Possible race condition in get_asynclib when threading

I’m not quite sure how to reproduce and it’s also possible I’m doing something wrong that’s causing this. I figured I’d open this just so anyone else that might be experiencing something similar has something to try as a fix, so feel free to close if there’s not enough info or shouldn’t be solved in AnyIO.

Here’s the traceback I was seeing when trying to use anyio from a thread:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
...
...
...
  File "/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/anyio/_core/_tasks.py", line 74, in create_task_group
    return get_asynclib().TaskGroup()
AttributeError: module 'anyio._backends._asyncio' has no attribute 'TaskGroup'

This seemed very weird until I got another traceback claiming that anyio._backends._asyncio was partially initialized which clued me into the fact that this was a race condition.

The solution was just to import anyio._backends._asyncio at the top of my file before starting the thread that imports anyio.

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 25 (11 by maintainers)

Most upvoted comments

My best guess is that the race condition occurs when two threads check whether anyio._backends._asyncio (M) needs to be imported…

The first thread (T1) checks sys.modules, finds M is absent, and begins to import it. Then the following thread (T2) arrives at sys.modules to find that M is present but, unbeknownst to T2, the module is only partially initialized because T1 has not finished importing it. Finally T2 attempts to use M, which is partially imported, under the assumption that M has actually been fully initialized, thus resulting in an AttributeError when T2 accesses a resource that has not been defined yet.

Given that this is in fact the problem, I can see two solutions:

eagerly import all available backends
insert a lock around access to the backend modules while they are being imported

rmorshea on Feb 24, 2021