poetry: Poetry install is stuck on specific dependency with "pending..."

I do not know how to help you all here…

My advice would be to try one of those (maybe in that order) if you have not yet:

• delete the poetry.lock file and retry
• delete the virtual environment and retry
• deactivate the new (parallel) installer with poetry config experimental.new-installer false and retry (do not forget to reactivate it afterwards if it proves to have no influence)
• or any combination of those

Also maybe provide some verbose output (-vvv), that might be helpful for when one of the maintainers gets the time to handle this ticket. Of course, the best as always is if you can provide a minimal reproducible example that triggers the failure.

The artifacts and cache dirs were in ~/Library/Caches/pypoetry on my M1 Mac for those who can’t find it in ~/.cache/pypoetry/cache/

cd ~/Library/Caches/pypoetry
rm -rf artifacts/ cache/

Doing that cleared things up for me. Before that, it was consistently hanging on trying to fetch flake8-bugbear from pypy

it also happens under:

pip 22.2.2
poetry 1.2.1
python 3.8.5 and also 3.10.0

traceback:

  • Installing cachetools (5.0.0): Pending...
  • Installing jmespath (0.10.0): Pending...
  • Installing oauthlib (3.2.0): Pending...
  • Installing pyasn1-modules (0.2.8): Pending...
  • Installing python-dateutil (2.8.2): Pending...
  • Updating requests (2.28.1 -> 2.27.1): Pending...
  • Installing rsa (4.8): Pending...
  • Installing smmap (5.0.0): Pending...
^C^C^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/home/zach/.asdf/installs/python/3.8.5/lib/python3.8/concurrent/futures/thread.py", line 40, in _python_exit
    t.join()
  File "/home/zach/.asdf/installs/python/3.8.5/lib/python3.8/threading.py", line 1011, in join
    self._wait_for_tstate_lock()
  File "/home/zach/.asdf/installs/python/3.8.5/lib/python3.8/threading.py", line 1027, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt

workaround:

poetry config experimental.new-installer false

it started to happen this week, last week the same command(s) worked fine 🤔

@Purg thank you sire! i’ve been stuck on this for hours… i tried all these:

1. turn off the experimental.new-installer

   poetry config experimental.new-installer false
   

   but this only worked “for a moment” then stopped working:

   

2. delete virtualenv - didn’t work

3. delete poetry.lock - didn’t work

4. clear all cache by running (see https://github.com/python-poetry/poetry/issues/2094#issuecomment-1231501255)

   $ poetry cache list
   $ poetry cache clear --all pypi 
   

   didn’t work

5. only yours worked:

   cd ~/.cache/pypoetry/cache/
   rm -rf artifacts/ cache/
   

Happens on WSL2 ubuntu 20.04 with python3.10

I do experience the same problem with poetry 1.1.4 within a python 3.9 docker container. The error goes away by following the suggestion of @sinoroc to deactivate the experimental installer.

pyproject.toml:

[tool.poetry]
name = "foobar"
version = "0.1.0"
description = ""
authors = []

[tool.poetry.dependencies]
python = "^3.8"
watchdog = "^1.0.1"

[tool.poetry.dev-dependencies]
docker = "^4.4.0"
pytest = "^6.2.1"

poetry.lock.txt

On Ubuntu 20.04 WSL2 here. Only disabling IPv6 did the trick:

sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=1

Artifacts are distfiles (wheels, sdists) and can be corrupted by an interrupted Poetry (ctrl-c, machine shutdown) as well as a concurrent Poetry. This will be fixed in the 1.3 release.

You can always find the cache directory with poetry config cache-dir. If you’re having issues you think might be related, try clearing the cache first. If you’re still having issues, I’d suggest opening a Discussion or joining Discord first as 90% of the time these are local config issues.

I would ask that if you don’t have anything to add that hasn’t already been said in this issue, you hold off, as generating notifications for 20+ people for a root cause that is solved is unnecessary (and if you have something novel, we want to identify that and make sure it’s tracked as a new issue).

Disabling experimental installer also fixes this for me, but also able to fix it using the experimental installer by adding a timeout for the requests to pypi, E.g.

diff --git a/poetry/repositories/pypi_repository.py b/poetry/repositories/pypi_repository.py
index 1c7b7be3..a1f0922a 100644
--- a/poetry/repositories/pypi_repository.py
+++ b/poetry/repositories/pypi_repository.py
@@ -321,14 +321,14 @@ class PyPiRepository(RemoteRepository):
 
         return data.asdict()
 
-    def _get(self, endpoint: str) -> Union[dict, None]:
+    def _get(self, endpoint: str, timeout: int = 5) -> Union[dict, None]:
         try:
-            json_response = self.session.get(self._base_url + endpoint)
+            json_response = self.session.get(self._base_url + endpoint, timeout=timeout)
         except requests.exceptions.TooManyRedirects:
             # Cache control redirect loop.
             # We try to remove the cache and try again
             self._cache_control_cache.delete(self._base_url + endpoint)
-            json_response = self.session.get(self._base_url + endpoint)
+            json_response = self.session.get(self._base_url + endpoint, timeout=timeout)
 
         if json_response.status_code == 404:
             return None

Could someone on the Poetry team provide a perspective on prioritization for this? It seems like there’s meaningful evidence that this is an issue independent of environment or dependencies. Turning off the installer is a good workaround in the interim, but it significantly increases install times. I’d love to know I have some hope of being able to turn it back on at some point.

I’m facing this issue in the latest Poetry version. This helps:

poetry config experimental.new-installer false

If you can reproduce this reliably in a container or clean environment, please open a new issue with the reproduction. There are many possible factors from cache corruption, network issues, etc that could influence this, so a clean reproduction is key.

I can confirm that disabling the experimental parallel installer worked. My environment, although, was Docker, particularly the python:3.8-slim image.

Same issue here. I just upgrade from v1.0.0, but got pending on some packages during installing , typically numpy , sklearn… The error message shows it got a read time out from my nexus repo, but it never happened before. And after I try the poetry install several times, it installed all package successfully. Does the parallel installation trigger this issue?