poetry: Poetry self update fails on Windows with Access Denied, leaving poetry in a broken state

  • Poetry version: Current version is 1.4.0, but it has happened to me multiple times when updating poetry <!-- Replace with version, e.g. from poetry --version -->
  • Python version: System Python is 3.10.5
  • OS version and name: Windows 10
  • pyproject.toml: N/A
  • I am on the latest stable Poetry version, installed using a recommended method. (not really applicable)
  • I have searched the issues of this repo and believe that this is not a duplicate. (there are similar issues, but none have a solution/activity)
  • I have consulted the FAQ and blog for any relevant entries or release notes.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below. (output included below, but not verbose)

Issue

This has happened to me for multiple version updates in a row now. Whenever upgrading poetry with poetry self update pip fails with an OSError and leaves Poetry in a broken state. A reinstall fixes that.

There are two problems here:

  1. Pip crashes due to an OSError: this does not happen when normally running pip or using poetry to install/update venvs, only when upgrading poetry itself
  2. Poetry does not rollback any changes after pip crashes, thus leaving itself in a broken and unusable state
poetry self update
Updating Poetry version ...

Using version ^1.4.0 for poetry

Updating dependencies
Resolving dependencies...

Writing lock file

Package operations: 4 installs, 4 updates, 0 removals

  • Installing colorama (0.4.6)
  • Updating poetry-core (1.4.0 -> 1.5.1)
  • Installing pyproject-hooks (1.0.0)
  • Installing build (0.10.0)
  • Updating dulwich (0.20.50 -> 0.21.3)
  • Updating poetry-plugin-export (1.2.0 -> 1.3.0)
  • Installing installer (0.6.0)

  CalledProcessError

  Command 'C:\Users\iTitus\AppData\Roaming\pypoetry\venv\Scripts\python.exe -m pip install --disable-pip-version-check --isolated --no-input --prefix C:\Users\iTitus\AppData\Roaming\pypoetry\venv --upgrade --no-deps C:\Users\iTitus\AppData\Local\pypoetry\Cache\artifacts\88\74\c6\b2de65e4cd5082ed96394c6c1ca241ad7839b76a5b9cc35af5c0b84ecb\dulwich-0.21.3-cp311-cp311-win_amd64.whl' returned non-zero exit status 1.

  at C:\Program Files\Python311\Lib\subprocess.py:569 in run
       565│             # We don't call process.wait() as .__exit__ does that for us.
       566│             raise
       567│         retcode = process.poll()
       568│         if check and retcode:
    →  569│             raise CalledProcessError(retcode, process.args,
       570│                                      output=stdout, stderr=stderr)
       571│     return CompletedProcess(process.args, retcode, stdout, stderr)
       572│
       573│

The following error occurred when trying to handle this error:


  EnvCommandError

  Command C:\Users\iTitus\AppData\Roaming\pypoetry\venv\Scripts\python.exe -m pip install --disable-pip-version-check --isolated --no-input --prefix C:\Users\iTitus\AppData\Roaming\pypoetry\venv --upgrade --no-deps C:\Users\iTitus\AppData\Local\pypoetry\Cache\artifacts\88\74\c6\b2de65e4cd5082ed96394c6c1ca241ad7839b76a5b9cc35af5c0b84ecb\dulwich-0.21.3-cp311-cp311-win_amd64.whl errored with the following return code 1, and output:
  Processing c:\users\ititus\appdata\local\pypoetry\cache\artifacts\88\74\c6\b2de65e4cd5082ed96394c6c1ca241ad7839b76a5b9cc35af5c0b84ecb\dulwich-0.21.3-cp311-cp311-win_amd64.whl
  Installing collected packages: dulwich
    Attempting uninstall: dulwich
      Found existing installation: dulwich 0.20.50
      Uninstalling dulwich-0.20.50:
        Successfully uninstalled dulwich-0.20.50
  ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'C:\\Users\\iTitus\\AppData\\Roaming\\pypoetry\\venv\\Lib\\site-packages\\~ulwich\\_diff_tree.cp311-win_amd64.pyd'
  Check the permissions.



  at venv\Lib\site-packages\poetry\utils\env.py:1540 in _run
      1536│                 output = subprocess.check_output(
      1537│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1538│                 )
      1539│         except CalledProcessError as e:
    → 1540│             raise EnvCommandError(e, input=input_)
      1541│
      1542│         return decode(output)
      1543│
      1544│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:


  PoetryException

  Failed to install C:/Users/iTitus/AppData/Local/pypoetry/Cache/artifacts/88/74/c6/b2de65e4cd5082ed96394c6c1ca241ad7839b76a5b9cc35af5c0b84ecb/dulwich-0.21.3-cp311-cp311-win_amd64.whl

  at venv\Lib\site-packages\poetry\utils\pip.py:58 in pip_install
       54│
       55│     try:
       56│         return environment.run_pip(*args)
       57│     except EnvCommandError as e:
    →  58│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
       59│

About this issue

  • Original URL
  • State: open
  • Created a year ago
  • Reactions: 10
  • Comments: 24 (8 by maintainers)

Most upvoted comments

But can there be a fix for the issue where poetry literally breaks itself when this exception is thrown? Catch it and roll-back maybe?

To be fair, this error was to do with poetry.exe: https://github.com/python-poetry/poetry/issues/7610#issuecomment-1483025421

And the issue with the Poetry executable is effectively the same cause as the issue with other permission errors reported here, I would argue it’s still relevant.

But you’re right - the OSErrors are also arising from Poetry’s dependencies being updated whilst Poetry is using them, especially DLLs (.pyd files) such as Dulwich. However, if a poetry self update happens where none of the in-use DLLs are updated, you will always have a permission error with poetry.exe - that one is guaranteed. The EXE renaming hack won’t prevent the DLL update errors. And I’m assuming it’s impossible for Poetry to prevent those DLLs from being loaded during the self update process.

Perhaps poetry self update should do nothing on Windows, and instead print an error message saying to update using the official installer again (if it was originally installed with the official installer)? It could print the command (Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | py -.

some but not all of the reports in this thread are about updating poetry.exe - https://github.com/python-poetry/poetry/issues/7610#issuecomment-1483025421 is, the opener is not

but in general: yes, this is what I have been saying throughout - the problem is poetry trying to update files while it is using them. poetry.exe is indeed one such file.

The recent update gave me this (different) error:

> poetry self update
Updating Poetry version ...

Using version ^1.4.1 for poetry

Updating dependencies
Resolving dependencies...

Writing lock file

Package operations: 0 installs, 3 updates, 0 removals

  • Updating poetry-core (1.5.1 -> 1.5.2)
  • Updating installer (0.6.0 -> 0.7.0)
  • Updating poetry (1.4.0 -> 1.4.1)

  CalledProcessError

  Command '['C:\\Users\\iTitus\\AppData\\Roaming\\pypoetry\\venv\\Scripts\\python.exe', '-m', 'pip', 'uninstall', 'poetry', '-y']' returned non-zero exit status 2.

  at C:\Program Files\Python311\Lib\subprocess.py:571 in run
       567│             # We don't call process.wait() as .__exit__ does that for us.
       568│             raise
       569│         retcode = process.poll()
       570│         if check and retcode:
    →  571│             raise CalledProcessError(retcode, process.args,
       572│                                      output=stdout, stderr=stderr)
       573│     return CompletedProcess(process.args, retcode, stdout, stderr)
       574│
       575│

The following error occurred when trying to handle this error:


  EnvCommandError

  Command ['C:\\Users\\iTitus\\AppData\\Roaming\\pypoetry\\venv\\Scripts\\python.exe', '-m', 'pip', 'uninstall', 'poetry', '-y'] errored with the following return code 2

  Output:
  Found existing installation: poetry 1.4.0
  Uninstalling poetry-1.4.0:
  ERROR: Exception:
  Traceback (most recent call last):
    File "C:\Program Files\Python311\Lib\shutil.py", line 825, in move
      os.rename(src, real_dst)
  PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'c:\\users\\ititus\\appdata\\roaming\\pypoetry\\venv\\scripts\\poetry.exe' -> 'C:\\Users\\iTitus\\AppData\\Local\\Temp\\pip-uninstall-ux9bwfj8\\poetry.exe'

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "C:\Users\iTitus\AppData\Roaming\pypoetry\venv\Lib\site-packages\pip\_internal\cli\base_command.py", line 160, in exc_logging_wrapper
      status = run_func(*args)
               ^^^^^^^^^^^^^^^
    File "C:\Users\iTitus\AppData\Roaming\pypoetry\venv\Lib\site-packages\pip\_internal\commands\uninstall.py", line 105, in run
      uninstall_pathset = req.uninstall(
                          ^^^^^^^^^^^^^^
    File "C:\Users\iTitus\AppData\Roaming\pypoetry\venv\Lib\site-packages\pip\_internal\req\req_install.py", line 664, in uninstall
      uninstalled_pathset.remove(auto_confirm, verbose)
    File "C:\Users\iTitus\AppData\Roaming\pypoetry\venv\Lib\site-packages\pip\_internal\req\req_uninstall.py", line 373, in remove
      moved.stash(path)
    File "C:\Users\iTitus\AppData\Roaming\pypoetry\venv\Lib\site-packages\pip\_internal\req\req_uninstall.py", line 271, in stash
      renames(path, new_path)
    File "C:\Users\iTitus\AppData\Roaming\pypoetry\venv\Lib\site-packages\pip\_internal\utils\misc.py", line 312, in renames
      shutil.move(old, new)
    File "C:\Program Files\Python311\Lib\shutil.py", line 846, in move
      os.unlink(src)
  PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'c:\\users\\ititus\\appdata\\roaming\\pypoetry\\venv\\scripts\\poetry.exe'


  at venv\Lib\site-packages\poetry\utils\env.py:1545 in _run


> poetry self update
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\iTitus\AppData\Roaming\Python\Scripts\poetry.exe\__main__.py", line 4, in <module>
ModuleNotFoundError: No module named 'poetry.console'