poetry: Poetry tries to remove setuptools and then complains it can't find it

I am on the latest Poetry version.
I have searched the issues of this repo and believe that this is not a duplicate.
If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

OS version and name: Debian 10
Poetry version: 1.1.7
Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/nyuszika7h/238727b812cfce957e0e34e1143b160f

Issue

When running poetry install, Poetry tries to remove setuptools, and then later complains that setuptools is not found when trying to install future.

I suspect this may be because win10toast depends on setuptools, but that package is skipped due to the platform not being Windows.

❯ poetry install -vvv
Creating virtualenv test in /home/nyuszika7h/test/.venv
Using virtualenv: /home/nyuszika7h/test/.venv
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 7 installs, 0 updates, 1 removal, 6 skipped

  • Removing importlib-metadata (4.6.0): Pending...
  • Removing importlib-metadata (4.6.0): Skipped for the following reason: Not currently installed
  • Removing pypiwin32 (223): Pending...
  • Removing pypiwin32 (223): Skipped for the following reason: Not currently installed
  • Removing pywin32 (301): Pending...
  • Removing pywin32 (301): Skipped for the following reason: Not currently installed
  • Removing win10toast (0.9): Pending...
  • Removing win10toast (0.9): Skipped for the following reason: Not currently installed
  • Removing typing-extensions (3.10.0.0): Pending...
  • Removing typing-extensions (3.10.0.0): Skipped for the following reason: Not currently installed
  • Removing zipp (3.4.1): Pending...
  • Removing zipp (3.4.1): Skipped for the following reason: Not currently installed
  • Removing setuptools (57.0.0): Pending...
  • Removing setuptools (57.0.0): Removing...
  • Removing setuptools (57.0.0)
  • Installing soupsieve (2.2.1): Pending...
  • Installing soupsieve (2.2.1): Installing...
  • Installing soupsieve (2.2.1)
  • Installing beautifulsoup4 (4.9.3): Pending...
  • Installing beautifulsoup4 (4.9.3): Installing...
  • Installing cssutils (2.3.0): Pending...
  • Installing cssutils (2.3.0): Installing...
  • Installing future (0.18.2): Pending...
  • Installing future (0.18.2): Installing...
  • Installing future (0.18.2): Failed

  EnvCommandError

  Command ['/home/nyuszika7h/test/.venv/bin/pip', 'install', '--no-deps', 'file:///home/nyuszika7h/.cache/pypoetry/artifacts/f8/58/55/86be1f567b212fdd98854d12815964a49db8fb1bcff725018e5f95c61d/future-0.18.2.tar.gz'] errored with the following return code 1, and output:
  Processing /home/nyuszika7h/.cache/pypoetry/artifacts/f8/58/55/86be1f567b212fdd98854d12815964a49db8fb1bcff725018e5f95c61d/future-0.18.2.tar.gz
      ERROR: Command errored out with exit status 1:
       command: /home/nyuszika7h/test/.venv/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-1yncrz1a/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-1yncrz1a/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-gd9atrnx
           cwd: /tmp/pip-req-build-1yncrz1a/
      Complete output (3 lines):
      Traceback (most recent call last):
        File "<string>", line 1, in <module>
      ModuleNotFoundError: No module named 'setuptools'
      ----------------------------------------
  WARNING: Discarding file:///home/nyuszika7h/.cache/pypoetry/artifacts/f8/58/55/86be1f567b212fdd98854d12815964a49db8fb1bcff725018e5f95c61d/future-0.18.2.tar.gz. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  WARNING: You are using pip version 21.1.2; however, version 21.1.3 is available.
  You should consider upgrading via the '/home/nyuszika7h/test/.venv/bin/python -m pip install --upgrade pip' command.


  at ~/.local/share/pypoetry/venv/lib/python3.9/site-packages/poetry/utils/env.py:1101 in _run
      1097│                 output = subprocess.check_output(
      1098│                     cmd, stderr=subprocess.STDOUT, **kwargs
      1099│                 )
      1100│         except CalledProcessError as e:
    → 1101│             raise EnvCommandError(e, input=input_)
      1102│
      1103│         return decode(output)
      1104│
      1105│     def execute(self, bin, *args, **kwargs):

  • Installing lxml (4.6.3): Pending...
  • Installing lxml (4.6.3): Installing...
  • Installing six (1.16.0): Pending...
  • Installing six (1.16.0): Installing...
  • Installing six (1.16.0)
  • Installing cssutils (2.3.0)
  • Installing future (0.18.2): Pending...
  • Installing future (0.18.2): Installing...
  • Installing future (0.18.2): Failed

  EnvCommandError

  Command ['/home/nyuszika7h/test/.venv/bin/pip', 'install', '--no-deps', 'file:///home/nyuszika7h/.cache/pypoetry/artifacts/f8/58/55/86be1f567b212fdd98854d12815964a49db8fb1bcff725018e5f95c61d/future-0.18.2.tar.gz'] errored with the following return code 1, and output:
  Processing /home/nyuszika7h/.cache/pypoetry/artifacts/f8/58/55/86be1f567b212fdd98854d12815964a49db8fb1bcff725018e5f95c61d/future-0.18.2.tar.gz
      ERROR: Command errored out with exit status 1:
       command: /home/nyuszika7h/test/.venv/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-1yncrz1a/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-1yncrz1a/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-gd9atrnx
           cwd: /tmp/pip-req-build-1yncrz1a/
      Complete output (3 lines):
      Traceback (most recent call last):
        File "<string>", line 1, in <module>
      ModuleNotFoundError: No module named 'setuptools'
      ----------------------------------------
  WARNING: Discarding file:///home/nyuszika7h/.cache/pypoetry/artifacts/f8/58/55/86be1f567b212fdd98854d12815964a49db8fb1bcff725018e5f95c61d/future-0.18.2.tar.gz. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  WARNING: You are using pip version 21.1.2; however, version 21.1.3 is available.
  You should consider upgrading via the '/home/nyuszika7h/test/.venv/bin/python -m pip install --upgrade pip' command.


  at ~/.local/share/pypoetry/venv/lib/python3.9/site-packages/poetry/utils/env.py:1101 in _run
      1097│                 output = subprocess.check_output(
      1098│                     cmd, stderr=subprocess.STDOUT, **kwargs
      1099│                 )
      1100│         except CalledProcessError as e:
    → 1101│             raise EnvCommandError(e, input=input_)
      1102│
      1103│         return decode(output)
      1104│
      1105│     def execute(self, bin, *args, **kwargs):

  • Installing lxml (4.6.3): Pending...
  • Installing lxml (4.6.3): Installing...
  • Installing lxml (4.6.3)
  • Installing six (1.16.0): Pending...
  • Installing six (1.16.0): Installing...
  • Installing six (1.16.0)
  • Installing beautifulsoup4 (4.9.3)
  • Installing cssutils (2.3.0): Pending...
  • Installing cssutils (2.3.0): Installing...
  • Installing cssutils (2.3.0)
  • Installing future (0.18.2): Pending...
  • Installing future (0.18.2): Installing...
  • Installing future (0.18.2): Failed

  EnvCommandError

  Command ['/home/nyuszika7h/test/.venv/bin/pip', 'install', '--no-deps', 'file:///home/nyuszika7h/.cache/pypoetry/artifacts/f8/58/55/86be1f567b212fdd98854d12815964a49db8fb1bcff725018e5f95c61d/future-0.18.2.tar.gz'] errored with the following return code 1, and output:
  Processing /home/nyuszika7h/.cache/pypoetry/artifacts/f8/58/55/86be1f567b212fdd98854d12815964a49db8fb1bcff725018e5f95c61d/future-0.18.2.tar.gz
      ERROR: Command errored out with exit status 1:
       command: /home/nyuszika7h/test/.venv/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-1yncrz1a/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-1yncrz1a/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-gd9atrnx
           cwd: /tmp/pip-req-build-1yncrz1a/
      Complete output (3 lines):
      Traceback (most recent call last):
        File "<string>", line 1, in <module>
      ModuleNotFoundError: No module named 'setuptools'
      ----------------------------------------
  WARNING: Discarding file:///home/nyuszika7h/.cache/pypoetry/artifacts/f8/58/55/86be1f567b212fdd98854d12815964a49db8fb1bcff725018e5f95c61d/future-0.18.2.tar.gz. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  WARNING: You are using pip version 21.1.2; however, version 21.1.3 is available.
  You should consider upgrading via the '/home/nyuszika7h/test/.venv/bin/python -m pip install --upgrade pip' command.


  at ~/.local/share/pypoetry/venv/lib/python3.9/site-packages/poetry/utils/env.py:1101 in _run
      1097│                 output = subprocess.check_output(
      1098│                     cmd, stderr=subprocess.STDOUT, **kwargs
      1099│                 )
      1100│         except CalledProcessError as e:
    → 1101│             raise EnvCommandError(e, input=input_)
      1102│
      1103│         return decode(output)
      1104│
      1105│     def execute(self, bin, *args, **kwargs):

  • Installing lxml (4.6.3): Pending...
  • Installing lxml (4.6.3): Installing...
  • Installing lxml (4.6.3)
  • Installing six (1.16.0): Pending...
  • Installing six (1.16.0): Installing...
  • Installing six (1.16.0)

To clarify: Running poetry lock from Poetry 1.2.0a1 seems to cause the lock file to include setuptools, which triggers the issue with Poetry <= 1.1.7. Running poetry lock from Poetry 1.1.7 makes it exclude setuptools and there is no issue.

	Install with Poetry 1.1.7	Install with Poetry 1.2.0a1
Lock file from Poetry 1.1.7	✅	✅
Lock file from Poetry 1.2.0a1	❌	✅

Also, both lock files were generated on Linux. I have not tested on Windows yet.

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 48
Comments: 19 (5 by maintainers)

Commits related to this issue

went back to poetry 1.2.0 but added ~/.local/bin to the system path in the github action the reason for this reversion is described in more detail here: https://github.com/python-poetry/poetry/issues/... — committed to schradert/pyrfume by schradert 2 years ago
Upgrade poetry installed in Github Actions to 1.2.1 https://github.com/python-poetry/poetry/issues/4242#issuecomment-912460384 — committed to inducer/relate by inducer 2 years ago
Upgrade poetry installed in Github Actions to 1.2.1 https://github.com/python-poetry/poetry/issues/4242#issuecomment-912460384 — committed to inducer/relate by inducer 2 years ago
Translate GitHub Action into poetry (#39) * refactor github action to perform linting before testing and use a typical poetry installation workflow * fix syntax errors in github action config *... — committed to pyrfume/pyrfume by schradert 2 years ago
Remove setuptools dependency Looks like this is not needed. This will allow to workaround some problems with build tools like this one https://github.com/python-poetry/poetry/issues/4242 — committed to UncleGoogle/zope.dottedname by UncleGoogle 2 years ago
Remove setuptools dependency Looks like this is not needed. This will allow to workaround some problems with build tools like this one https://github.com/python-poetry/poetry/issues/4242 — committed to UncleGoogle/zope.dottedname by UncleGoogle 2 years ago
Remove setuptools dependency Looks like this is not needed. This will allow to workaround some problems with build tools like this one https://github.com/python-poetry/poetry/issues/4242 — committed to UncleGoogle/zope.dottedname by UncleGoogle 2 years ago
chore: prevent poetry 1.3 from removing setuptools `poetry install` removes untracked packages by default in 1.3, among them setuptools. This causes all subsequent Python invocations (including the ... — committed to AmpelAstro/Ampel-HU-astro by jvansanten a year ago

Most upvoted comments

I still have this problem on 1.2.x

+30

jomach on Nov 30, 2022

Breaking backwards compatibility in a minor release is not the best experience. Is there really no way to make any required changes in a way that it doesn’t lead to mysterious breakage on earlier versions?

+29

0xallie on Sep 3, 2021

For me having explicit dependency on setuptools in normal dependencies does not make a difference, it still removes it.

It completely breaks our Cython builds.

+17

rr- on Jan 17, 2022

So the only issue here is that an older poetry cannot read a lock file generated by the newer version? If that is the case then I think this is expected behavior.

+17

apollo13 on Sep 3, 2021

In my case the issue is reproduced using poetry 1.2.0. The new versions adds in poetry.lock the following lines:

[package.dependencies]
setuptools = "*"

[[package]]
name = "setuptools"
version = "63.4.3"
description = "Easily download, build, install, upgrade, and uninstall Python packages"
category = "main"
optional = false
python-versions = ">=3.7"

Which when a another colleague (or our CI) runs poetry install using poetry 1.1.14 fails as follows:

Installing typing-extensions (4.3.0)
..
EnvCommandError
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'
  error: subprocess-exited-with-error
  × python setup.py egg_info did not run successfully.
  exit code: 1
  ╰─> [1 lines of output]
ERROR: Can not execute `setup.py` since setuptools is not available in the build environment.

antonakospanos on Sep 7, 2022

I’m on poetry 1.1.12 I’ve removed my poetry.lock and recreated it and I’m still facing the issue

rafalkrupinski on Dec 25, 2021

In case anyone are looking for a workaround to make a .lock file generated with Poetry 1.2.x compatible with Poetry 1.1.x, I created a small gist to do so,

curl https://gist.githubusercontent.com/emilhe/0c7b1a33b2d02f17331242bf4fffd07c/raw/8da0665a58f469c980e7661d7f8c36f3bd3af992/strip_setuptools.py | python - && poetry install

In my case, I needed it because I am using Poetry 1.2.x locally (I am having various issues with 1.1.x), while my cloud provider supports only 1.1.x.

emilhe on Jul 18, 2022

Does editing your pyproject.toml to add "setuptools" to the [build-system] requires stop the error from occurring? I don’t know if it’s a fix or a workaround.
[build-system]
-requires = ["poetry_core>=1.0"]
+requires = ["setuptools", "poetry_core>=1.0"]
build-backend = "poetry.core.masonry.api"

Sorry for the late reply about this. I think that fixed my issues before, but with 1.2.0 -> 1.1.x it does not fix the issue. But poetry add setuptools seems to fix it. So I guess the workaround is either that, or not using Poetry 1.2.0 on my end until it’s stable, and telling users to upgrade once it is. But it would be nice if there was a way to tell users that this project requires a specific version of Poetry rather than a scary error message. requires = ['poetry-core>=1.2.0'] does not seem to change anything,

0xallie on Sep 12, 2021

Does editing your pyproject.toml to add "setuptools" to the [build-system] requires stop the error from occurring? I don’t know if it’s a fix or a workaround.

[build-system]
-requires = ["poetry_core>=1.0"]
+requires = ["setuptools", "poetry_core>=1.0"]
build-backend = "poetry.core.masonry.api"

nyanpasu64 on Jul 13, 2021

The delta between 1.2.0 prerelease and 1.1.7 stable: (that is going back to stable)

diff --git a/poetry.lock b/poetry.lock
index dc6b6820f..f0e8e3fdd 100644
--- a/poetry.lock
+++ b/poetry.lock
@@ -499,7 +499,6 @@
 pickleshare = "*"
 prompt-toolkit = ">=2.0.0,<3.0.0 || >3.0.0,<3.0.1 || >3.0.1,<3.1.0"
 pygments = "*"
-setuptools = ">=18.5"
 traitlets = ">=4.2"

 [package.extras]
@@ -1020,18 +1019,6 @@

-[[package]]
-name = "setuptools"
-version = "57.4.0"
-description = "Easily download, build, install, upgrade, and uninstall Python packages"
-category = "dev"
-optional = false
-python-versions = ">=3.6"
-
-[package.extras]
-docs = ["sphinx", "jaraco.packaging (>=8.2)", "rst.linker (>=1.9)", "jaraco.tidelift (>=1.4)", "pygments-github-lexers (==0.0.5)", "sphinx-inline-tabs", "sphinxcontrib-towncrier", "furo"]
-testing = ["pytest (>=4.6)", "pytest-checkdocs (>=2.4)", "pytest-flake8", "pytest-cov", "pytest-enabler (>=1.0.1)", "mock", "flake8-2020", "virtualenv (>=13.0.0)", "pytest-virtualenv (>=1.2.7)", "wheel", "paver", "pip (>=19.1)", "jaraco.envs", "pytest-xdist", "sphinx", "jaraco.path (>=3.2.0)", "pytest-black (>=0.3.7)", "pytest-mypy"]
-
@@ -1082,7 +1069,6 @@
 packaging = "*"
 Pygments = ">=2.0"
 requests = ">=2.0.0"
-setuptools = "*"
 six = ">=1.5"
 snowballstemmer = ">=1.1"
 sphinxcontrib-websupport = "*"
@@ -1994,10 +1980,6 @@
-setuptools = [
-    {file = "setuptools-57.4.0-py3-none-any.whl", hash = "sha256:a49230977aa6cfb9d933614d2f7b79036e9945c4cdd7583163f4e920b83418d6"},
-    {file = "setuptools-57.4.0.tar.gz", hash = "sha256:6bac238ffdf24e8806c61440e755192470352850f3419a52f26ffe0a1a64f465"},
-]
 six = [
     {file = "six-1.15.0-py2.py3-none-any.whl", hash = "sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced"},
     {file = "six-1.15.0.tar.gz", hash = "sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259"},

In other words, 1.2.0 pre includes setuptools into poetry.lock while 1.1.7 does not.

I guess that 1.1.7 wants to build [package wheel? of a] dependency in a clean environment, and that’s why it tries to [temporarily] remove setuptools…

dimaqq on Jul 21, 2021

I am going to close this as it seems this will be resolved in 1.2 and is already on mater.

abn on May 10, 2022

The build-system.requires part is something poetry itself does not use at the moment. This is used by the non-poetry PEP 517 frontends (eg: pip, build etc.)