setuptools_scm: pip 23.0.1 and anyio==4.0.0 break installation with --upgrade --no-binary :all: options

Description

When trying to install dependency anyio, setuptools_scm installation is broken when using options --upgrade --no-binary :all: pip version is 23.0.1 Error observed:


LookupError: https://files.pythonhosted.org/packages/a7/16/8aef42ae27036ceb201f1774940357da18ae7daac45bdd721da554892012/setuptools-scm-8.0.1.tar.gz (from https://pypi.org/simple/setuptools-scm/) (requires-python:>=3.8) is already being built: setuptools_scm>=6.4 from https://files.pythonhosted.org/packages/a7/16/8aef42ae27036ceb201f1774940357da18ae7daac45bdd721da554892012/setuptools-scm-8.0.1.tar.gz

Steps to reproduce

pip3 install --upgrade --no-binary :all: anyio==4.0.0
Collecting anyio==4.0.0
  Using cached anyio-4.0.0.tar.gz (153 kB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error
Full log
$ pip3 install --upgrade --no-binary :all: anyio==4.0.0
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
Collecting anyio==4.0.0
  Using cached anyio-4.0.0.tar.gz (153 kB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [116 lines of output]
      DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
      Collecting setuptools>=64
        Using cached setuptools-68.2.2.tar.gz (2.2 MB)
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Installing backend dependencies: started
        Installing backend dependencies: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting setuptools_scm>=6.4
        Using cached setuptools-scm-8.0.1.tar.gz (71 kB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'error'
        error: subprocess-exited-with-error
      
        × pip subprocess to install build dependencies did not run successfully.
        │ exit code: 1
        ╰─> [85 lines of output]
            DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
            Ignoring typing_extensions: markers 'python_version < "3.8"' don't match your environment
            Collecting importlib-metadata>=4.6
              Using cached importlib_metadata-6.8.0.tar.gz (53 kB)
              Installing build dependencies: started
              Installing build dependencies: finished with status 'error'
              error: subprocess-exited-with-error
      
              × pip subprocess to install build dependencies did not run successfully.
              │ exit code: 2
              ╰─> [61 lines of output]
                  DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
                  Collecting setuptools>=56
                    Using cached setuptools-68.2.2.tar.gz (2.2 MB)
                    Getting requirements to build wheel: started
                    Getting requirements to build wheel: finished with status 'done'
                    Installing backend dependencies: started
                    Installing backend dependencies: finished with status 'done'
                    Preparing metadata (pyproject.toml): started
                    Preparing metadata (pyproject.toml): finished with status 'done'
                  Collecting setuptools_scm[toml]>=3.4.1
                    Using cached setuptools-scm-8.0.1.tar.gz (71 kB)
                  ERROR: Exception:
                  Traceback (most recent call last):
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
                      status = run_func(*args)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
                      return func(self, options, args)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 419, in run
                      requirement_set = resolver.resolve(
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
                      result = self._result = resolver.resolve(
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
                      state = resolution.resolve(requirements, max_rounds=max_rounds)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 348, in resolve
                      self._add_to_criteria(self.state.criteria, r, parent=None)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
                      if not criterion.candidates:
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
                      return bool(self._sequence)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
                      return any(self)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
                      return (c for c in iterator if id(c) not in self._incompatible_ids)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
                      candidate = func()
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
                      self._link_candidate_cache[link] = LinkCandidate(
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 297, in __init__
                      super().__init__(
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 162, in __init__
                      self.dist = self._prepare()
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 231, in _prepare
                      dist = self._prepare_distribution()
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
                      return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 491, in prepare_linked_requirement
                      return self._prepare_linked_requirement(req, parallel_builds)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 577, in _prepare_linked_requirement
                      dist = _get_prepared_distribution(
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 68, in _get_prepared_distribution
                      with build_tracker.track(req):
                    File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
                      return next(self.gen)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/operations/build/build_tracker.py", line 122, in track
                      self.add(req)
                    File "/home/ichvets/testenv/lib/python3.8/site-packages/pip/_internal/operations/build/build_tracker.py", line 92, in add
                      raise LookupError(message)
                  LookupError: https://files.pythonhosted.org/packages/a7/16/8aef42ae27036ceb201f1774940357da18ae7daac45bdd721da554892012/setuptools-scm-8.0.1.tar.gz (from https://pypi.org/simple/setuptools-scm/) (requires-python:>=3.8) is already being built: setuptools_scm>=6.4 from https://files.pythonhosted.org/packages/a7/16/8aef42ae27036ceb201f1774940357da18ae7daac45bdd721da554892012/setuptools-scm-8.0.1.tar.gz
      
                  [notice] A new release of pip is available: 23.0.1 -> 23.2.1
                  [notice] To update, run: pip install --upgrade pip
                  [end of output]
      
              note: This error originates from a subprocess, and is likely not a problem with pip.
            error: subprocess-exited-with-error
      
            × pip subprocess to install build dependencies did not run successfully.
            │ exit code: 2
            ╰─> See above for output.
      
            note: This error originates from a subprocess, and is likely not a problem with pip.
      
            [notice] A new release of pip is available: 23.0.1 -> 23.2.1
            [notice] To update, run: pip install --upgrade pip
            [end of output]
      
        note: This error originates from a subprocess, and is likely not a problem with pip.
      error: subprocess-exited-with-error
      
      × pip subprocess to install build dependencies did not run successfully.
      │ exit code: 1
      ╰─> See above for output.
      
      note: This error originates from a subprocess, and is likely not a problem with pip.
      
      [notice] A new release of pip is available: 23.0.1 -> 23.2.1
      [notice] To update, run: pip install --upgrade pip
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Expected behaviour

setuptools_scm is installed with no error messages.

Workaround

Not available.

About this issue

  • Original URL
  • State: closed
  • Created 9 months ago
  • Reactions: 2
  • Comments: 20 (10 by maintainers)

Commits related to this issue

Most upvoted comments

merge and release tommorow in the morning

Just verified and it still fails with 8.0.2 release.Can we re-open issue to track this?

Full log of test:

$ pip3 install --upgrade --no-binary :all: anyio==4.0.0
Collecting anyio==4.0.0
  Downloading anyio-4.0.0.tar.gz (153 kB)
     |████████████████████████████████| 153 kB 4.0 MB/s 
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-hyjnx6vc/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools >= 64' 'setuptools_scm >= 6.4'
       cwd: None
  Complete output (62 lines):
  Collecting setuptools>=64
    Downloading setuptools-68.2.2.tar.gz (2.2 MB)
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Installing backend dependencies: started
    Installing backend dependencies: finished with status 'done'
      Preparing wheel metadata: started
      Preparing wheel metadata: finished with status 'done'
  Collecting setuptools_scm>=6.4
    Downloading setuptools-scm-8.0.2.tar.gz (72 kB)
    Installing build dependencies: started
    Installing build dependencies: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-hov0ps1j/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'importlib-metadata>=4.6; python_version < "3.10"' rich 'setuptools>=61' 'tomli; python_version < "3.11"' 'typing_extensions; python_version < "3.8"'
         cwd: None
    Complete output (44 lines):
    Ignoring typing-extensions: markers 'python_version < "3.8"' don't match your environment
    Collecting importlib-metadata>=4.6
      Downloading importlib_metadata-6.8.0.tar.gz (53 kB)
      Installing build dependencies: started
      Installing build dependencies: finished with status 'error'
      ERROR: Command errored out with exit status 2:
       command: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-m7mg97oz/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=56' 'setuptools_scm[toml]>=3.4.1'
           cwd: None
      Complete output (33 lines):
      Collecting setuptools>=56
        Using cached setuptools-68.2.2.tar.gz (2.2 MB)
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Installing backend dependencies: started
        Installing backend dependencies: finished with status 'done'
          Preparing wheel metadata: started
          Preparing wheel metadata: finished with status 'done'
      Collecting setuptools_scm[toml]>=3.4.1
        Using cached setuptools-scm-8.0.2.tar.gz (72 kB)
      ERROR: Exception:
      Traceback (most recent call last):
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
          status = self.run(options, args)
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 357, in run
          resolver.resolve(requirement_set)
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve
          discovered_reqs.extend(self._resolve_one(requirement_set, req))
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one
          abstract_dist = self._get_abstract_dist_for(req_to_install)
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_for
          abstract_dist = self.preparer.prepare_linked_requirement(req)
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 515, in prepare_linked_requirement
          abstract_dist = _get_prepared_distribution(
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 94, in _get_prepared_distribution
          with req_tracker.track(req):
        File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
          return next(self.gen)
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/req/req_tracker.py", line 148, in track
          self.add(req)
        File "/home/ubuntu/testenv/lib/python3.8/site-packages/pip/_internal/req/req_tracker.py", line 115, in add
          raise LookupError(message)
      LookupError: https://files.pythonhosted.org/packages/51/11/58d3bdef61f2090330292b52d390d57b76dfc08f003174952eff0fafa10e/setuptools-scm-8.0.2.tar.gz#sha256=e45c8c87719b753b6d47cf09907d1239540c7e150cd44f06f658b602f402b005 (from https://pypi.org/simple/setuptools-scm/) (requires-python:>=3.8) is already being built: setuptools_scm>=6.4 from https://files.pythonhosted.org/packages/51/11/58d3bdef61f2090330292b52d390d57b76dfc08f003174952eff0fafa10e/setuptools-scm-8.0.2.tar.gz#sha256=e45c8c87719b753b6d47cf09907d1239540c7e150cd44f06f658b602f402b005
      ----------------------------------------
    ERROR: Command errored out with exit status 2: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-m7mg97oz/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=56' 'setuptools_scm[toml]>=3.4.1' Check the logs for full command output.
    ----------------------------------------
  ERROR: Command errored out with exit status 1: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-hov0ps1j/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'importlib-metadata>=4.6; python_version < "3.10"' rich 'setuptools>=61' 'tomli; python_version < "3.11"' 'typing_extensions; python_version < "3.8"' Check the logs for full command output.
  ----------------------------------------
ERROR: Command errored out with exit status 1: /home/ubuntu/testenv/bin/python3 /home/ubuntu/testenv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-hyjnx6vc/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools >= 64' 'setuptools_scm >= 6.4' Check the logs for full command output.

Everything seems to be working now, thanks!

@addyess same hack works

@RonnyPfannschmidt how about python 3.8?

i found a hack that resolves it, mr incoming

I identified that a pip downgrade IS necessary atm

I’ve got one that demonstrates the issue with 8.0.2:

# syntax=docker/dockerfile:1

FROM debian:11

RUN apt-get update && apt-get install -y python3-pip
RUN pip3 install --upgrade pip==23.2.1
RUN pip3 --version
RUN pip3 install "anyio==4.0.0" --no-binary :all:

When running the docker build, if you add in the env var BUILDKIT_PROGRESS=plain, you can see the full output, including the pip3 --version command, which reports: pip 23.2.1 from /usr/local/lib/python3.9/dist-packages/pip (python 3.9)

EDIT: Probably better to replace that last line with RUN pip3 install "setuptools_scm==8.0.2" --no-binary :all: to get to the heart of the issue.

@i-chvets so the current recommendation is a pip upgrade it seems

packaging will be removed as build dependency and i’ll use the setuptools vendored one in that case

this cycle was previously missed, im not quite sure how to unravel