setuptools: [BUG] setuptools>=60.0.0 causes virtualenv to create abnormal tree structure on ubuntu 22.04
setuptools version
setuptools 60, 61, 62
59.x was the last version I found to not cause trouble. This also happens to be the default version installed with ubuntu 22.04 via apt.
Python version
Python 3.10
OS
Ubuntu 22.04
Also, I was unable to reproduce this on python:3.10-buster
Additional environment information
No response
Description
Installing setuptools==62.1.0 causes virtualenv venv to create an abnormal directory structure. Uninstalling setuptools solves the problem. This is 100% reproducible in ubuntu:22.04 docker image (ubuntu@sha256:06b5d30fabc1fc574f2ecab87375692299d45f8f190d9b71f512deb494114e1f)
The failure of virtualenv also affects poetry, which has no error checking for venv activate and will attempt to modify system packages when it thinks the venv is activated but it is not.
Without setuptools, venv/bin exists. With setuptools, venv/local/bin exists and no venv/bin dir exists.
Expected behavior
Setuptools does not interfere with virtualenv directory structure.
How to Reproduce
- Create Dockerfile:
FROM ubuntu:22.04 RUN apt-get update RUN apt-get install -y python3 python3-pip python3-virtualenv tree RUN pip install setuptools==62.1.0 ; \ virtualenv venv ; \ tree -L 3 venv ; \ . venv/bin/activate - Build it.
- Notice error.
Output
$ cat Dockerfile
FROM ubuntu:22.04
RUN apt-get update
RUN apt-get install -y python3 python3-pip python3-virtualenv tree
RUN pip install setuptools==62.1.0 ; \
virtualenv venv ; \
tree -L 3 venv ; \
. venv/bin/activate
$ docker build -t test1 .
Sending build context to Docker daemon 24.01MB
Step 1/4 : FROM ubuntu:22.04
---> f0b07b45d05b
Step 2/4 : RUN apt-get update
---> Using cache
---> f3a881e3158b
Step 3/4 : RUN apt-get install -y python3 python3-pip python3-virtualenv tree
---> Using cache
---> 1b02afb3b2a1
Step 4/4 : RUN pip install setuptools==62.1.0 ; virtualenv venv ; tree -L 3 venv ; . venv/bin/activate
---> Running in 5a7b54dc3a32
Collecting setuptools==62.1.0
Downloading setuptools-62.1.0-py3-none-any.whl (1.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 8.8 MB/s eta 0:00:00
Installing collected packages: setuptools
Attempting uninstall: setuptools
Found existing installation: setuptools 59.6.0
Not uninstalling setuptools at /usr/lib/python3/dist-packages, outside environment /usr
Can't uninstall 'setuptools'. No files were found to uninstall.
Successfully installed setuptools-62.1.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
created virtual environment CPython3.10.4.final.0-64 in 308ms
creator CPython3Posix(dest=/venv, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
added seed packages: pip==22.0.2, setuptools==59.6.0, wheel==0.37.1
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
venv
|-- lib
| `-- python3.10
|-- local
| |-- bin
| | |-- activate
| | |-- activate.csh
| | |-- activate.fish
| | |-- activate.nu
| | |-- activate.ps1
| | |-- activate_this.py
| | |-- deactivate.nu
| | |-- pip
| | |-- pip-3.10
| | |-- pip3
| | |-- pip3.10
| | |-- python -> /usr/bin/python3
| | |-- python3 -> python
| | |-- python3.10 -> python
| | |-- wheel
| | |-- wheel-3.10
| | |-- wheel3
| | `-- wheel3.10
| `-- lib
| `-- python3.10
`-- pyvenv.cfg
6 directories, 19 files
/bin/sh: 1: .: cannot open venv/bin/activate: No such file
The command '/bin/sh -c pip install setuptools==62.1.0 ; virtualenv venv ; tree -L 3 venv ; . venv/bin/activate' returned a non-zero code: 2
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 14
- Comments: 19 (8 by maintainers)
Links to this issue
Commits related to this issue
- WIP pactivate: proposed fix for Debian Python 3.10 pkg w/o distutils *** Tests (all manual so far): - Ubuntu 22.04 / python3 3.10 / NO python3-distutils: ** problem fixed - Ubuntu 22.04 / python3 ... — committed to cynic-net/pactivate by deleted user 4 years ago
- WIP pactivate: proposed fix for Debian Python 3.10 pkg w/o distutils *** Tests (all manual so far): - Ubuntu 22.04 / python3 3.10 / NO python3-distutils: ** problem fixed - Ubuntu 22.04 / python3 ... — committed to cynic-net/pactivate by 0cjs 4 years ago
- WIP pactivate: proposed fix for Debian Python 3.10 pkg w/o distutils *** Tests (all manual so far): - Ubuntu 22.04 / python3 3.10 / NO python3-distutils: ** problem fixed - Ubuntu 22.04 / python3 ... — committed to cynic-net/pactivate by 0cjs 4 years ago
- WIP pactivate: proposed fix for Debian Python 3.10 pkg w/o distutils *** Tests (all manual so far): ----- Debian package ------ Tested on python3 python3-distutils resu... — committed to cynic-net/pactivate by 0cjs 4 years ago
- Work-a-round for https://github.com/pypa/setuptools/issues/3278 Create .venv, so that Poetry will use it. See: https://github.com/pypa/setuptools/issues/3278#issuecomment-1312803069 — committed to jedie/cookiecutter_templates by jedie 4 years ago
- Work around https://github.com/pypa/setuptools/issues/3278 — committed to UniversalSuperBox/drc-sim by UniversalSuperBox 4 years ago
- use an earlier version of setuptools because modern versions have a bug https://github.com/pypa/setuptools/issues/3278 — committed to green-coding-solutions/green-metrics-tool by dan-mm 4 years ago
Having this issue on WSL Ubuntu 22.04. This
export SETUPTOOLS_USE_DISTUTILS=stdlibworks around the issue.I just upgraded multiple computers to Ubuntu 22.04 and all of them have this problem. Replacing
os.environ.get(var, 'local')withos.environ.get(var, 'stdlib')fixes the problem.I’ve filed a bug with Debian about this problem, since as far as I can tell nobody explicitly informed them that there is a problem: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1022749
I used this dockerfile:
Because it allows me to see the failed expectation but also replace it and inspect the environment.
The problem can be observed by adding a breakpoint to
virtualenv/discovery/py_info.py:88:And inspecting the value:
The fact that
scriptsislocal/binis probably what is leading virtualenv to install executables to that directory.My guess is that something about the assumptions made between
_distutils_system_modandvirtualenvare in conflict.Probably what we need is someone from virtualenv to analyze what expectation is being missed here and determine what’s the best way to meet that expectation. @gaborbernat Do you have any advice here?
I also observe that virtualenv has a codepath that bypasses distutils if a
venvscheme is supplied. Perhaps the best thing would be for debian to supply that install scheme. @FFY00 Do you have a recommendation on what Debian should be doing to support a virtualenv?@doko42, It does appear to me you authored the patch. Will you be able to help work through this issue?
setuptools 60 didn’t ship in Ubuntu 22.04, it came later in 23.04.
The setuptools solution to this problem is the addition of the
venvscheme to the python interpreter. That isn’t something that is reasonable to do in a stable Ubuntu, to resolve a bug like this, that occurs only when a package is updated to a version newer than Ubuntu ships. That’s outside of what’s reasonable for Ubuntu support.I suggest leaving your system setuptools as shipped by Ubuntu, and using the system
virtualenvto create virtualenvs. Use the newer setuptools inside virtualenvs.Or use
venvinstead ofvirtualenv.Workaround / fix
I have verified with 62.1.0 that changing
$HOME/.local/lib/python3.10/site-packages/distutils-precedence.pthfrom:to
Solves the problem.
See also