virtualenv: Scripts installed with wrong shebang under subprocess
On Python 3.5.1 but not on 2.7.10, when I invoke pip through a subprocess in a virtualenv, the scripts are created with the wrong shebang and thus fail to run:
$ cat > invoke.py
import sys
import subprocess
subprocess.Popen(sys.argv[1:]).wait()
$ rm -Rf env
$ python -m virtualenv --version
14.0.0
$ python --version
Python 3.5.1
$ python -m virtualenv env
Using base prefix '/Library/Frameworks/Python.framework/Versions/3.5'
New python executable in /Users/jaraco/Dropbox/code/public/aspen/env/bin/python3
Also creating executable in /Users/jaraco/Dropbox/code/public/aspen/env/bin/python
Installing setuptools, pip, wheel...done.
$ python invoke.py env/bin/pip install pytest
Collecting pytest
Using cached pytest-2.8.5-py2.py3-none-any.whl
Collecting py>=1.4.29 (from pytest)
Using cached py-1.4.31-py2.py3-none-any.whl
Installing collected packages: py, pytest
Successfully installed py-1.4.31 pytest-2.8.5
$ head -n 1 env/bin/py.test
#!/Library/Frameworks/Python.framework/Versions/3.5/bin/python3
One would expect the py.test script to have a shebang with ‘env’ in it.
Using pyvenv instead is not subject to the issue. Removing the __PYVENV_LAUNCHER__ environment variable before launching the subprocess works around the issue, as reported here. The issue was observed with 13.1.2 and 14.0.0.
Is this behavior expected? If so, is removing the environment variable the appropriate thing for a parent process to do to work around the issue?
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 27 (15 by maintainers)
I was just bitten by this exact bug. It’s quite a severe bug in my opinion, mostly because it’s so extremely confusing and hard to figure out when it hits.
Some thoughts:
__PYVENV_LAUNCHER__was added because for earlier versions of Python (< 3.3, according to Ronald) but not for later versions, Python’ssite.pyon OS X needed to distinguish the location of stub launchers from framework executables. According to this comment by Ned Deily,sys.executablenow points to the stub launcher, and this is whydistlibstopped using__PYVENV_LAUNCHER__. The only usage of it now is insite.py- if available, on OS X, it’s used to locate thepyvenv.cfgfile.os.path.realpath()call when it shouldn’t be. It doesn’t look like it’sdistlibdoing that. If (as per Ned’s comment)sys.executableand the env var are always the same, then it shouldn’t matter; but if something does arealpathon asys.executableand executes Python through the result, then that would result in an unexpected shebang (becausesys.executablewould then be a dereferenced path).