pybuilder: OSX + homebrew python@3.9: pybuilder/_vendor/virtualenv.py change_prefix does not consider symlinks, raises invalid AssertionError

I have installed Python 3 on macOS 11.3 (Big Sur) with brew install python@3.9. Homebrew installs Python 3.9 under /usr/local/Cellar/python@3.9, and symlinks /usr/local/opt/python@3.9 -> ../Cellar/python@3.9/3.9.4.

python3 is a symlink /usr/local/bin/python3 -> ../Cellar/python@3.9/3.9.4/bin/python3. pip3 is a symlink /usr/local/bin/pip3 -> ../Cellar/python@3.9/3.9.4/bin/pip3.

$ head -1 $(which pip3)
#!/usr/local/opt/python@3.9/bin/python3.9

I used pip3 install --user pybuilder to install pybuilder:

$ which pyb
/Users/username/Library/Python/3.9/bin/pyb
$ head -1 $(which pyb)
#!/usr/local/opt/python@3.9/bin/python3.9
$ pyb
PyBuilder version 0.12.10
Build started at 2021-04-28 10:36:15
------------------------------------------------------------
------------------------------------------------------------
BUILD FAILED - AssertionError: Filename /usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py does not start with any of these prefixes: ['/usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/Extras/lib/python', '/usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9', '/usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9', '/Users/username/Library/Python/3.9/lib/python/site-packages', '/Users/username/.local/lib/python/3.9/site-packages', '/Users/username/Library/Python/3.9/site-packages', '/Library/Python/3.9/site-packages'] (pybuilder/_vendor/virtualenv.py:1272)
------------------------------------------------------------
Build finished at 2021-04-28 10:36:15
Build took 0 seconds (64 ms)

AssertionError is raised from change_prefix:

https://github.com/pybuilder/pybuilder/blob/6ac1ef4e3f504fec7d4f86378fde1c5420d6a261/src/main/python/pybuilder/_vendor/virtualenv.py#L1272

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 1
  • Comments: 39 (37 by maintainers)

Commits related to this issue

Most upvoted comments

Working on other projects, but a quick test with --pre PyBuilder version 0.13.0.dev20210917052200 & Python 3.9.7 seems to indicate that this issue is fixed! Awesome, good job!

I think this can be closed after a release.

Rejoice! I was able to reproduce this issue on Linux 😃 At least the StopIteration part that is.

Same exception with pybuilder installed withpip3 install pybuilder as well.

I run pybuilder mostly with a custom build script in a virtualenv like this:

python3 -m venv vebuild
. vebuild/bin/activate
pip --disable-pip-version-check install pybuilder

But I get the same AssertionError outside virtualenv with pybuilder installed with pip3 install --user pybuilder==0.12.10.

Apologies for not having enough time to dig properly into this.

Maybe @dindurthy could take a look at the GitHub Actions workflow logs and see if there are any differences to local execution?

Not M1.

macOS Big Sur 11.5.2 on MacBook Pro (13-inch, 2019, Four Thunderbolt 3 ports).

$ head -n1 $(which virtualenv)
#!/usr/local/opt/python@3.9/bin/python3.9
$ realpath /usr/local/opt/python@3.9/bin/python3.9
/usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/bin/python3.9
$ virtualenv -vvv --with-traceback /tmp/testvenv
114 setup logging to NOTSET [DEBUG report:43]
140 find interpreter for spec PythonSpec(path=/usr/local/opt/python@3.9/bin/python3.9) [INFO builtin:62]
140 proposed PythonInfo(spec=CPython3.9.7.final.0-64, exe=/usr/local/opt/python@3.9/bin/python3.9, platform=darwin, version='3.9.7 (default, Sep  3 2021, 12:37:55) \n[Clang 12.0.5 (clang-1205.0.22.9)]', encoding_fs_io=utf-8-utf-8) [INFO builtin:69]
140 accepted PythonInfo(spec=CPython3.9.7.final.0-64, exe=/usr/local/opt/python@3.9/bin/python3.9, platform=darwin, version='3.9.7 (default, Sep  3 2021, 12:37:55) \n[Clang 12.0.5 (clang-1205.0.22.9)]', encoding_fs_io=utf-8-utf-8) [DEBUG builtin:71]
143 filesystem is not case-sensitive [DEBUG info:28]
186 create virtual environment via CPython3Posix(dest=/private/tmp/testvenv, clear=False, no_vcs_ignore=False, global=False) [INFO session:52]
186 create folder /private/tmp/testvenv/bin [DEBUG _sync:25]
186 create folder /private/tmp/testvenv/lib/python3.9/site-packages [DEBUG _sync:25]
187 write /private/tmp/testvenv/pyvenv.cfg [DEBUG pyenv_cfg:34]
187     home = /usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9 [DEBUG pyenv_cfg:38]
187     implementation = CPython [DEBUG pyenv_cfg:38]
187     version_info = 3.9.7.final.0 [DEBUG pyenv_cfg:38]
187     virtualenv = 20.7.2 [DEBUG pyenv_cfg:38]
187     include-system-site-packages = false [DEBUG pyenv_cfg:38]
187     base-prefix = /usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9 [DEBUG pyenv_cfg:38]
187     base-exec-prefix = /usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9 [DEBUG pyenv_cfg:38]
187     base-executable = /usr/local/opt/python@3.9/bin/python3.9 [DEBUG pyenv_cfg:38]
187 symlink /usr/local/opt/python@3.9/bin/python3.9 to /private/tmp/testvenv/bin/python [DEBUG _sync:44]
188 create virtualenv import hook file /private/tmp/testvenv/lib/python3.9/site-packages/_virtualenv.pth [DEBUG api:95]
189 create /private/tmp/testvenv/lib/python3.9/site-packages/_virtualenv.py [DEBUG api:98]
190 ============================== target debug ============================== [DEBUG session:54]
190 debug via /private/tmp/testvenv/bin/python /Users/username/Library/Python/3.9/lib/python/site-packages/virtualenv/create/debug.py [DEBUG creator:224]
190 {
  "sys": {
    "executable": "/private/tmp/testvenv/bin/python",
    "_base_executable": "/private/tmp/testvenv/bin/python",
    "prefix": "/private/tmp/testvenv",
    "base_prefix": "/usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9",
    "real_prefix": null,
    "exec_prefix": "/private/tmp/testvenv",
    "base_exec_prefix": "/usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9",
    "path": [
      "/usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python39.zip",
      "/usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9",
      "/usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload",
      "/private/tmp/testvenv/lib/python3.9/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.9.7 (default, Sep  3 2021, 12:37:55) \n[Clang 12.0.5 (clang-1205.0.22.9)]",
  "makefile_filename": "/usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/config-3.9-darwin/Makefile",
  "os": "<module 'os' from '/usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py'>",
  "site": "<module 'site' from '/usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site.py'>",
  "datetime": "<module 'datetime' from '/usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/datetime.py'>",
  "math": "<module 'math' from '/usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload/math.cpython-39-darwin.so'>",
  "json": "<module 'json' from '/usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py'>"
} [DEBUG session:55]
248 add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/username/Library/Application Support/virtualenv) [INFO session:59]
251 got embed update of distribution setuptools from /Users/username/Library/Application Support/virtualenv/wheel/3.9/embed/1/setuptools.json [DEBUG via_disk_folder:135]
252 got embed update of distribution wheel from /Users/username/Library/Application Support/virtualenv/wheel/3.9/embed/1/wheel.json [DEBUG via_disk_folder:135]
252 got embed update of distribution pip from /Users/username/Library/Application Support/virtualenv/wheel/3.9/embed/1/pip.json [DEBUG via_disk_folder:135]
256 got embed update of distribution setuptools from /Users/username/Library/Application Support/virtualenv/wheel/3.9/embed/1/setuptools.json [DEBUG via_disk_folder:135]
256 got embed update of distribution pip from /Users/username/Library/Application Support/virtualenv/wheel/3.9/embed/1/pip.json [DEBUG via_disk_folder:135]
256 got embed update of distribution wheel from /Users/username/Library/Application Support/virtualenv/wheel/3.9/embed/1/wheel.json [DEBUG via_disk_folder:135]
257 install setuptools from wheel /Users/username/Library/Python/3.9/lib/python/site-packages/virtualenv/seed/wheels/embed/setuptools-57.4.0-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
257 install pip from wheel /Users/username/Library/Python/3.9/lib/python/site-packages/virtualenv/seed/wheels/embed/pip-21.2.3-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
257 install wheel from wheel /Users/username/Library/Python/3.9/lib/python/site-packages/virtualenv/seed/wheels/embed/wheel-0.37.0-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:49]
258 Attempting to acquire lock 4507837008 on /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any.lock [DEBUG filelock:270]
258 Attempting to acquire lock 4507839552 on /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-21.2.3-py3-none-any.lock [DEBUG filelock:270]
258 Attempting to acquire lock 4507839648 on /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.37.0-py2.py3-none-any.lock [DEBUG filelock:270]
259 Lock 4507839552 acquired on /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-21.2.3-py3-none-any.lock [INFO filelock:274]
259 Lock 4507839648 acquired on /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.37.0-py2.py3-none-any.lock [INFO filelock:274]
259 build install image for pip-21.2.3-py3-none-any.whl to /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-21.2.3-py3-none-any [DEBUG base:52]
259 Lock 4507837008 acquired on /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any.lock [INFO filelock:274]
259 build install image for wheel-0.37.0-py2.py3-none-any.whl to /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.37.0-py2.py3-none-any [DEBUG base:52]
259 build install image for setuptools-57.4.0-py3-none-any.whl to /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any [DEBUG base:52]
287 Attempting to release lock 4507839648 on /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.37.0-py2.py3-none-any.lock [DEBUG filelock:315]
287 Lock 4507839648 released on /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.37.0-py2.py3-none-any.lock [INFO filelock:318]
288 copy directory /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.37.0-py2.py3-none-any/wheel-0.37.0.dist-info to /private/tmp/testvenv/lib/python3.9/site-packages/wheel-0.37.0.dist-info [DEBUG _sync:52]
335 copy directory /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.37.0-py2.py3-none-any/wheel to /private/tmp/testvenv/lib/python3.9/site-packages/wheel [DEBUG _sync:52]
412 Attempting to release lock 4507837008 on /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any.lock [DEBUG filelock:315]
412 Lock 4507837008 released on /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any.lock [INFO filelock:318]
413 copy directory /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any/setuptools-57.4.0.dist-info to /private/tmp/testvenv/lib/python3.9/site-packages/setuptools-57.4.0.dist-info [DEBUG _sync:52]
414 copy /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.37.0-py2.py3-none-any/wheel-0.37.0.virtualenv to /private/tmp/testvenv/lib/python3.9/site-packages/wheel-0.37.0.virtualenv [DEBUG _sync:52]
417 generated console scripts wheel-3.9 wheel3 wheel wheel3.9 [DEBUG base:48]
427 copy /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any/distutils-precedence.pth to /private/tmp/testvenv/lib/python3.9/site-packages/distutils-precedence.pth [DEBUG _sync:52]
432 copy /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any/setuptools-57.4.0.virtualenv to /private/tmp/testvenv/lib/python3.9/site-packages/setuptools-57.4.0.virtualenv [DEBUG _sync:52]
433 copy directory /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any/setuptools to /private/tmp/testvenv/lib/python3.9/site-packages/setuptools [DEBUG _sync:52]
529 Attempting to release lock 4507839552 on /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-21.2.3-py3-none-any.lock [DEBUG filelock:315]
529 Lock 4507839552 released on /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-21.2.3-py3-none-any.lock [INFO filelock:318]
530 copy /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-21.2.3-py3-none-any/pip-21.2.3.virtualenv to /private/tmp/testvenv/lib/python3.9/site-packages/pip-21.2.3.virtualenv [DEBUG _sync:52]
530 copy directory /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-21.2.3-py3-none-any/pip to /private/tmp/testvenv/lib/python3.9/site-packages/pip [DEBUG _sync:52]
657 copy directory /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any/pkg_resources to /private/tmp/testvenv/lib/python3.9/site-packages/pkg_resources [DEBUG _sync:52]
684 copy directory /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-57.4.0-py3-none-any/_distutils_hack to /private/tmp/testvenv/lib/python3.9/site-packages/_distutils_hack [DEBUG _sync:52]
688 generated console scripts  [DEBUG base:48]
1776 copy directory /Users/username/Library/Application Support/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-21.2.3-py3-none-any/pip-21.2.3.dist-info to /private/tmp/testvenv/lib/python3.9/site-packages/pip-21.2.3.dist-info [DEBUG _sync:52]
1785 generated console scripts pip-3.9 pip3 pip3.9 pip [DEBUG base:48]
1786 add activators for Bash, CShell, Fish, PowerShell, Python [INFO session:64]
1789 write /private/tmp/testvenv/pyvenv.cfg [DEBUG pyenv_cfg:34]
1789    home = /usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9 [DEBUG pyenv_cfg:38]
1789    implementation = CPython [DEBUG pyenv_cfg:38]
1789    version_info = 3.9.7.final.0 [DEBUG pyenv_cfg:38]
1789    virtualenv = 20.7.2 [DEBUG pyenv_cfg:38]
1789    include-system-site-packages = false [DEBUG pyenv_cfg:38]
1789    base-prefix = /usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9 [DEBUG pyenv_cfg:38]
1789    base-exec-prefix = /usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9 [DEBUG pyenv_cfg:38]
1789    base-executable = /usr/local/opt/python@3.9/bin/python3.9 [DEBUG pyenv_cfg:38]
1790 created virtual environment CPython3.9.7.final.0-64 in 1677ms
  creator CPython3Posix(dest=/private/tmp/testvenv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/username/Library/Application Support/virtualenv)
    added seed packages: pip==21.2.3, setuptools==57.4.0, wheel==0.37.0
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator [WARNING __main__:19]