pipenv: 2023.8.19 through to 2023.8.22 break editable package installation

Issue description

Editable package installation appears to be broken from pipenv 2023.8.19 onwards. For example, given a package in the directory test-hello under the pipenv root installing the package with pipenv install -e ./test-hello fails.

Expected result

pipenv 2023.7.23 and prior versions would install editable packages in Pipfile and in the venv.

Actual result

Here’s a log of package installation across recent versions of pipenv. try_pipenv_versions.log

For example:

+ pipx install pipenv==2023.8.19
creating virtual environment...
installing pipenv from spec 'pipenv==2023.8.19'...
done! ✨ 🌟 ✨
  installed package pipenv 2023.8.19, installed using Python 3.8.10
  These apps are now globally available
    - pipenv
    - pipenv-resolver
+ git clean -dfx
Removing .venv/
Removing applications/test-hello/test_hello.egg-info/
Removing test-hello/test_hello.egg-info/
+ echo '=== Using pipenv version 2023.8.19 ==='
=== Using pipenv version 2023.8.19 ===
+ pipenv sync
Creating a virtualenv for this project...
Pipfile: /home/stewart/src/pipenv_editable_install_repro/Pipfile
Using /usr/bin/python3.8 (3.8.10) to create virtualenv...
created virtual environment CPython3.8.10.final.0-64 in 48ms
  creator CPython3Posix(dest=/home/stewart/src/pipenv_editable_install_repro/.venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/stewart/.local/share/virtualenv)
    added seed packages: pip==23.2.1, setuptools==68.0.0, wheel==0.41.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /home/stewart/src/pipenv_editable_install_repro/.venv
Installing dependencies from Pipfile.lock (db4242)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
All dependencies are now up-to-date!
+ install_test_and_uninstall_package ./test-hello
+ path=./test-hello
+ pipenv run pipenv install -e ./test-hello
Installing -e ./test-hello...
Resolving -e ./test-hello...
Error: Traceback (most recent call last):
  File "/home/stewart/.local/pipx/venvs/pipenv/lib/python3.8/site-packages/pipenv/routines/install.py", line 
279, in do_install
    added, cat, normalized_name = project.add_package_to_pipfile(
  File "/home/stewart/.local/pipx/venvs/pipenv/lib/python3.8/site-packages/pipenv/project.py", line 1159, in 
add_package_to_pipfile
    name, normalized_name, entry = self.generate_package_pipfile_entry(
  File "/home/stewart/.local/pipx/venvs/pipenv/lib/python3.8/site-packages/pipenv/project.py", line 1110, in 
generate_package_pipfile_entry
    req_name = determine_package_name(package)
  File "/home/stewart/.local/pipx/venvs/pipenv/lib/python3.8/site-packages/pipenv/utils/dependencies.py", line 
745, in determine_package_name
    raise ValueError(f"Could not determine package name from {package}")
ValueError: Could not determine package name from 
file:///file:///home/stewart/src/pipenv_editable_install_repro/test-hello

✘ Failed adding package to Pipfile
✔ Installation Succeeded

It’s also interesting that installation fails, due to the mangled URI that starts with file:///file:///, but the final message is that installation succeeded.

Steps to replicate

https://github.com/stewartmiles/pipenv_editable_install_repro contains a minimal test environment. I generated the log file using:

git clone https://github.com/stewartmiles/pipenv_editable_install_repro
cd pipenv_editable_install_repro
./try_pipenv_versions.sh 2>&1 | tee ../try_pipenv_versions.log

Please run $ pipenv --support, and paste the results here. Don’t put backticks (`) around it! The output already contains Markdown formatting.

$ pipenv --support

Pipenv version: '2023.8.22'

Pipenv location: '/home/stewart/.local/pipx/venvs/pipenv/lib/python3.8/site-packages/pipenv'

Python location: '/home/stewart/.local/pipx/venvs/pipenv/bin/python'

OS Name: 'posix'

User pip version: '23.2.1'

user Python installations found:

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.8.10',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.15.0-69-generic',
 'platform_system': 'Linux',
 'platform_version': '#76~20.04.1-Ubuntu SMP Mon Mar 20 15:54:19 UTC 2023',
 'python_full_version': '3.8.10',
 'python_version': '3.8',
 'sys_platform': 'linux'}

System environment variables:

  • SHELL
  • PIPENV_VENV_IN_PROJECT
  • TERMCAP
  • SSH_AUTH_SOCK
  • WINDOW
  • SSH_AGENT_PID
  • PWD
  • LOGNAME
  • XDG_SESSION_TYPE
  • MOTD_SHOWN
  • HOME
  • LANG
  • LS_COLORS
  • SSH_CONNECTION
  • LESSCLOSE
  • XDG_SESSION_CLASS
  • TERM
  • LESSOPEN
  • USER
  • XLA_FLAGS
  • SHLVL
  • XDG_SESSION_ID
  • KUBECONFIG
  • LD_LIBRARY_PATH
  • XDG_RUNTIME_DIR
  • SSH_CLIENT
  • XDG_DATA_DIRS
  • PATH
  • STY
  • DBUS_SESSION_BUS_ADDRESS
  • SSH_TTY
  • USE_GKE_GCLOUD_AUTH_PLUGIN
  • OLDPWD
  • _
  • PIP_DISABLE_PIP_VERSION_CHECK
  • PYTHONDONTWRITEBYTECODE
  • PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

  • PIPENV_VENV_IN_PROJECT: true

Debug–specific environment variables:

  • PATH: ~/.local/bin:/home/stewart/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
  • SHELL: /bin/bash
  • LANG: en_US.UTF-8
  • PWD: /home/stewart/src/pipenv_editable_install_repro

Contents of Pipfile (‘/home/stewart/src/pipenv_editable_install_repro/Pipfile’):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]

[dev-packages]

[requires]
python_version = "3.8"

Contents of Pipfile.lock (‘/home/stewart/src/pipenv_editable_install_repro/Pipfile.lock’):

{
    "_meta": {
        "hash": {
            "sha256": "7f7606f08e0544d8d012ef4d097dabdd6df6843a28793eb6551245d4b2db4242"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.8"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {},
    "develop": {}
}

About this issue

  • Original URL
  • State: closed
  • Created 10 months ago
  • Reactions: 1
  • Comments: 19 (10 by maintainers)

Most upvoted comments

@lewiswolf I believe that is a separate issue, there are a couple reports about the git+ vcs issue and an open PR that some have reported fix it.

Regarding this actual issue report, I think part of the problem (maybe not the whole problem) is the ast parser logic for the old setup.py file logic is not accounting for all cases. I tried to correct for that some locally over lunch in the reproduction link (super helpful by the way) and was able to get it to install for the top level test-hello, but when I try the one in the applications sub-dir, I get a pathing issue.

I’ll be able to cut a release going into late evening for the PR I have open, but I am not sure I have an immediate fix for what’s going on with editable file installs in this issue report.

+3
matteius on Aug 22, 2023

@stewartmiles I have a PR out that I believe does a better job at determining the relative path: https://github.com/pypa/pipenv/pull/5896

Example:

matteius@matteius-VirtualBox:~/pipenv_editable_install_repro/test-hello$ pipenv install ../applications/test-hello/
Creating a virtualenv for this project...
Pipfile: /home/matteius/pipenv_editable_install_repro/test-hello/Pipfile
Using default python from /usr/bin/python (3.10.12) to create virtualenv...
⠹ Creating virtual environment...created virtual environment CPython3.10.12.final.0-64 in 212ms
  creator CPython3Posix(dest=/home/matteius/.virtualenvs/test-hello--_GSX-an, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/matteius/.local/share/virtualenv)
    added seed packages: pip==23.2.1, setuptools==68.0.0, wheel==0.41.2
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /home/matteius/.virtualenvs/test-hello--_GSX-an
Creating a Pipfile for this project...
Installing ../applications/test-hello/...
Resolving ../applications/test-hello/...
Added test-hello to Pipfile's [packages] ...
✔ Installation Succeeded
Pipfile.lock not found, creating...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
⠏ Locking...Warning: INFO:pipenv.patched.pip._internal.operations.prepare:Obtaining file:///home/matteius/pipenv_editable_install_repro/applications/test-hello (from -r /tmp/pipenv-lleb236s-requirements/pipenv-o8ryjw3e-constraints.txt (line 2))
INFO:pipenv.patched.pip._internal.cli.spinners:Installing build dependencies: started
INFO:pipenv.patched.pip._internal.cli.spinners:Installing build dependencies: finished with status 'done'
INFO:pipenv.patched.pip._internal.cli.spinners:Checking if build backend supports build_editable: started
INFO:pipenv.patched.pip._internal.cli.spinners:Checking if build backend supports build_editable: finished with status 'done'
INFO:pipenv.patched.pip._internal.cli.spinners:Getting requirements to build wheel: started
INFO:pipenv.patched.pip._internal.cli.spinners:Getting requirements to build wheel: finished with status 'done'
INFO:pipenv.patched.pip._internal.cli.spinners:Preparing metadata (pyproject.toml): started
INFO:pipenv.patched.pip._internal.cli.spinners:Preparing metadata (pyproject.toml): finished with status 'done'
Locking [dev-packages] dependencies...
Updated Pipfile.lock (23b9cf861bceae95e2dbb7d2a1bcc9307d3e05244dd1e1e5a0c7eed37108a154)!
Installing dependencies from Pipfile.lock (08a154)...
cat PipTo activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
matteius@matteius-VirtualBox:~/pipenv_editable_install_repro/test-hello$ cat Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
test-hello = {file = "../applications/test-hello"}

[dev-packages]

[requires]
python_version = "3.9"
matteius@matteius-VirtualBox:~/pipenv_editable_install_repro/test-hello$ cat Pipfile.lock 
{
    "_meta": {
        "hash": {
            "sha256": "23b9cf861bceae95e2dbb7d2a1bcc9307d3e05244dd1e1e5a0c7eed37108a154"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.9"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "test-hello": {
            "file": "../applications/test-hello"
        }
    },
    "develop": {}
}

Note: I had to remove the Pipfile/lock file from the parent directory so it wouldn’t get confused about where it was starting from.

+1
matteius on Aug 30, 2023

@stbth01 that issue may be what is tracked with #5878 and #5880

+1
matteius on Aug 28, 2023

Good news (I think) – this branch works for your reproduction examples that I tried on Windows and the changes are still passing the existing pipenv tests. Please have a look and help me verify if it resolves the issues or if there is still something to address regarding local file installs: https://github.com/pypa/pipenv/pull/5870

+1
matteius on Aug 26, 2023
Read more comments on GitHub
← pipenv: 2022.10.9 fails to build from Git (missing poetry)
pipenv: 2nd-order dependencies missing from Pipfile.lock →