pipenv: pipenv (patched pip) corrupts path for editable requirement on Windows
Issue description
I try to install a package into pipenv’s virtual environment as editable (either using relative path pipenv install -e . or absolute path pipenv install -e D:\long\windows\path\project). I get the error with a traceback with this exception: pipenv.patched.pip._internal.exceptions.InstallationError: D:longwindowspathproject is not a valid editable requirement. It should either be a path to a local project or a VCS URL.... (Full traceback provided below.)
With some adhoc debugging I found that the path is first resolved if it’s relative, and then the absolute path gets corrupted (maybe not only) here (link):
options = shlex.split(options_str)
While running install -e ... commands, this line executes twice:
-
options_str == '-i https://pypi.org/simple C:\\Users\\user\\AppData\\Local\\pipenv\\pipenv\\Cache',options == ['-i', 'https://pypi.org/simple', 'C:UsersuserAppDataLocalpipenvpipenvCache'] -
options_str == '-e D:\\long\\windows\\path\\project'options == ['-e', 'D:longwindowspathproject']
shlex.split for some reason removes every backslash from a Windows path, I have tested it in IDLE. I have also tried to run pipenv install -e D:/long/windows/path/project, no changes happened.
When I tried to replicate it with just virtualenv and pip, this line in pip was never executed. (theories ahead) I suppose this is because pipenv creates a temporary requirements.txt to supply it to pip, and somehow it goes wrong. Maybe for local files it should use URLs with file scheme instead of plain absolute paths? ( hinted link)
Steps to replicate
-
be on Windows (10)
-
have a Python project which already has associated virtual environment managed by pipenv (I created it with
python -m pipenv --python=<python-path>at project root)
3.1. from the root directory of the project, run python -m pipenv install -e . (as in docs)
OR
3.2. run python -m pipenv install -e <project-absolute-path>
Expected result
The package is installed into associated virtual environment as editable.
Actual result
The package is not installed, pipenv exits with an error, producing this:
long traceback
d:\long\windows\path\project>python -m pipenv install -e . --dev --verbose
Loading .env environment variables...
Installing -e ....
Resolving -e ....
Installation Succeeded
Pipfile.lock (c606b7) out of date, updating to (916e2e)...
Locking [packages] dependencies...
Locking [dev-packages] dependencies...
Building requirements...
Resolving dependencies...
Traceback (most recent call last):
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\resolver.py", line 675, in <module>
main()
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\resolver.py", line 661, in main
_main(
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\resolver.py", line 645, in _main
resolve_packages(
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\resolver.py", line 612, in resolve_packages
results, resolver = resolve(
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\resolver.py", line 592, in resolve
return resolve_deps(
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\utils\resolver.py", line 918, in resolve_deps
results, hashes, internal_resolver = actually_resolve_deps(
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\utils\resolver.py", line 691, in actually_resolve_deps
resolver.resolve()
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\utils\resolver.py", line 443, in resolve
constraints = self.constraints
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\utils\resolver.py", line 401, in constraints
possible_constraints_list = self.possible_constraints
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\utils\resolver.py", line 388, in possible_constraints
possible_constraints_list = [
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\utils\resolver.py", line 389, in <listcomp>
install_req_from_parsed_requirement(
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\patched\pip\_internal\req\constructors.py", line 485, in install_req_from_parsed_requirement
req = install_req_from_editable(
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\patched\pip\_internal\req\constructors.py", line 240, in install_req_from_editable
parts = parse_req_from_editable(editable_req)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\patched\pip\_internal\req\constructors.py", line 209, in parse_req_from_editable
name, url, extras_override = parse_editable(editable_req)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\patched\pip\_internal\req\constructors.py", line 126, in parse_editable
raise InstallationError(
pipenv.patched.pip._internal.exceptions.InstallationError: D:longwindowspathproject is not a valid editable requirement. It should either be a path to a local project or a VCS URL (beginning with bzr+http, bzr+https, bzr+ssh, bzr+sftp, bzr+ftp, bzr+lp, bzr+file, git+http, git+https, git+ssh, git+git, git+file, hg+file, hg+http, hg+https, hg+ssh, hg+static-http, svn+ssh, svn+http, svn+https, svn+svn, svn+file).
Locking Failed!
[ ] Locking...
Traceback (most recent call last):
File "C:\Program Files\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Program Files\Python310\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\__main__.py", line 4, in <module>
cli()
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\vendor\click\core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\cli\options.py", line 58, in main
return super().main(*args, **kwargs, windows_expand_args=False)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\vendor\click\core.py", line 1078, in main
rv = self.invoke(ctx)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\vendor\click\core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\vendor\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\vendor\click\core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\vendor\click\decorators.py", line 92, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\vendor\click\core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\cli\command.py", line 209, in install
do_install(
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\routines\install.py", line 297, in do_install
raise e
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\routines\install.py", line 281, in do_install
do_init(
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\routines\install.py", line 648, in do_init
do_lock(
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\routines\lock.py", line 65, in do_lock
venv_resolve_deps(
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\utils\resolver.py", line 859, in venv_resolve_deps
c = resolve(cmd, st, project=project)
File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\pipenv\utils\resolver.py", line 728, in resolve
raise RuntimeError("Failed to lock Pipfile.lock!")
RuntimeError: Failed to lock Pipfile.lock!
$ pipenv --support
Pipenv version: '2023.11.17'
Pipenv location: 'C:\\Users\\user\\AppData\\Roaming\\Python\\Python310\\site-packages\\pipenv'
Python location: 'C:\\Program Files\\Python310\\python.exe'
OS Name: 'nt'
User pip version: '23.3.2'
user Python installations found:
PEP 508 Information:
{'implementation_name': 'cpython',
'implementation_version': '3.10.11',
'os_name': 'nt',
'platform_machine': 'AMD64',
'platform_python_implementation': 'CPython',
'platform_release': '10',
'platform_system': 'Windows',
'platform_version': '10.0.19045',
'python_full_version': '3.10.11',
'python_version': '3.10',
'sys_platform': 'win32'}
System environment variables:
…
PIP_DISABLE_PIP_VERSION_CHECKPYTHONDONTWRITEBYTECODEPIPENV_VENV_IN_PROJECTPYTHONFINDER_IGNORE_UNSUPPORTED
Pipenv-specific environment variables:
PIPENV_VENV_IN_PROJECT:1
Debug-specific environment variables:
PATH:...;C:\Program Files\Python310\Scripts\;C:\Program Files\Python310\;...;C:\Windows\System32;C:\Windows;C:\Windows\System32\wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;...;C:\Users\user\AppData\Roaming\Python\Python310\Scripts;...;
Contents of Pipfile (‘D:\long\windows\path\project\Pipfile’):
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
[dev-packages]
[requires]
python_version = "3.10"
python_full_version = "3.10.11"
Contents of Pipfile.lock (‘D:\long\windows\path\project\Pipfile.lock’):
{
"_meta": {
"hash": {
"sha256": "..."
},
"pipfile-spec": 6,
"requires": {
"python_full_version": "3.10.11",
"python_version": "3.10"
},
"sources": [
{
"name": "pypi",
"url": "https://pypi.org/simple",
"verify_ssl": true
}
]
},
"default": {},
"develop": {}
}
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 1
- Comments: 16 (6 by maintainers)
Can you check out: https://github.com/pypa/pipenv/pull/6276
I mean, I have the fresh project dir at
D:\long\windows\path\projectwith no pipenv’s virtualenv and no pipfiles. I copy the wholeprojectdir toD:\long\project, either renaming project and package or not.From within original project path, I run
python -m pipenv install -e ..\..\..\project(orpython -m pipenv install -e D:\long\project, but it comes toshlex.splitas relative path still). It fails because pipenv tries to find something at......project(comment where this particular error is first mentioned).And absolute paths on different drives don’t work as well.
I expect that it should work because
pipenv --helpoutput and man page do not mention particular limitations on a local path for the-eoption, and because it works with pip itself as I just tested.There is
pyproject.tomlat root of pipenv-issue-6068-demo, I’m not aware what is wrong with that.Yes,
virtualenv+pip install -e .successfully installs the package. This was the session from within this same repo in cmd.exe:cmd.exe session
upd: and it is actually editable