pipenv: Pipenv 2023.8.21 parse error with packages from git

I have this in my Pipfile:

foobar_pkg_name = {ref = "v2023.08.18.1", git = "https://gitlab%2Bdeploy-token:TheToken@gitlab.example.tld/group_name/foobar_pkg_name.git"}

Results in Pipfile.lock to:

"foobar_pkg_name": {
    "git": "https://gitlab+deploy-token:TheToken@gitlab.example.tld/group_name/foobar_pkg_name.git",
    "ref": "0000-the-pkg-hash-0000"
},

The installation results in a error, because of completely wrong URI:

[pipenv.exceptions.InstallError]: Looking in indexes: https://pypi.python.org/simple
[pipenv.exceptions.InstallError]: Collecting foobar_pkg_name@ git+https://gitlab+deploy-token:TheToken@0000-the-pkg-hash-0000 (from -r /tmp/pipenv-cn09v7m7-requirements/pipenv-98f1ayok-reqs.txt (line 1))
[pipenv.exceptions.InstallError]:   Cloning https://gitlab%2Bdeploy-token:****@0000-the-pkg-hash-0000 to /tmp/pip-install-5uk38bc4/foobar_pkg_name_4300623bc17942a0ae003fe8d1f0500e

The same works with older Pipenv v2023.7.23

About this issue

Original URL
State: closed
Created 10 months ago
Reactions: 3
Comments: 19 (10 by maintainers)

Most upvoted comments

Ah bummer, sorry about this! This is tricky because some folks have the @ symbol as an extra ref and so I was thinking that would be easier to peel back off, but that breaks ssh:// without the extra @ symbol. 🤔

matteius on Aug 22, 2023

I have the same problem with editable GitHub dependecies.

Pipfile:

mypackage = {editable = true,git = "ssh://git@github.com/myorg/mypackage",ref = "master"}

Pipfile.lock:

"mypackage": {
    "editable": true,
    "git": "ssh://git@github.com/myorg/mypackage",
    "ref": "c807****d402"
},

Error:

Installing dependencies from Pipfile.lock (057240)...
[pipenv.exceptions.InstallError]: Collecting mypackage@ git+ssh://git@c807****d402 (from -r /var/folders/2y/4s08qm216w53054z5h2jl4zh0000gn/T/pipenv-q8sxrgdh-requirements/pipenv-q27ppm60-reqs.txt (line 1))
[pipenv.exceptions.InstallError]:   Cloning ssh://****@c807****d402 to /private/var/folders/2y/4s08qm216w53054z5h2jl4zh0000gn/T/pip-install-lv7r7wwo/mypackage_9814293569e9490e8c7f19e26dd67e7c
[pipenv.exceptions.InstallError]: Running command git clone --filter=blob:none --quiet 'ssh://****@c807****d402' /private/var/folders/2y/4s08qm216w53054z5h2jl4zh0000gn/T/pip-install-lv7r7wwo/mypackage_9814293569e9490e8c7f19e26dd67e7c
[pipenv.exceptions.InstallError]:   fatal: No path specified. See 'man git-pull' for valid url syntax
[pipenv.exceptions.InstallError]:   error: subprocess-exited-with-error
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:   × git clone --filter=blob:none --quiet 'ssh://****@c807****d402' /private/var/folders/2y/4s08qm216w53054z5h2jl4zh0000gn/T/pip-install-lv7r7wwo/mypackage_9814293569e9490e8c7f19e26dd67e7c did not run successfully.
[pipenv.exceptions.InstallError]:   │ exit code: 128
[pipenv.exceptions.InstallError]:   ╰─> See above for output.
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]:   note: This error originates from a subprocess, and is likely not a problem with pip.
[pipenv.exceptions.InstallError]: error: subprocess-exited-with-error
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]: × git clone --filter=blob:none --quiet 'ssh://****@c807****d402' /private/var/folders/2y/4s08qm216w53054z5h2jl4zh0000gn/T/pip-install-lv7r7wwo/mypackage_9814293569e9490e8c7f19e26dd67e7c did not run successfully.
[pipenv.exceptions.InstallError]: │ exit code: 128
[pipenv.exceptions.InstallError]: ╰─> See above for output.
[pipenv.exceptions.InstallError]:
[pipenv.exceptions.InstallError]: note: This error originates from a subprocess, and is likely not a problem with pip.

If I’m not mistaken the bug came in with this PR by @matteius: https://github.com/pypa/pipenv/pull/5845

marceltschoppch on Aug 22, 2023

@julio-tl Are you able to check the tagged draft PR to see if it fixes the regression in your case? Also, I apologize for the inconvenience.

matteius on Aug 22, 2023

@bpicardat That might be viable, but if I recall when working on the original refactor I was having some trouble with urllib parse but I can’t recall specifically if it was about evaluating the netloc env vars and loosing them in the evaluated string (they get masked) or something similar that makes it hard to reconstruct back to the original line after parsing it. I opened an alternative PR that I think might solve it, assuming there aren’t examples that lack a slash after the netloc @

matteius on Aug 22, 2023

Maybe with urllib.parse.urlparse? Only remove the trailing @ and ref string from the “path” but don’t touch the “netloc”

bpicardat on Aug 22, 2023

Thanks @matteius , your PR fixed the issue for my case 👍

marceltschoppch on Aug 22, 2023