pipenv: ERROR: No matching distribution found for torch==1.11.0+cu113 when using --extra-index-url

It seems that this error is also a problem when installing from other sources, such as in #5021.

Issue description

After lots of debugging and trial and error, we have run into a bug with pipenv relating to installing libraries while using another index url (–extra-index-url).

When trying to install PyTorch using the following command pipenv install --extra-index-url https://download.pytorch.org/whl/cu113/ "torch==1.11.0+cu113" Installing torch==1.11.0+cu113, as specified under this issue: https://github.com/pypa/pipenv/issues/4961, we get the following error: ERROR: No matching distribution found for torch==1.11.0+cu113.

According to @Bananaman this was possible to do in a previous version of pipenv.

Expected result

Expected an installation using a 3rd party index url, in this case PyTorch.

Actual result

daniel@daniel-desktop:~/Documents$ pipenv install --extra-index-url https://download.pytorch.org/whl/cu113/ "torch==1.11.0+cu113"
Installing torch==1.11.0+cu113...
Adding torch to Pipfile's [packages]...
✔ Installation Succeeded 
Pipfile.lock (db4242) out of date, updating to (7d86a3)...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✘ Locking Failed! 

CRITICAL:pipenv.patched.notpip._internal.resolution.resolvelib.factory:Could not find a version that satisfies the requirement torch==1.11.0+cu113 (from versions: 1.4.0, 1.5.0, 1.5.1, 1.6.0, 1.7.0, 1.7.1, 1.8.0, 1.8.1, 1.9.0, 1.9.1, 1.10.0, 1.10.1, 1.10.2, 1.11.0)
[ResolutionFailure]:   File "/home/daniel/.local/lib/python3.8/site-packages/pipenv/resolver.py", line 743, in _main
[ResolutionFailure]:       resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages, dev)
[ResolutionFailure]:   File "/home/daniel/.local/lib/python3.8/site-packages/pipenv/resolver.py", line 704, in resolve_packages
[ResolutionFailure]:       results, resolver = resolve(
[ResolutionFailure]:   File "/home/daniel/.local/lib/python3.8/site-packages/pipenv/resolver.py", line 685, in resolve
[ResolutionFailure]:       return resolve_deps(
[ResolutionFailure]:   File "/home/daniel/.local/lib/python3.8/site-packages/pipenv/utils.py", line 1398, in resolve_deps
[ResolutionFailure]:       results, hashes, markers_lookup, resolver, skipped = actually_resolve_deps(
[ResolutionFailure]:   File "/home/daniel/.local/lib/python3.8/site-packages/pipenv/utils.py", line 1127, in actually_resolve_deps
[ResolutionFailure]:       resolver.resolve()
[ResolutionFailure]:   File "/home/daniel/.local/lib/python3.8/site-packages/pipenv/utils.py", line 905, in resolve
[ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: No matching distribution found for torch==1.11.0+cu113

Steps to replicate

Be sure to use the latest version of pipenv, if I use a previous version (latest confirmed is 2022.1.8) the error goes away.

Create clean environment
pipenv install --extra-index-url https://download.pytorch.org/whl/cu113/ "torch==1.11.0+cu113"
Error should appear on latest version of pipenv

$ pipenv --support

Pipenv version: '2022.3.28'

Pipenv location: '/home/daniel/.local/lib/python3.8/site-packages/pipenv'

Python location: '/usr/bin/python3'

Python installations found:

3.8.12: /home/daniel/.pyenv/versions/3.8.12/bin/python
3.8.10: /usr/bin/python3
3.8.10: /usr/bin/python3.8
3.8.10: /bin/python3
3.8.10: /bin/python3.8

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.8.10',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.13.0-37-generic',
 'platform_system': 'Linux',
 'platform_version': '#42~20.04.1-Ubuntu SMP Tue Mar 15 15:44:28 UTC 2022',
 'python_full_version': '3.8.10',
 'python_version': '3.8',
 'sys_platform': 'linux'}

System environment variables:

SHELL
SESSION_MANAGER
QT_ACCESSIBILITY
COLORTERM
PYENV_SHELL
XDG_CONFIG_DIRS
XDG_MENU_PREFIX
GNOME_DESKTOP_SESSION_ID
MANDATORY_PATH
LC_ADDRESS
GNOME_SHELL_SESSION_MODE
LC_NAME
SSH_AUTH_SOCK
XMODIFIERS
DESKTOP_SESSION
LC_MONETARY
SSH_AGENT_PID
GTK_MODULES
PWD
LOGNAME
XDG_SESSION_DESKTOP
XDG_SESSION_TYPE
GPG_AGENT_INFO
XAUTHORITY
GJS_DEBUG_TOPICS
WINDOWPATH
HOME
USERNAME
IM_CONFIG_PHASE
LC_PAPER
LANG
LS_COLORS
XDG_CURRENT_DESKTOP
VTE_VERSION
GNOME_TERMINAL_SCREEN
INVOCATION_ID
MANAGERPID
GJS_DEBUG_OUTPUT
LESSCLOSE
XDG_SESSION_CLASS
SETUPTOOLS_USE_DISTUTILS
TERM
LC_IDENTIFICATION
DEFAULTS_PATH
LESSOPEN
USER
GNOME_TERMINAL_SERVICE
DISPLAY
SHLVL
LC_TELEPHONE
QT_IM_MODULE
LC_MEASUREMENT
LD_LIBRARY_PATH
XDG_RUNTIME_DIR
PYENV_ROOT
LC_TIME
JOURNAL_STREAM
XDG_DATA_DIRS
PATH
GDMSESSION
DBUS_SESSION_BUS_ADDRESS
LC_NUMERIC
OLDPWD
_
PIP_SHIMS_BASE_MODULE
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONDONTWRITEBYTECODE
PIP_PYTHON_PATH
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

PATH: /home/daniel/.pyenv/shims:/home/daniel/.pyenv/bin:/usr/local/cuda-11.3/bin:/home/daniel/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
SHELL: /bin/bash
LANG: en_US.UTF-8
PWD: /home/daniel/Documents

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 9
Comments: 25 (12 by maintainers)

Most upvoted comments

Also, I just checked the wheel archive torch-1.11.0+cu113-cp310-cp310-win_amd64.whl and I don’t see where it would ever figure out that it need numpy implicitly – probably something you always have to manually install or add to the Pipfile as well I am guessing with that particular archive.

matteius on Mar 31, 2022

Thanks for confirming this @DanielPerezJensen – I have a commit that ensure that the 1-hash that gets installed for “torch” or other index restricted packages gets included, but I’ve had more difficulties getting it to include all 8 hashes again. I think the ideal, would be it includes the two hashes that match your required python_version or all of them (all 8) if a version is not specified. I will be looking to find more time to look into this, at a minimum will ship new version in April that includes at least the matching hash, but ideally will figure out how to include all the relevant hashes as well for private index restricted packages.

matteius on Mar 31, 2022

@matteius

Running pipenv install "torch==1.11.0+cu113" --index https://download.pytorch.org/whl/cu113/ -v yields the following pip command: /home/daniel/.local/share/virtualenvs/test-Kb_OZVBf/bin/python -m pip install --verbose --upgrade --exists-action=i -r /tmp/pipenv-zz18ayop-requirements/pipenv-edwy4lp9-requirement.txt -i https://download.pytorch.org/whl/cu113/ --extra-index-url https://pypi.org/simple --extra-index-url https://pypi.org/simple. This is with Python 3.8.12, and pipenv 2022.3.28.

The verbose output is this:

(test) daniel@daniel-desktop:~/Documents/test$ pipenv install "torch==1.11.0+cu113" --index https://download.pytorch.org/whl/cu113/ -v
Installing torch==1.11.0+cu113...
Installing package: torch==1.11.0+cu113
Writing supplied requirement line to temporary file: 'torch==1.11.0+cu113'
Installing 'torch'
⠸ Installing torch...$ /home/daniel/.local/share/virtualenvs/test-Kb_OZVBf/bin/python -m pip install --verbose --upgrade --exists-action=i -r /tmp/pipenv-zz18ayop-requirements/pipenv-edwy4lp9-requirement.txt -i https://download.pytorch.org/whl/cu113/ --extra-index-url https://pypi.org/simple --extra-index-url https://pypi.org/simple
Using source directory: '/home/daniel/.local/share/virtualenvs/test-Kb_OZVBf/src'
Adding torch to Pipfile's [packages]...
✔ Installation Succeeded 
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
Reporter.starting()
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.starting()
Reporter.adding_requirement(SpecifierRequirement('torch==1.11.0+cu113'), None)
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.adding_requirement(SpecifierRequirement('torch==1.11.0+cu113'), None)
Reporter.starting_round(0)
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.starting_round(0)
Reporter.adding_requirement(SpecifierRequirement('typing-extensions'), LinkCandidate('https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (from https://download.pytorch.org/whl/cu113/torch/)'))
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.adding_requirement(SpecifierRequirement('typing-extensions'), LinkCandidate('https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (from https://download.pytorch.org/whl/cu113/torch/)'))
Reporter.pinning(LinkCandidate('https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (from https://download.pytorch.org/whl/cu113/torch/)'))
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.pinning(LinkCandidate('https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (from https://download.pytorch.org/whl/cu113/torch/)'))
Reporter.ending_round(0, state)
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.ending_round(0, state)
Reporter.starting_round(1)
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.starting_round(1)
Reporter.pinning(LinkCandidate('https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl#sha256=21c85e0fe4b9a155d0799430b0ad741cdce7e359660ccbd8b530613e8df88ce2 (from https://pypi.org/simple/typing-extensions/) (requires-python:>=3.6)'))
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.pinning(LinkCandidate('https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl#sha256=21c85e0fe4b9a155d0799430b0ad741cdce7e359660ccbd8b530613e8df88ce2 (from https://pypi.org/simple/typing-extensions/) (requires-python:>=3.6)'))
Reporter.ending_round(1, state)
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.ending_round(1, state)
Reporter.starting_round(2)
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.starting_round(2)
Reporter.ending(State(mapping=OrderedDict([('torch', LinkCandidate('https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (from https://download.pytorch.org/whl/cu113/torch/)')), ('typing-extensions', LinkCandidate('https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl#sha256=21c85e0fe4b9a155d0799430b0ad741cdce7e359660ccbd8b530613e8df88ce2 (from https://pypi.org/simple/typing-extensions/) (requires-python:>=3.6)'))]), criteria={'torch': Criterion((SpecifierRequirement('torch==1.11.0+cu113'), via=None)), 'typing-extensions': Criterion((SpecifierRequirement('typing-extensions'), via=LinkCandidate('https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (from https://download.pytorch.org/whl/cu113/torch/)')))}))
INFO:pipenv.patched.notpip._internal.resolution.resolvelib.reporter:Reporter.ending(State(mapping=OrderedDict([('torch', LinkCandidate('https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (from https://download.pytorch.org/whl/cu113/torch/)')), ('typing-extensions', LinkCandidate('https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl#sha256=21c85e0fe4b9a155d0799430b0ad741cdce7e359660ccbd8b530613e8df88ce2 (from https://pypi.org/simple/typing-extensions/) (requires-python:>=3.6)'))]), criteria={'torch': Criterion((SpecifierRequirement('torch==1.11.0+cu113'), via=None)), 'typing-extensions': Criterion((SpecifierRequirement('typing-extensions'), via=LinkCandidate('https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp38-cp38-linux_x86_64.whl (from https://download.pytorch.org/whl/cu113/torch/)')))}))
Warning: Error generating hash for torch
⠋ Locking...
✔ Success! 
Updated Pipfile.lock (20805a)!
Installing dependencies from Pipfile.lock (20805a)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00

With the following Pipfile:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[[source]]
url = "https://download.pytorch.org/whl/cu113/"
verify_ssl = true
name = "downloadpytorch"

[packages]
torch = {version = "==1.11.0+cu113", index = "downloadpytorch"}

[dev-packages]

[requires]
python_version = "3.8"

Importing torch python -c "import torch" yields:

(test) daniel@daniel-desktop:~/Documents/test$ python -c "import torch"
/home/daniel/.local/share/virtualenvs/test-Kb_OZVBf/lib/python3.8/site-packages/torch/_masked/__init__.py:223: UserWarning: Failed to initialize NumPy: numpy.core.multiarray failed to import (Triggered internally at  ../torch/csrc/utils/tensor_numpy.cpp:68.)
  example_input = torch.tensor([[-3, -2, -1], [0, 1, 2]])

The numpy dependency not being installed is a “problem” that I have seen many times when installing PyTorch through their index. Installing numpy solves the issue and torch can be imported.

Furthermore running some test code to verify that the gpu is being used shows that indeed the GPU is getting used.

DanielPerezJensen on Mar 31, 2022