poetry: Poetry unable to find installation candidates for Nvidia c11 dependencies

  • Poetry version: 1.2.0
  • Python version: poetry defaults to 3.8.14 from 3.10.6
  • OS version and name: macOS 13 Ventura
  • pyproject.toml: see in the file below:
  • I am on the latest stable Poetry version, installed using a recommended method.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have consulted the FAQ and blog for any relevant entries or release notes.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

Hi there,

Having trouble using poetry with certain NLP libraries that have Nvidia c11 related dependencies. In this case I’m using EasyNMT for machine text translation, which has a dependency on torch.

The torch related issue #4231 was recently closed, so I wanted to open a new separate issue here.

I’m unable to install the 4 Nvidia c11 dependencies:

  • nvidia-cublas-cu11 (11.10.3.66)
  • nvidia-cuda-nvrtc-cu11 (11.7.99)
  • nvidia-cuda-runtime-cu11 (11.7.99)
  • nvidia-cudnn-cu11 (8.5.0.96)

Would appreciate any advice on whether this issue is due to my setup, or an issue with poetry. Please let me know if you’re able to reproduce this issue, so I can review my environment accordingly. I have been able to lock and install these dependencies with pipenv, and then run the relevant EasyNMT code.

Thanks for the help! Alex

pyproject.toml

[tool.poetry]
name = "Translation Project"
version = "1.0"
description = "Translating text"
authors = ["me <me@email.com>"]

[tool.poetry.dependencies]
python = "==3.8.14"
easynmt = "==2.0.2"

Checking works correctly:

poetry check

All set!

Locking works correctly:

poetry lock

The currently activated Python version 3.10.6 is not supported by the project (==3.8.14).
Trying to find and use a compatible version. 
Using python3 (3.8.14)
Creating virtualenv translation_porject-pPGuLrxo-py3.8 in /Users/alex/Library/Caches/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies... Downloading https://files.pythonhosted.org/packages/ec/87/f26695307c0aa00e6938f5de795fc7f2c718a448b48d29a4c8c8dbf829d3/sentencepiece-0.1.97.tResolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz   Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  1Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  2Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  3Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  5Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  6Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  7Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  8Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  9Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  9Resolving dependencies... Downloading https://files.pythonhosted.org/packages/64/8e/9929b64e146d240507edaac2185cd5516f00b133be5b39250d253be25a64/numpy-1.23.4.tar.gz  9Resolving dependencies... Downloading https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3Resolving dependencies... (84.8s)

Writing lock file

Install does not work correctly:

poetry install -vvv

The currently activated Python version 3.10.6 is not supported by the project (==3.8.14).
Trying to find and use a compatible version. 
Trying python3
Using python3 (3.8.14)
Virtualenv translation_porject-pPGuLrxo-py3.8 already exists.
Using virtualenv: /Users/alex/Library/Caches/pypoetry/virtualenvs/translation_porject-pPGuLrxo-py3.8
Project environment contains an empty path in sys_path, ignoring.
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 11 installs, 0 updates, 0 removals, 20 skipped

  • Installing nvidia-cublas-cu11 (11.10.3.66): Pending...
  • Installing nvidia-cuda-nvrtc-cu11 (11.7.99): Pending...
  • Installing nvidia-cuda-runtime-cu11 (11.7.99): Pending...
  • Installing nvidia-cudnn-cu11 (8.5.0.96): Pending...
[keyring.backend] Loading KWallet
[keyring.backend] Loading KWallet
[keyring.backend] Loading KWallet
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading SecretService
[keyring.backend] Loading SecretService
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading Windows
[keyring.backend] Loading Windows
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading chainer
[keyring.backend] Loading chainer
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading libsecret
[keyring.backend] Loading libsecret
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
[keyring.backend] Loading macOS
[keyring.backend] Loading macOS
[keyring.backend] Loading macOS
Creating new session for pypi.org
Creating new session for pypi.org
Skipping wheel nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl as this is not supported by the current environment
Skipping wheel nvidia_cublas_cu11-11.10.3.66-py3-none-win_amd64.whl as this is not supported by the current environment
Skipping wheel nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl as this is not supported by the current environment
Skipping wheel nvidia_cuda_nvrtc_cu11-11.7.99-py3-none-manylinux1_x86_64.whl as this is not supported by the current environment
Skipping wheel nvidia_cuda_nvrtc_cu11-11.7.99-py3-none-win_amd64.whl as this is not supported by the current environment
Skipping wheel nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl as this is not supported by the current environment
  • Installing nvidia-cublas-cu11 (11.10.3.66): Failed

  Stack trace:

  5  /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/executor.py:250 in _execute_operation
        248│ 
      249│             try:
    → 250│                 result = self._do_execute_operation(operation)
      251│             except EnvCommandError as e:
      252│                 if e.e.returncode == -2:

  4  /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/executor.py:323 in _do_execute_operation
      321│             return 0
      322│ 
    → 323│         result: int = getattr(self, f"_execute_{method}")(operation)
      324│ 
      325│         if result != 0:

  3  /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/executor.py:443 in _execute_install
      441│ 
      442│     def _execute_install(self, operation: Install | Update) -> int:
    → 443│         status_code = self._install(operation)
      444│ 
      445│         self._save_url_reference(operation)

  2  /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/executor.py:477 in _install
      475│             archive = self._download_link(operation, Link(package.source_url))
      476│         else:
    → 477│             archive = self._download(operation)
      478│ 
      479│         operation_message = self.get_operation_message(operation)

  1  /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/executor.py:622 in _download
      620│ 
      621│     def _download(self, operation: Install | Update) -> Path:
    → 622│         link = self._chooser.choose_for(operation.package)
      623│ 
      624│         if link.yanked:

  RuntimeError

  Unable to find installation candidates for nvidia-cublas-cu11 (11.10.3.66)

  at /usr/local/Cellar/poetry/1.2.0/libexec/lib/python3.10/site-packages/poetry/installation/chooser.py:103 in choose_for
       99│ 
      100│             links.append(link)
      101│ 
      102│         if not links:
    → 103│             raise RuntimeError(f"Unable to find installation candidates for {package}")
      104│ 
      105│         # Get the best link
      106│         chosen = max(links, key=lambda link: self._sort_key(package, link))
      107│

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 2
  • Comments: 20 (4 by maintainers)

Most upvoted comments

Is no one else still having problems with this? I get same error with 1.13.1.

+15
ccharp on Feb 14, 2023

Seems to have been reported in pytorch: https://github.com/pytorch/pytorch/issues/88049

Edit [Nov 18 2022]: For future reference https://github.com/pytorch/pytorch/pull/88826 The fix seems to be merged with torch. I will update again when the wheels with the fix land in pypi

Edit [Dec 17 2022]: 1.31.1 has landed in pypi, and it works! setting pytorch requirement to “^1.31.1”

+10
jspaezp on Dec 17, 2022

torch 2.0.1 also shows the same problem 😦 I used python 3.10.

+7
kyaryunha on Jun 20, 2023

For the new influx of people with the Mac M1 chip bumping into this thread, the current workaround is adding the correct wheel directly to your pyproject.toml:

[tool.poetry.dependencies]
python = "3.10"
torch = {url = "https://download.pytorch.org/whl/cpu/torch-1.13.1%2Bcpu-cp310-cp310-linux_x86_64.whl"}

If you’re developing locally but using docker for publishing, you can add markers as seen here.

+6
remy-radix on Feb 28, 2023

I believe the torch 1.13.1 version with the fixes in it has indeed landed in PyPI: https://pypi.org/project/torch/

+6
eric79 on Dec 16, 2022

Make sure you’re not using Python 3.11 (or 3.12, for that matter). I just downgraded to 3.10, and I was able to successfully install PyTorch!

+3
heyajulia on Mar 1, 2023

For python 3.9 I had to add

torch = "<1.13"

to my pyproject.toml file to avoid the issue.

+1
asmaier on Jul 4, 2023

no distribution is available for your platform. There is nothing poetry can do about this. This is not a bug.

https://pypi.org/project/nvidia-cublas-cu11/11.10.3.66/#files

Ok cool – what’s confusing is that pipenv can install the exact same library with these dependencies (EasyNMT) on my platform (macOS) without any problems, so I thought it might be an issue since we recently switched to poetry. If you don’t think it’s an issue you can close this thanks, and we will try find a workaround.

+1
Alexander-Mark on Nov 2, 2022
Read more comments on GitHub
← poetry: Poetry tries to remove setuptools and then complains it can't find it
poetry: Poetry update never finishes resolve and Poetry show --outdated hangs →