poetry: Failure to detect dependencies of scikit-learn from private source

I am on the latest Poetry version.
I have searched the issues of this repo and believe that this is not a duplicate.
If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

OS version and name: macOS 10.13.6
Poetry version: 0.12.16
Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/gmacon/00003531106df72b0ae00415d32d4cf1

Issue

Poetry is failing to detect the dependencies of scikit-learn, but only when using a private source. If you modify the pyproject.toml by removing the tool.poetry.source entry, the dependencies (numpy, scipy, joblib) are correctly detected from PyPI.

This may be the same issue mentioned in https://github.com/sdispater/poetry/issues/607#issuecomment-437008459, but I’m not sure of that.

I had thought that this was caused by something strange that scikit-learn is doing, but I think that’s less likely since it works correctly with PyPI.

Debug output

foo % python3.7 -m venv env
foo % . env/bin/activate
foo % poetry install -vvv                                                                                                                                                     Using virtualenv: /.../foo/env
Updating dependencies
Resolving dependencies...
   1: fact: foo is 0.1.0
   1: derived: foo
   1: fact: foo depends on scikit-learn (^0.21.2)
   1: selecting foo (0.1.0)
   1: derived: scikit-learn (^0.21.2)
nexus: 1 packages found for scikit-learn >=0.21.2,<0.22.0
   1: selecting scikit-learn (0.21.2)
   1: Version solving took 0.236 seconds.
   1: Tried 1 solutions.

Writing lock file


Package operations: 1 install, 0 updates, 0 removals

  - Installing scikit-learn (0.21.2)
foo % poetry debug:info

Poetry
======

 * Version: 0.12.16
 * Python:  3.7.3


Virtualenv
==========

 * Python:         3.7.3
 * Implementation: CPython
 * Path:           /.../foo/env
 * Valid:          True


System
======

 * Platform: darwin
 * OS:       posix
 * Python:   /usr/local/var/pyenv/versions/3.7.3/Python.framework/Versions/3.7

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 9
Comments: 15 (1 by maintainers)

Most upvoted comments

I am using poetry 1.0.5, and it fails to install scikit-learn correctly.

I am using poetry inside docker, with python 3.8, and a private repo.

$ poetry add scikit-learn
Using version ^0.23.1 for scikit-learn

Updating dependencies
Resolving dependencies... (11.0s)An error occurred when reading setup.py or setup.cfg: 'Call' object has no attribute 's'
Resolving dependencies... (13.4s)

Writing lock file

Package operations: 1 install, 0 updates, 0 removals

  - Installing scikit-learn (0.23.1)

Scikit-learn’s dependencies are missing in poetry.lock:

[[package]]
category = "main"
description = "A set of python modules for machine learning and data mining"
name = "scikit-learn"
optional = false
python-versions = "*"
version = "0.23.1"

[package.source]
reference = "our-private-repo"
url = "https://out-private-repo/simple"

An attempt to import sklearn in Python fails, because joblib, one of scikit-learn’s dependencies, is not installed:

# python3.8
Python 3.8.3 (default, May 14 2020, 20:11:43)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sklearn
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/sklearn/__init__.py", line 80, in <module>
    from .base import clone
  File "/usr/local/lib/python3.8/dist-packages/sklearn/base.py", line 21, in <module>
    from .utils import _IS_32BIT
  File "/usr/local/lib/python3.8/dist-packages/sklearn/utils/__init__.py", line 23, in <module>
    from .class_weight import compute_class_weight, compute_sample_weight
  File "/usr/local/lib/python3.8/dist-packages/sklearn/utils/class_weight.py", line 7, in <module>
    from .validation import _deprecate_positional_args
  File "/usr/local/lib/python3.8/dist-packages/sklearn/utils/validation.py", line 22, in <module>
    import joblib
ModuleNotFoundError: No module named 'joblib'

Please reopen this issue.

earshinov on May 24, 2020

Welp, same issue as the others now. joblib can’t be found after installing scikit-learn from a private PyPI repo.

Poetry version 1.0.10 Python version 3.6.11

louwers on Sep 14, 2020

I was having this same issue specifically with scikit-learn and was able to resolve it by updating poetry to the latest preview version (1.0.0b1)

dekim24 on Aug 13, 2019