pip: PIP 19.0.2 breaks numpy/pandas install

Environment

pip version: 19.0.2
Python version: 3.4
OS: ubuntu

Docker build off base of ubuntu:trusty

Description

Numpy install appears to succeed, but with warnings Subsequent pandas install can’t find numpy

Pinning pip to 19.0.1 changes the error to the issue reported here: https://github.com/pypa/pip/issues/6163 Pinning pip to <19.0 resolves it.

Expected behavior

The packages should install

How to Reproduce

Create a “requirements.txt” containing

setuptools
openpyxl
numpy
pandas

Create the following Dockerfile:

FROM ubuntu:trusty AS python-packages
RUN apt-get update && \
    apt-get -qy install python3-pip

COPY /config/requirements /tmp/requirements
RUN pip3 install --upgrade pip --ignore-installed -r /tmp/requirements

Attempt to build the docker image
The error below occurs

Output

Downloading/unpacking numpy (from -r /tmp/requirements (line 3))
  Running setup.py (path:/tmp/pip_build_root/numpy/setup.py) egg_info for package numpy
    Running from numpy source directory.
    /usr/lib/python3.4/distutils/dist.py:260: UserWarning: Unknown distribution option: 'python_requires'
      warnings.warn(msg)

    no previously-included directories found matching 'doc/build'
    no previously-included directories found matching 'doc/source/generated'
    no previously-included directories found matching 'benchmarks/env'
    no previously-included directories found matching 'benchmarks/results'
    no previously-included directories found matching 'benchmarks/html'
    no previously-included directories found matching 'benchmarks/numpy'
    no previously-included directories found matching '*/__pycache__'
    warning: no previously-included files matching '*.pyc' found anywhere in distribution
    warning: no previously-included files matching '*.pyo' found anywhere in distribution
    warning: no previously-included files matching '*.pyd' found anywhere in distribution
    warning: no previously-included files matching '*.swp' found anywhere in distribution
    warning: no previously-included files matching '*.bak' found anywhere in distribution
    warning: no previously-included files matching '*~' found anywhere in distribution
Downloading/unpacking pandas (from -r /tmp/requirements (line 4))
  Running setup.py (path:/tmp/pip_build_root/pandas/setup.py) egg_info for package pandas
    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/pkg_resources.py", line 225, in get_provider
        module = sys.modules[moduleOrReq]
    KeyError: 'numpy'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/pip_build_root/pandas/setup.py", line 732, in <module>
        ext_modules=maybe_cythonize(extensions, compiler_directives=directives),
      File "/tmp/pip_build_root/pandas/setup.py", line 475, in maybe_cythonize
        numpy_incl = pkg_resources.resource_filename('numpy', 'core/include')
      File "/usr/lib/python3/dist-packages/pkg_resources.py", line 935, in resource_filename
        return get_provider(package_or_requirement).get_resource_filename(
      File "/usr/lib/python3/dist-packages/pkg_resources.py", line 227, in get_provider
        __import__(moduleOrReq)
    ImportError: No module named 'numpy'
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 225, in get_provider

    module = sys.modules[moduleOrReq]

KeyError: 'numpy'

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 17 (7 by maintainers)

Most upvoted comments

I was thinking… Issues like this make me wonder if pip should be doing something like logging its version number prior to the first log message, possibly along with other info about itself. The message could be as unobtrusive as a single [pip 19.0.1] prefixing the very first log message.

This could perhaps cut down on a lot of the troubleshooting time for both users and maintainers because they would immediately see that the version of pip they thought was running isn’t. @cdagraca, would that have helped in this case?

cjerdonek on Feb 18, 2019

@uchida-takumi thanks for the tip, it does seem like it’s happening in -alpine images. Switching to -slim and everything worked as expected.

mikew on Mar 12, 2019

@daa , @ncoghlan - a few more attempts showed that pip3 install --upgrade wasn’t actually doing anything. pip3 install --upgrade pip, on the other hand, fixes it. Sorry for any wasted time, and thanks to all who’ve helped look into this.

cdagraca on Feb 18, 2019

@cdagraca as @daa says in your docker script, you’re upgrading (the system) pip and installing the other requirements all in a single command. That doesn’t guarantee which version of pip will be getting used to install the other requirements.

Do you still see the error if you split the pip upgrade and the requirements install into separate steps?

RUN pip3 install --upgrade pip
RUN pip3 install -r /tmp/requirements

pfmoore on Feb 14, 2019