cartopy: pip install of cartopy 0.18.0 fails when installing numpy at the same time

Description

I am provisioning a docker image using pip. In a single pip command, I installed a number of packages, including cartopy and numpy. This worked in versions prior to 0.18.0, and no longer works with 0.18.0.

Code to reproduce

In a docker image with vanilla python3 install and no pip packages installed, run

pip3 install --upgrade pip   && pip3 install --no-cache-dir cartopy==0.18.0 numpy

Traceback

    ERROR: Command errored out with exit status 1:
     command: /usr/local/pyenv/versions/3.7.6/bin/python3.7 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-cdx7ek5c/cartopy/setup.py'"'"'; __file__='"'"
'/tmp/pip-install-cdx7ek5c/cartopy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(cod
e, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-377te2k_
         cwd: /tmp/pip-install-cdx7ek5c/cartopy/
    Complete output (12 lines):
    Traceback (most recent call last):
      File "/tmp/pip-install-cdx7ek5c/cartopy/setup.py", line 43, in <module>
        import numpy as np
    ModuleNotFoundError: No module named 'numpy'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-cdx7ek5c/cartopy/setup.py", line 45, in <module>
        raise ImportError('NumPy 1.10+ is required to install cartopy.')
    ImportError: NumPy 1.10+ is required to install cartopy.
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Full environment definition

Operating system

Ubuntu 18.04 Python 3.7.6 installed via pyenv

Cartopy version

0.18.0

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 5
Comments: 15 (15 by maintainers)

Most upvoted comments

I totally agree about the difficulties with pip and non-python dependencies. Unfortunately, I don’t think conda has kept up with some of the innovations of the broader community around lockfiles/deterministic builds and project management. I would rather use two package managers (apt-get for system libraries) and pip/poetry/pipenv for python dependencies than sacrifice determinism. conda install cartopy xarray fsspec scikit-learn zarr is great, but put that in a CI system and eventually some dependency update will break the build. In my experience with this stack, this happens about twice a month and takes 30+ minutes to debug each time. I would much rather spend 10 minutes up-front writing a dockerfile/shell script with poetry +apt-get if it is possible to achieve a deterministic build.

At any rate, I don’t think it should be too hard to make cartopy PEP-517 compliant.

nbren12 on Nov 18, 2020

Thanks for the explanation. It sounds like it is important to test against specific versions of numpy, and that you don’t find pip s interpretation of PEP-517 useful for testing purposes. Fair enough. However, instead of breaking pip installs for users, would it be possible to avoid using pip in the tests (e.g. modify them to use python setup.py install)?

This issues matters because “serverless” cloud python runtimes like AWS lambda and Google Cloud dataflow tend to support pip but not anaconda. Also, IMO anaconda has fallen a bit behind the curve in terms of dependency locking and other features that tools like poetry and pipenv provide.

nbren12 on Nov 18, 2020

I don’t think it’s so much an end-user issues as a developer one. If I install numpy 1.16 to test, I want cartopy to build against it. And if I’ve already installed numpy from conda, I sure as hell don’t want it downloading something from pypi–especially an sdist. See https://github.com/SciTools/cartopy/pull/1264#issuecomment-454366232 and the surrounding discussion.

dopplershift on Nov 18, 2020

@dopplershift if the version in install_requires matches the version in build-system requires then this won’t be a problem. You’d want to pick the widest possible range of numpy versions with a stable ABI

Also a set of manylinux,macos and windows abi3 binary wheel would solve this for most users

graingert on Nov 18, 2020