h5py: cython: compilation error while building h5py (solved by workaround)

Hello, here’s the error-logs and steps to reproduce when I try to rebuild h5py with python3/pip on ubuntu18.04.

Operating System Ubuntu 18.04
Python version 3.6
Where Python was acquired: system Python on Linux
h5py version: 2.10.0 (from pip)
HDF5 version: 1.10.0

I provide a docker file to reproduce the problem locally, that you can run from a clean directory with

$ mv Dockerfile.txt Dockerfile
$ docker build .

Logs:

[...]
    Error compiling Cython file:
    ------------------------------------------------------------
    ...
    
    
    # --- open, open_by_name, open_by_idx ---
    
    @with_phil
    def open(ObjectID loc not None, char* name=NULL, int index=-1, *,
                                   ^
    ------------------------------------------------------------
    
    h5py/h5a.pyx:54:32: Cannot convert 'void *' to Python object
[...]

Dockerfile.txt

which essentially does:

RUN python3 -m pip install --no-cache-dir mpi4py
RUN echo ------- available system libraries:; find /usr/lib -name "libhdf5.so*"; echo -------
RUN LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/hdf5/serial  python3 -m pip install --no-cache-dir --no-binary=h5py h5py

Full log: log.txt

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 4
Comments: 18 (7 by maintainers)

Most upvoted comments

I did some more digging into the problem. Actually, I was wrong regarding Cython. We did not have Cython installed. This means that when we install h5py with pip3 install h5py the installer downloads a temporary copy of Cython with version 3.0a1. Note that this is the latest release that was released 4/12/2020 (five days ago). With this version of Cython, pip3 install h5py fails as reported. OTOH, dowloading the sources from the master branch of the repo, and using pip3 install . compiles fine, still using Cython 3.0a1.

Finally, if we install Cython with pip3 install Cython which installs Cython version 0.29.16 before running pip3 install h5py the process runs smoothly and h5py installs fine.

To summarize: compiling the files from master branch work with latest version of Cython compiling files from tarball requires previous version of Cython

Hope this helps you guys fixing the problem.

On Fri, Apr 17, 2020 at 6:46 AM Thomas Kluyver notifications@github.com wrote:

Yes, any change in master won’t be found until there’s a new release, which is not imminent at the moment.

If you want to install the stable version, you’ll need to identify what change has broken it (Cython 3?) and ensure you have a compatible version of that for building h5py.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/h5py/h5py/issues/1533#issuecomment-615178270, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACAM4Y4MW25C623SS5XIOZDRNAXQXANCNFSM4MIYVIRA .

ululi1970 on Apr 17, 2020

It looks like there has been a recent change to the Cython API which must have broken h5py as fetched by pip.
Cloning the master branch on my machine and compiling from it worked all right. Hopefully, the fixed code will make its way to the tarball that pip fetches from.

ululi1970 on Apr 15, 2020