tensorflow: [ppc64le] Error installing custom build tensorflow 1.12 wheel with python 3.6

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04):

LSB Version: :core-4.1-noarch:core-4.1-ppc64le Distributor ID: RedHatEnterpriseServer Description: Red Hat Enterprise Linux Server release 7.5 (Maipo) Release: 7.5 Codename: Maipo

  • TensorFlow installed from (source or binary): source
  • TensorFlow version: 1.12
  • Python version: 3.6.7
  • Installed using virtualenv? pip? conda?: pip
  • Bazel version (if compiling from source): 0.15.0
  • GCC/Compiler version (if compiling from source): 4.8.5
  • CUDA/cuDNN version: 9.2/7.1
  • GPU model and memory: NVIDIA Pascal V100 16GB

Describe the problem Trying to install a custom tensorflow 1.12 wheel on ppc64le IBM Power9 system with NVIDIA Pascal V100 but the installation breaks while installing h5py with the error shown below.

Provide the exact sequence of commands / steps that you executed before running into the problem

Build tensorflow wheel from source using standard procedure described: here (though modified as required) and install using pip.

Any other info / logs

gcc -pthread -B /ccs/home/shubhankar/miniconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DH5_USE_16_API -I./h5py -I/tmp/pip-install-d__8pdpg/h5py/lzf -I/opt/local/include -I/usr/local/include -I/ccs/home/shubhankar/miniconda3/lib/python3.6/site-packages/numpy/core/include -I/ccs/home/shubhankar/miniconda3/include/python3.6m -c /tmp/pip-install-d__8pdpg/h5py/h5py/defs.c -o build/temp.linux-ppc64le-3.6/tmp/pip-install-d__8pdpg/h5py/h5py/defs.o
    In file included from /ccs/home/shubhankar/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:1821:0,
                     from /ccs/home/shubhankar/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:18,
                     from /ccs/home/shubhankar/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                     from /tmp/pip-install-d__8pdpg/h5py/h5py/api_compat.h:26,
                     from /tmp/pip-install-d__8pdpg/h5py/h5py/defs.c:657:
    /ccs/home/shubhankar/miniconda3/lib/python3.6/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
     #warning "Using deprecated NumPy API, disable it by " \
      ^
    In file included from /tmp/pip-install-d__8pdpg/h5py/h5py/defs.c:657:0:
    /tmp/pip-install-d__8pdpg/h5py/h5py/api_compat.h:27:18: fatal error: hdf5.h: No such file or directory
     #include "hdf5.h"
                      ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
  Rolling back uninstall of h5py
Command "/ccs/home/shubhankar/miniconda3/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-d__8pdpg/h5py/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-5_3ofwib/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-d__8pdpg/h5py/

It breaks at the installation of h5py for some reason. It all starts with a warning message. I am not sure if it is related to it.

Cythonizing /tmp/pip-install-d__8pdpg/h5py/h5py/h5z.pyx
    /tmp/pip-install-d__8pdpg/h5py/.eggs/Cython-0.29.1-py3.6-linux-ppc64le.egg/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-install-d__8pdpg/h5py/h5py/h5z.pxd

And finally

Rolling back uninstall of h5py
Command "/ccs/home/shubhankar/miniconda3/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-d__8pdpg/h5py/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-5_3ofwib/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-d__8pdpg/h5py/

EDIT: Using miniconda for package management.

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 1
  • Comments: 17 (10 by maintainers)

Most upvoted comments

pip install of h5py fails on power because it has to be compiled from source. (The same error would happen on x86 also if pip installing h5py from source.)

To install h5py run: yum install hdf5-devel pip install --global-option=build_ext --global-option=-I/usr/include/hdf5/serial/ --global-option=-L/usr/lib/powerpc64le-linux-gnu/hdf5/serial h5py

Then install tensorflow.

@ghltshubh , -mcpu=power8 -mtune=power8 can be used.

@jayfurmanek - FYI