PyTables: Install failure on macOS 11.0.1 Homebrew python 3.9 on Intel Hardware
Pip install of PyTables fails with an error.
On a freshly installed MacOS 11.0.1, on Intel CPU based MBP, using Homebrew python 3.9. (3.9.0_1):
>> pip3 install tables
Collecting tables
Using cached tables-3.6.1.tar.gz (4.6 MB)
Requirement already satisfied: numpy>=1.9.3 in /usr/local/lib/python3.9/site-packages (from tables) (1.19.4)
Requirement already satisfied: numexpr>=2.6.2 in /usr/local/lib/python3.9/site-packages (from tables) (2.7.1)
Building wheels for collected packages: tables
Building wheel for tables (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /usr/local/opt/python@3.9/bin/python3.9 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/x6/37vsz40n4nbfx0nb9xdq0ghr0000gn/T/pip-install-do9yfy3k/tables/setup.py'"'"'; __file__='"'"'/private/var/folders/x6/37vsz40n4nbfx0nb9xdq0ghr0000gn/T/pip-install-do9yfy3k/tables/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/x6/37vsz40n4nbfx0nb9xdq0ghr0000gn/T/pip-wheel-3hnmjui6
cwd: /private/var/folders/x6/37vsz40n4nbfx0nb9xdq0ghr0000gn/T/pip-install-do9yfy3k/tables/
Complete output (297 lines):
* Using Python 3.9.0 (default, Oct 27 2020, 14:15:17)
* USE_PKGCONFIG: True
* Found HDF5 headers at ``/usr/local/include``, library at ``/usr/local/lib``.
* Found LZO 2 headers at ``/usr/local/include``, library at ``/usr/local/lib``.
* Skipping detection of LZO 1 since LZO 2 has already been found.
/var/folders/x6/37vsz40n4nbfx0nb9xdq0ghr0000gn/T/BZ2_bzlibVersion9v9eowo8.c:2:5: error: implicit declaration of function 'BZ2_bzlibVersion' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BZ2_bzlibVersion();
^
1 error generated.
* Could not find bzip2 headers and library; disabling support for it.
/var/folders/x6/37vsz40n4nbfx0nb9xdq0ghr0000gn/T/blosc_list_compressorsqnh3sxrx.c:2:5: error: implicit declaration of function 'blosc_list_compressors' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
blosc_list_compressors();
^
1 error generated.
* Could not find blosc headers and library; using internal sources.
SSE2 detected and enabled
AVX2 detected and enabled
/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py:274: UserWarning: Unknown distribution option: 'extra_require'
warnings.warn(msg)
.....
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:256:24: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
state->start = LSEEK(state->fd, 0, SEEK_CUR);
^
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
# define LSEEK lseek
^
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:256:24: note: did you mean 'fseek'?
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
# define LSEEK lseek
^
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stdio.h:162:6: note: 'fseek' declared here
int fseek(FILE *, long, int);
^
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:355:9: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
if (LSEEK(state->fd, state->start, SEEK_SET) == -1)
^
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
# define LSEEK lseek
^
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:396:15: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
ret = LSEEK(state->fd, offset - state->x.have, SEEK_CUR);
^
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
# define LSEEK lseek
^
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:492:14: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
offset = LSEEK(state->fd, 0, SEEK_CUR);
^
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
# define LSEEK lseek
^
4 errors generated.
error: command '/usr/bin/clang' failed with exit code 1
----------------------------------------
ERROR: Failed building wheel for tables
Running setup.py clean for tables
Failed to build tables
Installing collected packages: tables
Running setup.py install for tables ... error
ERROR: Command errored out with exit status 1:
command: /usr/local/opt/python@3.9/bin/python3.9 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/x6/37vsz40n4nbfx0nb9xdq0ghr0000gn/T/pip-install-do9yfy3k/tables/setup.py'"'"'; __file__='"'"'/private/var/folders/x6/37vsz40n4nbfx0nb9xdq0ghr0000gn/T/pip-install-do9yfy3k/tables/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/x6/37vsz40n4nbfx0nb9xdq0ghr0000gn/T/pip-record-wyb_qo9g/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.9/tables
cwd: /private/var/folders/x6/37vsz40n4nbfx0nb9xdq0ghr0000gn/T/pip-install-do9yfy3k/tables/
Complete output (297 lines):
* Using Python 3.9.0 (default, Oct 27 2020, 14:15:17)
* USE_PKGCONFIG: True
* Found HDF5 headers at ``/usr/local/include``, library at ``/usr/local/lib``.
* Found LZO 2 headers at ``/usr/local/include``, library at ``/usr/local/lib``.
* Skipping detection of LZO 1 since LZO 2 has already been found.
/var/folders/x6/37vsz40n4nbfx0nb9xdq0ghr0000gn/T/BZ2_bzlibVersion6dn80ei1.c:2:5: error: implicit declaration of function 'BZ2_bzlibVersion' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
BZ2_bzlibVersion();
^
1 error generated.
* Could not find bzip2 headers and library; disabling support for it.
/var/folders/x6/37vsz40n4nbfx0nb9xdq0ghr0000gn/T/blosc_list_compressorsab9_e1pl.c:2:5: error: implicit declaration of function 'blosc_list_compressors' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
blosc_list_compressors();
^
1 error generated.
* Could not find blosc headers and library; using internal sources.
SSE2 detected and enabled
AVX2 detected and enabled
/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py:274: UserWarning: Unknown distribution option: 'extra_require'
warnings.warn(msg)
running install
....
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:256:24: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
state->start = LSEEK(state->fd, 0, SEEK_CUR);
^
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
# define LSEEK lseek
^
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:256:24: note: did you mean 'fseek'?
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
# define LSEEK lseek
^
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stdio.h:162:6: note: 'fseek' declared here
int fseek(FILE *, long, int);
^
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:355:9: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
if (LSEEK(state->fd, state->start, SEEK_SET) == -1)
^
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
# define LSEEK lseek
^
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:396:15: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
ret = LSEEK(state->fd, offset - state->x.have, SEEK_CUR);
^
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
# define LSEEK lseek
^
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:492:14: error: implicit declaration of function 'lseek' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
offset = LSEEK(state->fd, 0, SEEK_CUR);
^
c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
# define LSEEK lseek
^
4 errors generated.
error: command '/usr/bin/clang' failed with exit code 1
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/opt/python@3.9/bin/python3.9 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/x6/37vsz40n4nbfx0nb9xdq0ghr0000gn/T/pip-install-do9yfy3k/tables/setup.py'"'"'; __file__='"'"'/private/var/folders/x6/37vsz40n4nbfx0nb9xdq0ghr0000gn/T/pip-install-do9yfy3k/tables/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/x6/37vsz40n4nbfx0nb9xdq0ghr0000gn/T/pip-record-wyb_qo9g/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.9/tables Check the logs for full command output.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 2
- Comments: 28 (4 by maintainers)
Commits related to this issue
- gha workflow fix putative fix for macos+py3.9 `tables` install error: https://github.com/PyTables/PyTables/issues/828#issuecomment-729306723 — committed to chanzuckerberg/cellxgene by atolopko-czi 3 years ago
FWIW i came here with the same problem as the OP, based on this suggestion appear to have solved it with
by itself didn’t work for me. I adapted this comment: https://github.com/freqtrade/freqtrade/issues/4162#issuecomment-890377818 to let the build process find my brew-installed HDF5 and Blosc:
@YubinXie could you try executing
brew install hdf5 c-blosc lzo bzip2? Probably you forgot to link c-blosc or any other library withbrew link --overwrite c-blosc.I tried these commands on my MAC but second command fails on my system.
If it helps, had the same problem when tried to install tables in a venv and tried every possible solution listed here and elsewhere without success. Also tried to install tables with brew without success. Today I tried one more time with
python3 -m pip install tablesand it worked. No explanation to it whatsoever as, if memory serves me well (because I’ve been working on this for almost two weeks) I am pretty sure I tried with pip and pip3 with no success.
Hope this helps in the future.
P.S. I have a M2 Mac and py3.11.5
I worked around the use of pytables in my python code base by replacing
x.to_hdf( )withx.to_pickle( ). Saved files are smaller and the code is faster.I also verified that on the Intel machine in which pytables fails with a run-time crash, h5py works just fine for creating and reading hdf5 files. So, this means that hdf5 installation under brew works for h5py but not for pytables.
Moreover, h5py.run_tests() also crashes when running
tables.openfile( )with the same crash on hdf5 version mismatch. Usingfixed the crash in
h5py.runt_tests()intables.openfile().Now I am on Mac OS 11.5.2, Homebrew version 3.2.6 Python version 3.9.6.
pip3 install tables.works fine afterHomebrew reports hdf5 version 1.12.1, c-blosc version 1.20.1. Components of hdf5 work fine in other software packages under brew (octave version 3.6.3). Yet, tables reports linking to hdf5 version 1.12.0 and crashes at runtime.
See #903
@avalentino,
I reinstalled using brew but issue isn’t solved.
Log of reinstallation using brew:
Dear @sunilshah,
I believe this is related to the switch to clang 12 starting with macOS 10.15 (Catalina), see also https://github.com/microsoft/jericho/issues/37 and https://github.com/kaldi-asr/kaldi/issues/4155.
Maybe you can apply the gist how @FrancescAlted fixed this with https://github.com/Blosc/c-blosc/commit/5fc66f7cdcff969b47332ed843be5f344a5543cc and https://github.com/Blosc/c-blosc/commit/406f5960da86f0932bd3609dae0bc70d5c785919, originally coming from https://github.com/Blosc/python-blosc/issues/229 by @xytxytxyt.
With kind regards, Andreas.