LightGBM: LightGBM and gcc 8 in MacOS: `Library not loaded: /usr/local/opt/gcc/lib/gcc/7/libgomp.1.dylib`
For Mac Users
Version 10.13.4
Python 3.6.5
The 2nd of May gcc-8 was released, which is a major released.
Therefore, if, since then, you run
brew update
brew upgrade
gcc-8 version will be installed:
➜ ~ gcc-8 --version
gcc-8 (Homebrew GCC 8.1.0) 8.1.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
ISSUE
If the lightGBM version you have was built with gcc-7, when you try to load it now you might get:
OSError: dlopen(/usr/local/lib/python3.6/site-packages/lightgbm/lib_lightgbm.so, 6): Library not loaded: /usr/local/opt/gcc/lib/gcc/7/libgomp.1.dylib
Referenced from: /usr/local/lib/python3.6/site-packages/lightgbm/lib_lightgbm.so
Reason: image not found
SOLUTION
I came around this error by installing it from source doing:
pip uninstall lightgbm
git clone --recursive https://github.com/Microsoft/LightGBM ; cd LightGBM
export CXX=g++-8 CC=gcc-8
mkdir build ; cd build
cmake ..
make -j4
and then
pip install --no-binary :all: lightgbm
In [1]: import lightgbm
In [2]: lightgbm.__version__
Out[2]: '2.1.1'
By the way, similar issue will happen to xgboost.
If someone thinks this is more for stackoverflow I am happy to remove it from here immediately
Cheers
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 200
- Comments: 25 (1 by maintainers)
Most developers use Mac, and lgb can’t be installed on mac? Why did you do that?
To solve the problem I installed the libomp using brew:
brew install libompsource: https://stackoverflow.com/a/55958281/1074245
I’m getting the following error after being able to successfully install lightgbm, even though
libompis already installed. Does anyone have any suggestions ?Env OSX,
export CXX=g++-9 CC=gcc-9solved my issueI think one should do
brew install gccand thenbrew info gccto check the version. Mine returnso I use
gcc-9instead ofgcc-8in this threadIf you don’t mind doing a conda install, try:
import sys !conda install --yes --prefix {sys.prefix} -c conda-forge lightgbm
This resolved the problem for me (run in a jupyter notebook cell)
I also found a small hack for OsX High Sierra which has by default gcc 8, and lightgbm required gcc 7. I build the lightgbm using gcc7 but the error still persisted with OSError: dlopen(/usr/local/lib/python3.6/site-packages/lightgbm/lib_lightgbm.so, 6): Library not loaded: /usr/local/opt/gcc/lib/gcc/7/libgomp.1.dylib So I decided to investigate what is in the /usr/local/opt/gcc/lib/gcc folder. … Interestingly it had a folder called 8( as default is gcc8). Then I tried compiling again the lightgbm with gcc7 but to no help. Finally, I made a small hack I copied the contents in a similar folder /usr/local/opt/gcc@7/lib/gcc/7 to /usr/local/opt/gcc/lib/gcc/ and voila it worked!!!
sudo cp -a /usr/local/opt/gcc@7/lib/gcc/7 /usr/local/opt/gcc/lib/gcc/Here is a great doc for lightgbm if u want to use the highPerformance using MPI or GPU or HDFS then refer here: https://media.readthedocs.org/pdf/lightgbm/latest/lightgbm.pdf
For Mac OS, this worked for me: https://lightgbm.readthedocs.io/en/latest/Installation-Guide.html
macOS On macOS LightGBM can be built using CMake and Apple Clang or gcc.
Apple Clang Only Apple Clang version 8.1 or higher is supported.
Install CMake (3.12 or higher):
brew install cmake Install OpenMP:
brew install libomp Run the following commands:
git clone --recursive https://github.com/microsoft/LightGBM ; cd LightGBM mkdir build ; cd build
For Mojave (10.14) cmake -DOpenMP_C_FLAGS=“-Xpreprocessor -fopenmp -I$(brew --prefix libomp)/include” -DOpenMP_C_LIB_NAMES=“omp” -DOpenMP_CXX_FLAGS=“-Xpreprocessor -fopenmp -I$(brew --prefix libomp)/include” -DOpenMP_CXX_LIB_NAMES=“omp” -DOpenMP_omp_LIBRARY=$(brew --prefix libomp)/lib/libomp.dylib …
For High Sierra or earlier (<= 10.13) cmake …
make -j4 gcc Install CMake (3.2 or higher):
brew install cmake Install gcc:
brew install gcc Run the following commands:
git clone --recursive https://github.com/microsoft/LightGBM ; cd LightGBM export CXX=g+±7 CC=gcc-7 # replace “7” with version of gcc installed on your machine mkdir build ; cd build cmake … make -j4
@sshleifer Please read this https://github.com/microsoft/LightGBM/issues/1369#issuecomment-428971946.