LightGBM: LightGBM and gcc 8 in MacOS: `Library not loaded: /usr/local/opt/gcc/lib/gcc/7/libgomp.1.dylib`

For Mac Users

Version 10.13.4
Python 3.6.5

The 2nd of May gcc-8 was released, which is a major released.

Therefore, if, since then, you run

brew update
brew upgrade

gcc-8 version will be installed:

➜  ~ gcc-8 --version
gcc-8 (Homebrew GCC 8.1.0) 8.1.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

ISSUE

If the lightGBM version you have was built with gcc-7, when you try to load it now you might get:

OSError: dlopen(/usr/local/lib/python3.6/site-packages/lightgbm/lib_lightgbm.so, 6): Library not loaded: /usr/local/opt/gcc/lib/gcc/7/libgomp.1.dylib
  Referenced from: /usr/local/lib/python3.6/site-packages/lightgbm/lib_lightgbm.so
  Reason: image not found

SOLUTION

I came around this error by installing it from source doing:

pip uninstall lightgbm
git clone --recursive https://github.com/Microsoft/LightGBM ; cd LightGBM
export CXX=g++-8 CC=gcc-8
mkdir build ; cd build
cmake ..
make -j4

and then

pip install --no-binary :all: lightgbm
In [1]: import lightgbm

In [2]: lightgbm.__version__
Out[2]: '2.1.1'

By the way, similar issue will happen to xgboost.

If someone thinks this is more for stackoverflow I am happy to remove it from here immediately

Cheers

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 200
  • Comments: 25 (1 by maintainers)

Most upvoted comments

Most developers use Mac, and lgb can’t be installed on mac? Why did you do that?

To solve the problem I installed the libomp using brew: brew install libomp

source: https://stackoverflow.com/a/55958281/1074245

I’m getting the following error after being able to successfully install lightgbm, even though libomp is already installed. Does anyone have any suggestions ?

~/Downloads/LightGBM/build  python -c "import lightgbm"                                                                              18:12:37   master ?
/Users/user121/anaconda3/lib/python3.6/site-packages/lightgbm/__init__.py:46: UserWarning: Starting from version 2.2.1, the library file in distribution wheels for macOS is built by the Apple Clang (Xcode_9.4.1) compiler.
This means that in case of installing LightGBM from PyPI via the ``pip install lightgbm`` command, you don't need to install the gcc compiler anymore.
Instead of that, you need to install the OpenMP library, which is required for running LightGBM on the system with the Apple Clang compiler.
You can install the OpenMP library by the following command: ``brew install libomp``.
  "You can install the OpenMP library by the following command: ``brew install libomp``.", UserWarning)```

Env OSX, export CXX=g++-9 CC=gcc-9 solved my issue

I think one should do brew install gcc and then brew info gcc to check the version. Mine return

gcc: stable 9.1.0 (bottled), HEAD GNU compiler collection

so I use gcc-9 instead of gcc-8 in this thread

If you don’t mind doing a conda install, try:

import sys !conda install --yes --prefix {sys.prefix} -c conda-forge lightgbm

This resolved the problem for me (run in a jupyter notebook cell)

I also found a small hack for OsX High Sierra which has by default gcc 8, and lightgbm required gcc 7. I build the lightgbm using gcc7 but the error still persisted with OSError: dlopen(/usr/local/lib/python3.6/site-packages/lightgbm/lib_lightgbm.so, 6): Library not loaded: /usr/local/opt/gcc/lib/gcc/7/libgomp.1.dylib So I decided to investigate what is in the /usr/local/opt/gcc/lib/gcc folder. … Interestingly it had a folder called 8( as default is gcc8). Then I tried compiling again the lightgbm with gcc7 but to no help. Finally, I made a small hack I copied the contents in a similar folder /usr/local/opt/gcc@7/lib/gcc/7 to /usr/local/opt/gcc/lib/gcc/ and voila it worked!!! sudo cp -a /usr/local/opt/gcc@7/lib/gcc/7 /usr/local/opt/gcc/lib/gcc/

Here is a great doc for lightgbm if u want to use the highPerformance using MPI or GPU or HDFS then refer here: https://media.readthedocs.org/pdf/lightgbm/latest/lightgbm.pdf

For Mac OS, this worked for me: https://lightgbm.readthedocs.io/en/latest/Installation-Guide.html

macOS On macOS LightGBM can be built using CMake and Apple Clang or gcc.

Apple Clang Only Apple Clang version 8.1 or higher is supported.

Install CMake (3.12 or higher):

brew install cmake Install OpenMP:

brew install libomp Run the following commands:

git clone --recursive https://github.com/microsoft/LightGBM ; cd LightGBM mkdir build ; cd build

For Mojave (10.14) cmake -DOpenMP_C_FLAGS=“-Xpreprocessor -fopenmp -I$(brew --prefix libomp)/include” -DOpenMP_C_LIB_NAMES=“omp” -DOpenMP_CXX_FLAGS=“-Xpreprocessor -fopenmp -I$(brew --prefix libomp)/include” -DOpenMP_CXX_LIB_NAMES=“omp” -DOpenMP_omp_LIBRARY=$(brew --prefix libomp)/lib/libomp.dylib …

For High Sierra or earlier (<= 10.13) cmake …

make -j4 gcc Install CMake (3.2 or higher):

brew install cmake Install gcc:

brew install gcc Run the following commands:

git clone --recursive https://github.com/microsoft/LightGBM ; cd LightGBM export CXX=g+±7 CC=gcc-7 # replace “7” with version of gcc installed on your machine mkdir build ; cd build cmake … make -j4