xgboost: OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized.

For bugs or installation issues, please provide the following information. The more information you provide, the more easily we will be able to offer help and advice.

Environment info

Operating System: Mac OSX Sierra 10.12.1

Compiler:

Package used (python):

xgboost version used: xgboost 0.6a2

If you are using python package, please provide

  1. The python version and distribution: Pythong 2.7.12
  2. The command to install xgboost if you are not installing from source pip install xgboost

Steps to reproduce

  1. from xgboost import XGBClassifier import numpy as np import matplotlib.pyplot as plt x = np.array([[1,2],[3,4]]) y = np.array([0,1]) clf = XGBClassifier(base_score = 0.005) clf.fit(x,y) plt.hist(clf.feature_importances_)

What have you tried?

See the error message: “OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized. OMP: Hint: This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.

I tried: import os os.environ[‘KMP_DUPLICATE_LIB_OK’]=‘True’

It can do the job for me. But it is kind of ugly.


I know it might be not the problem of xgboost, but I’m pretty sure this problem happened after I upgrade xgboost using ‘pip install xgboost’. I post the issue here to see if someone had the same problem as me. I have very little knowledge about OpenMP. Please help!
Thanks in advance!

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Reactions: 54
  • Comments: 80 (2 by maintainers)

Commits related to this issue

Most upvoted comments

I tried this and the error stopped !

import os

os.environ[‘KMP_DUPLICATE_LIB_OK’]=‘True’

I had the same error on my Mac with a python program using numpy, keras, and matplotlib. I solved it with ‘conda install nomkl’.

I tried this and the error stopped !

import os

os.environ[‘KMP_DUPLICATE_LIB_OK’]=‘True’

thanks, it works!!!

At least for Mac,

brew uninstall libiomp clang-omp

as long as u got gcc v5 from brew it come with openmp

follow steps in: https://github.com/dmlc/xgboost/tree/master/python-package

We can close this issue now.

For people googling this error, the best solution that I found is listed here.

The site lists two method. I prefer method 1, which is to go into your anaconda3/lib directory and remove the old libiomp5.dylib file. Nice and simple.

conda install nomkl worked for me.

Can anyone explain what this does and why it fixes the problem?

I tried this and the error stopped !

import os

os.environ[‘KMP_DUPLICATE_LIB_OK’]=‘True’

It should be noted that this is strongly discouraged by the OpenMP devs. From their error message:

The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results.

I’m having the same issue when I try to use both xgboost and matplotlib.

OS X 10.10.5 Python 2.7.12 xgboost 0.6 matplotlib 1.5.1

My issues were completely unrelated to xgboost, but I got here via google so that I’d share for the sake of others.

I am using keras and matplotlib installed via conda. Setting KMP_DUPLICATE_LIB_OK just changed my experience from a warning to an exception/crash. Install nomkl didn’t change anything for me. Eventually I downgraded my version of matplotlib and that fixed things for me conda install matplotlib=2.2.3

On Mac M1, conda install nomkl works.
But if it doesn’t work for the first time, you should uninstall and install that command again.

conda uninstall nomkl
conda install nomkl

Above didn’t fix my issue after update mac os version ❌

these 2 works:

conda uninstall intel-openmp
conda install -c intel openmp

Thanks!! For me on MAC: import os os.environ[‘KMP_DUPLICATE_LIB_OK’]=‘True’

Thank you this solved my case. My Mac did not have libiomp and clamp-omp, and I did not install nomkl. So I only do these two steps.

conda uninstall intel-openmp
conda install -c intel openmp

For me, this combination worked:

brew uninstall libiomp clamp-omp
conda uninstall intel-openmp -n base
conda install -c intel openmp -n myenv
conda install nomkl -n myenv

The primary conflict came from the brew libiomp but then later conflicts with the conda base env and my python 3.7 env over openmp sent me in circles until i sorted out the dependency chain/fallback. The channel source distinction made a huge difference.

I’ve encountered the same problem with xgboost and matplotlib. Same symptom…kernel dies with the same error message as reported by @symPhysics

Mac OS X 10.12.3

Python 3.5.2 |Anaconda custom (x86_64)| (default, Jul 2 2016, 17:52:12) [GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] on darwin

xgboost 0.6a2

matplotlib 2.0.0

I met the same error(not with xgboost though), and I fix it by change

import numpy 
import torch

to

import torch
import numpy

it seems that import numpy later would solve this problem. But I don’t understand why this bug would be fixed in this way .

For me, this combination worked:

brew uninstall libiomp clamp-omp
conda uninstall intel-openmp -n base
conda install -c intel openmp -n myenv
conda install nomkl -n myenv

The primary conflict came from the brew libiomp but then later conflicts with the conda base env and my python 3.7 env over openmp sent me in circles until i sorted out the dependency chain/fallback. The channel source distinction made a huge difference.

Got the same issue on my mac, tried suggested method and it works

import os
os.environ['KMP_DUPLICATE_LIB_OK']='True'

For people googling this error, the best solution that I found is listed here.

The site lists two method. I prefer method 1, which is to go into your anaconda3/lib directory and remove the old libiomp5.dylib file. Nice and simple.

Please be really careful if you want to try method 1. Everything crashed after I deleted the file and I have to reinstall openmp. Same problem reported in the original csdn link (in Chinese).

I tried this and the error stopped ! import os os.environ[‘KMP_DUPLICATE_LIB_OK’]=‘True’

[MacOS] This was program termination due to segmentation fault. Through conda I reinstalled pytorch and then also reinstalled torchvision this fixed the issue for me.

We now provide a binary wheel for Mac OSX, which links to /usr/local/lib/libomp.dylib. Install XGBoost by running

python -m pip install xgboost

Thanks for this; conda update --all was what worked for me. Running Mac OS Mojave with Python 3.7 Anaconda (not the new install… dates from last year).

have same issue on my Mac Just run - conda update -n base conda it install and update needed dependences

brew uninstall libiomp clang-omp

Error: No such keg: /usr/local/Cellar/libiomp

Strangely conda install nomkl didn’t work for me (don’t have libiomp clang-omp installed on Brew either), only

os.environ['KMP_DUPLICATE_LIB_OK']='True'

could work

On Mac M1, conda install nomkl works. But if it doesn’t work for the first time, you should uninstall and install that command again.

Above didn’t fix my issue after update mac os version ❌

these 2 works:

conda uninstall intel-openmp
conda install -c intel openmp

Thank you so much, it’s the only thing that worked for me !!

Uninstalling and Reinstalling intel-openmp within the Conda Env solved the issue on my Intel Based MacBook

On Mac M1, conda install nomkl works. But if it doesn’t work for the first time, you should uninstall and install that command again.

Above didn’t fix my issue after update mac os version ❌

these 2 works:

conda uninstall intel-openmp
conda install -c intel openmp

This works for my intel-Mac!. Thanks alot!

I had the same error on my Mac with a python program using numpy, keras, and matplotlib. I solved it with ‘conda install nomkl’.

Thanks so much @gpetty !! You’re a life saver! 😄

import os os.environ[‘KMP_DUPLICATE_LIB_OK’]=‘True’

Followed by

conda install nomkl

Worked for me. Thanks!

you can run this comment conda install nomkl

(macOS Catalina 10.15.5)

Going through the installed package in my env, I noticed that there was an update for mkl.

So I was able to solve my case by updating mkl.

conda install -c intel mkl

I ran into this problem on a fresh install of anaconda on an a brand new Mac book pro in Anaconda trying to use tensor-flow and keras. I used. ‘conda update --all’ and it fixed the problem.

OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized.

OMP: Hint: This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.

Had the same problem after switching from homebrew/virtualenv installed numpy/keras/matplotlib/tensorflow to miniconda installed. Solved by running conda install nomkl though I reckon keras (and numpy) wont run as fast as they could if they don’t make use of Intel’s Math Kernel Lib but at least my code still run so ¯_(ツ)_/¯

^^^^ This is what did the trick for me. Thanks!

Had the same problem after switching from homebrew/virtualenv installed numpy/keras/matplotlib/tensorflow to miniconda installed. Solved by running conda install nomkl though I reckon keras (and numpy) wont run as fast as they could if they don’t make use of Intel’s Math Kernel Lib but at least my code still run so ¯\_(ツ)_/¯

@gpetty You sir are a true hero. Thank you!

I had the same error on my Mac with a python program using numpy, keras, and matplotlib. I solved it with ‘conda install nomkl’.

Thanks too, it works for me too.