xgboost: OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized.
For bugs or installation issues, please provide the following information. The more information you provide, the more easily we will be able to offer help and advice.
Environment info
Operating System: Mac OSX Sierra 10.12.1
Compiler:
Package used (python):
xgboost version used: xgboost 0.6a2
If you are using python package, please provide
- The python version and distribution: Pythong 2.7.12
- The command to install
xgboostif you are not installing from source pip install xgboost
Steps to reproduce
- from xgboost import XGBClassifier import numpy as np import matplotlib.pyplot as plt x = np.array([[1,2],[3,4]]) y = np.array([0,1]) clf = XGBClassifier(base_score = 0.005) clf.fit(x,y) plt.hist(clf.feature_importances_)
What have you tried?
See the error message: “OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized. OMP: Hint: This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.”
I tried: import os os.environ[‘KMP_DUPLICATE_LIB_OK’]=‘True’
It can do the job for me. But it is kind of ugly.
I know it might be not the problem of xgboost, but I’m pretty sure this problem happened after I upgrade xgboost using ‘pip install xgboost’. I post the issue here to see if someone had the same problem as me. I have very little knowledge about OpenMP. Please help!
Thanks in advance!
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Reactions: 54
- Comments: 80 (2 by maintainers)
Links to this issue
Commits related to this issue
- fix issue with macOS and tensor https://github.com/dmlc/xgboost/issues/1715#issuecomment-420305786 — committed to XenuIsWatching/eep596_machine_vision_final by XenuIsWatching 5 years ago
- test: fix tests on macOS with OpenMP conflicts In recent tests on both Travis-CI and Github Actions, we found that running test_e2e.py and test_visualizer.py together (regardless of other tests that ... — committed to HazyResearch/fonduer by lukehsiao 4 years ago
- build(travis-ci): fix tests on macOS with OpenMP conflicts In recent tests on both Travis-CI and Github Actions, we found that running test_e2e.py and test_visualizer.py together (regardless of other... — committed to HazyResearch/fonduer by lukehsiao 4 years ago
- build(github-actions): fix tests on macOS with OpenMP conflicts In recent tests on both Travis-CI and Github Actions, we found that running test_e2e.py and test_visualizer.py together (regardless of ... — committed to HazyResearch/fonduer by lukehsiao 4 years ago
- build(github-actions): fix tests on macOS with OpenMP conflicts In recent tests on both Travis-CI and Github Actions, we found that running test_e2e.py and test_visualizer.py together (regardless of ... — committed to HazyResearch/fonduer by lukehsiao 4 years ago
- build(github-actions): fix tests on macOS with OpenMP conflicts In recent tests on both Travis-CI and Github Actions, we found that running test_e2e.py and test_visualizer.py together (regardless of ... — committed to HazyResearch/fonduer by lukehsiao 4 years ago
- build(travis-ci): fix tests on macOS with OpenMP conflicts In recent tests on both Travis-CI and Github Actions, we found that running test_e2e.py and test_visualizer.py together (regardless of other... — committed to HazyResearch/fonduer by lukehsiao 4 years ago
- build(github-actions): fix tests on macOS with OpenMP conflicts In recent tests on both Travis-CI and Github Actions, we found that running test_e2e.py and test_visualizer.py together (regardless of ... — committed to HazyResearch/fonduer by lukehsiao 4 years ago
I tried this and the error stopped !
import os
os.environ[‘KMP_DUPLICATE_LIB_OK’]=‘True’
I had the same error on my Mac with a python program using numpy, keras, and matplotlib. I solved it with ‘conda install nomkl’.
thanks, it works!!!
At least for Mac,
brew uninstall libiomp clang-ompas long as u got gcc v5 from brew it come with openmp
follow steps in: https://github.com/dmlc/xgboost/tree/master/python-package
We can close this issue now.
For people googling this error, the best solution that I found is listed here.
The site lists two method. I prefer method 1, which is to go into your
anaconda3/libdirectory and remove the oldlibiomp5.dylibfile. Nice and simple.conda install nomklworked for me.Can anyone explain what this does and why it fixes the problem?
It should be noted that this is strongly discouraged by the OpenMP devs. From their error message:
I’m having the same issue when I try to use both xgboost and matplotlib.
OS X 10.10.5 Python 2.7.12 xgboost 0.6 matplotlib 1.5.1
My issues were completely unrelated to xgboost, but I got here via google so that I’d share for the sake of others.
I am using keras and matplotlib installed via conda. Setting
KMP_DUPLICATE_LIB_OKjust changed my experience from a warning to an exception/crash. Installnomkldidn’t change anything for me. Eventually I downgraded my version of matplotlib and that fixed things for meconda install matplotlib=2.2.3On Mac M1,
conda install nomklworks.But if it doesn’t work for the first time, you should uninstall and install that command again.
Above didn’t fix my issue after update mac os version ❌
these 2 works:
Thanks!! For me on MAC: import os os.environ[‘KMP_DUPLICATE_LIB_OK’]=‘True’
Thank you this solved my case. My Mac did not have libiomp and clamp-omp, and I did not install nomkl. So I only do these two steps.
I’ve encountered the same problem with xgboost and matplotlib. Same symptom…kernel dies with the same error message as reported by @symPhysics
Mac OS X 10.12.3
Python 3.5.2 |Anaconda custom (x86_64)| (default, Jul 2 2016, 17:52:12) [GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] on darwin
xgboost 0.6a2
matplotlib 2.0.0
I met the same error(not with xgboost though), and I fix it by change
to
it seems that import numpy later would solve this problem. But I don’t understand why this bug would be fixed in this way .
For me, this combination worked:
The primary conflict came from the
brewlibiompbut then later conflicts with thecondabaseenv and mypython 3.7env overopenmpsent me in circles until i sorted out the dependency chain/fallback. The channel source distinction made a huge difference.Got the same issue on my mac, tried suggested method and it works
Please be really careful if you want to try method 1. Everything crashed after I deleted the file and I have to reinstall openmp. Same problem reported in the original csdn link (in Chinese).
[MacOS] This was program termination due to segmentation fault. Through conda I reinstalled pytorch and then also reinstalled torchvision this fixed the issue for me.
We now provide a binary wheel for Mac OSX, which links to
/usr/local/lib/libomp.dylib. Install XGBoost by runningThanks for this;
conda update --allwas what worked for me. Running Mac OS Mojave with Python 3.7 Anaconda (not the new install… dates from last year).have same issue on my Mac Just run - conda update -n base conda it install and update needed dependences
Strangely
conda install nomkldidn’t work for me (don’t havelibiomp clang-ompinstalled on Brew either), onlycould work
Thank you so much, it’s the only thing that worked for me !!
Uninstalling and Reinstalling
intel-openmpwithin the Conda Env solved the issue on my Intel Based MacBookThis works for my intel-Mac!. Thanks alot!
Thanks so much @gpetty !! You’re a life saver! 😄
import os os.environ[‘KMP_DUPLICATE_LIB_OK’]=‘True’
Followed by
Worked for me. Thanks!
you can run this comment
conda install nomkl(macOS Catalina 10.15.5)
Going through the installed package in my env, I noticed that there was an update for mkl.
So I was able to solve my case by updating mkl.
conda install -c intel mklI ran into this problem on a fresh install of anaconda on an a brand new Mac book pro in Anaconda trying to use tensor-flow and keras. I used. ‘conda update --all’ and it fixed the problem.
OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized.
OMP: Hint: This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.
^^^^ This is what did the trick for me. Thanks!
Had the same problem after switching from homebrew/virtualenv installed numpy/keras/matplotlib/tensorflow to miniconda installed. Solved by running
conda install nomklthough I reckon keras (and numpy) wont run as fast as they could if they don’t make use of Intel’s Math Kernel Lib but at least my code still run so ¯\_(ツ)_/¯@gpetty You sir are a true hero. Thank you!
Thanks too, it works for me too.