tensorflow: Illegal instruction (core dumped) after running import tensorflow

System information

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
  • TensorFlow installed from (source or binary): binary
  • TensorFlow version (use command below): 1.6.0-cp27-cp27mu-manylinux1_x86_64 (can only guess since python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)" gives me an error already)
  • Python version: Python 2.7.12
  • Exact command to reproduce: import tensorflow

I created a fresh virtual environment: virtualenv -p python2 test_venv/ And installed tensorflow: pip install --upgrade --no-cache-dir tensorflow import tensorflow gives me Illegal instruction (core dumped)

Please help me understand what’s going on and how I can fix it. Thank you.

CPU information:

-cpu
          description: CPU
          product: Intel(R) Core(TM) i3 CPU       M 330  @ 2.13GHz
          bus info: cpu@0
          version: CPU Version
          capabilities: x86-64 fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm tpr_shadow vnmi flexpriority ept vpid dtherm arat cpufreq

EDIT Stacktrace obtained with gdb:

#0  0x00007fffe5793880 in std::pair<std::__detail::_Node_iterator<std::pair<tensorflow::StringPiece const, std::function<bool (tensorflow::Variant*)> >, false, true>, bool> std::_Hashtable<tensorflow::StringPiece, std::pair<tensorflow::StringPiece const, std::function<bool (tensorflow::Variant*)> >, std::allocator<std::pair<tensorflow::StringPiece const, std::function<bool (tensorflow::Variant*)> > >, std::__detail::_Select1st, std::equal_to<tensorflow::StringPiece>, tensorflow::StringPieceHasher, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_emplace<std::pair<tensorflow::StringPiece, std::function<bool (tensorflow::Variant*)> > >(std::integral_constant<bool, true>, std::pair<tensorflow::StringPiece, std::function<bool (tensorflow::Variant*)> >&&) ()
   from /media/gerry/hdd_1/ws_hdd/test_venv/local/lib/python2.7/site-packages/tensorflow/python/../libtensorflow_framework.so
#1  0x00007fffe5795735 in tensorflow::UnaryVariantOpRegistry::RegisterDecodeFn(std::string const&, std::function<bool (tensorflow::Variant*)> const&) () from /media/gerry/hdd_1/ws_hdd/test_venv/local/lib/python2.7/site-packages/tensorflow/python/../libtensorflow_framework.so
#2  0x00007fffe5770a7c in tensorflow::variant_op_registry_fn_registration::UnaryVariantDecodeRegistration<tensorflow::Tensor>::UnaryVariantDecodeRegistration(std::string const&) ()
   from /media/gerry/hdd_1/ws_hdd/test_venv/local/lib/python2.7/site-packages/tensorflow/python/../libtensorflow_framework.so
#3  0x00007fffe56ea165 in _GLOBAL__sub_I_tensor.cc ()
   from /media/gerry/hdd_1/ws_hdd/test_venv/local/lib/python2.7/site-packages/tensorflow/python/../libtensorflow_framework.so
#4  0x00007ffff7de76ba in call_init (l=<optimized out>, argc=argc@entry=2, argv=argv@entry=0x7fffffffd5c8, env=env@entry=0xa7b4d0)
    at dl-init.c:72
#5  0x00007ffff7de77cb in call_init (env=0xa7b4d0, argv=0x7fffffffd5c8, argc=2, l=<optimized out>) at dl-init.c:30
#6  _dl_init (main_map=main_map@entry=0xa11920, argc=2, argv=0x7fffffffd5c8, env=0xa7b4d0) at dl-init.c:120
#7  0x00007ffff7dec8e2 in dl_open_worker (a=a@entry=0x7fffffffb5c0) at dl-open.c:575
#8  0x00007ffff7de7564 in _dl_catch_error (objname=objname@entry=0x7fffffffb5b0, errstring=errstring@entry=0x7fffffffb5b8, 
    mallocedp=mallocedp@entry=0x7fffffffb5af, operate=operate@entry=0x7ffff7dec4d0 <dl_open_worker>, args=args@entry=0x7fffffffb5c0)
    at dl-error.c:187
#9  0x00007ffff7debda9 in _dl_open (
    file=0x7fffea7cbc34 "/media/gerry/hdd_1/ws_hdd/test_venv/local/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so", mode=-2147483646, caller_dlopen=0x51ad19 <_PyImport_GetDynLoadFunc+233>, nsid=-2, argc=<optimized out>, argv=<optimized out>, env=0xa7b4d0)
    at dl-open.c:660
#10 0x00007ffff75ecf09 in dlopen_doit (a=a@entry=0x7fffffffb7f0) at dlopen.c:66
#11 0x00007ffff7de7564 in _dl_catch_error (objname=0x9b1870, errstring=0x9b1878, mallocedp=0x9b1868, operate=0x7ffff75eceb0 <dlopen_doit>, 
    args=0x7fffffffb7f0) at dl-error.c:187
#12 0x00007ffff75ed571 in _dlerror_run (operate=operate@entry=0x7ffff75eceb0 <dlopen_doit>, args=args@entry=0x7fffffffb7f0) at dlerror.c:163
#13 0x00007ffff75ecfa1 in __dlopen (file=<optimized out>, mode=<optimized out>) at dlopen.c:87
#14 0x000000000051ad19 in _PyImport_GetDynLoadFunc ()
#15 0x000000000051a8e4 in _PyImport_LoadDynamicModule ()
#16 0x00000000005b7b1b in ?? ()
#17 0x00000000004bc3fa in PyEval_EvalFrameEx ()
#18 0x00000000004c136f in PyEval_EvalFrameEx ()
#19 0x00000000004b9ab6 in PyEval_EvalCodeEx ()
#20 0x00000000004b97a6 in PyEval_EvalCode ()
#21 0x00000000004b96df in PyImport_ExecCodeModuleEx ()
#22 0x00000000004b2b06 in ?? ()
#23 0x00000000004a4ae1 in ?? ()

EDIT 2 Bazel version: N/A CUDA/cuDNN version: N/A GPU model and memory: N/A

After downgrading to an older version of tensorflow the error goes away. I’ve been advised that my CPU (see information above) might not work with some improvements in the new API. If this is the case, I suppose there’s no solution for my problem. Therefore, I will close this thread. Feel free to correct me though. Thank you for your support

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 57
  • Comments: 100 (5 by maintainers)

Commits related to this issue

Most upvoted comments

Try running pip uninstall tensorflow And then pip install tensorflow==1.5

EDIT just to give credit, solution is from here: https://stackoverflow.com/questions/49094597/illegal-instruction-core-dumped-after-running-import-tensorflow

I’m having the same (or similar) “illegal instruction” problem when I run

import tensorflow as tf

I’m only using the CPU 1.6 version on 64 bit Ubuntu Linux.

After downgrading to the CPU 1.5 version, it doesn’t have this problem.

I am encountering this issue as well with tensorflow-gpu 1.6.0, on linux, using python 3.6.4. I have installed tensorflow using pip itself. Simply running this produces a SIGILL:

$ python3 -m tensorflow
zsh: illegal hardware instruction  python3 -m tensorflow

I get stack traces similar to what is mentioned in this ticket’s description.

This seems to be occurring due to the use of AVX instructions in the latest Tensorflow packages uploaded to pip. Running python3 through GDB and disassembling the crashing function points to this instruction:

=> 0x00007fffb9689660 <+80>:    vmovdqu 0x10(%r13),%xmm1

Which is an AVX instruction not supported on older or less-featureful CPUs that do not have AVX support. The tensorflow(-gpu) 1.5.0 pip packages do not use AVX instructions, and thus there are no problems using it with these CPUs.

The solution would be for a build of tensorflow(-gpu) that is not compiled with AVX instructions to be published (or to build a copy locally). The provided installation instructions do not mention any specific CPU requirements nor how to determine compatibility with the provided binaries.

In the meantime, reverting to tensorflow(-gpu) 1.5.0 using something like what @NinemillaKA mentioned above is an effective workaround.

Same here. With the latest wheel, I had the illegal instruction problem on Ubuntu 16.04, however I downgraded to tensorflow-gpu==1.5 and it works!

Why is this issue closed? It’s still occurring in TensorFlow 1.10.

The alternative to having a different build for each architecture type is to use dynamic dispatch. IE, PyTorch has one binary for all architectures and selects most efficient ops during runtime @caisq

@konnerthg Downgrading to 1.5 is just work around, this issue is not solved yet. Which commit/PR solved this issue?

I have the same issue, and, as many have commented, downgrade from 1.6.0 to 1.5.0.

For the record, I tried running tensorflow (CPU-only version) on 2 different computers:

Computer 1:

OS = Ubuntu 16.04 x64 LTS
Python = Python 3.6
pip version = 9.0.1
tensorflow version = TensorFlow 1.6.0
CPU = Intel Core 2 Quad Q6600  @2.40GHz

Computer 2:

OS = Ubuntu 16.04 x64 LTS
Python = Python 3.6
pip version = 9.0.1
tensorflow version = TensorFlow 1.6.0
CPU = Intel Celeron N2820 @2.413GHz

I agree with @nacl that we should have those requirements about the instruction set more clear, and if possible, a separated, updated build for processors that doesn’t support AVX instructions. To be honest, I find a bit discouraging have to work with outdated version of any technology, I think many feel the same.

downgrade to 1.5 worked for me, too

I’m having this problem with Tensorflow 2 runing in a virtual environment in Ubuntu 18.04. It just blows my mind that the Tensorflow developers would put TF 2 as ready and available with this crap happening. NOT Impressed, you TF developers .

This is stated at https://www.tensorflow.org/install/install_sources Note: Starting from 1.6 release, our prebuilt binaries will use AVX instructions. Older CPUs may not be able to execute these binaries.

I think that might have been mentioned at the much more prominent location!

This is still an issue in 1.6 and potentially in 1.7. Why is this closed? @yaroslavvb 's solution seems reasonable. I have downgraded to 1.5 for now.

This is really weird. Does anyone know what causes the issue? I’m surprised that TensorFlow 1.6 would have a bug this big.

To make this issue more constructive, I think it would be useful it TensorFlow would check for instructions it requires on the CPU first, and print an error if they are missing. Similarly how it currently says that some instructions are available but not compiled against. Then it would be easier to differentiate between bugs and simply not using correct binary for a given CPU.

well this issue seems to still going on… but i have a very simple solution that will make you follow the trends at the same time 😃.

INSTALL CONDA !!! to install conda, just google it 😃

create a conda evironment if you like:

conda create <name_of_environment>

and simply run:

conda install tensorflow

if the last command doesn’t work then search through another channel by running:

anaconda search tensorflow

this outputs a list of channels from where you can download it. choose one channel and run:

anaconda show <channel>

this will prompt you the command you need to enter to downloading tensorflow. Run that command.

That’s it !

MOVE TO CONDA !!!

On Thu, Aug 23, 2018 at 10:38 PM Alexis Wilke notifications@github.com wrote:

Indeed, I followed the instructions found at https://www.tensorflow.org/install/install_linux and get nothing more than the “Illegal instruction (core dumped)” when testing as requested on that installation page.

This issue should either not be closed or have an actual solution that makes sense (i.e. not installing version 1.5)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/17411#issuecomment-415592389, or mute the thread https://github.com/notifications/unsubscribe-auth/AWESkUXD-4XomnnPgF6D5QToWVH74JzAks5uTy7PgaJpZM4SbSGu .

You have to build from sources it’s not that complicated

For those that do not want to downgrade if you build from source the problem is resolved.

Same problem on this Docker image running in a Paperspace notebook.

Issue is with versions 1.6 and 1.7

Still have the problem with tensorflow 1.14.0 and 2.0.0b1.

Yes. Indeed. It would be better, though, if the software would tell me rather than just crash. I don’t have a problem with the requirement, just the way it is handled… On Linux, it would be very easy to check in /proc/cpuinfo for the flags line where avx would need to appear. If not, generate an error and exit(1).

Here is my flags on my old computer without AVX

flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ssbd ibrs ibpb stibp kaiser tpr_shadow vnmi flexpriority ept vpid dtherm ida flush_l1d

To fix this on v1.6 and above, you need to compile tensorflow from source.

I don’t have the knowledge to say if the requirement of AVX makes sense or not. What I know is that the problem presents itself not only with older CPUs, but also with fairly recent ones, like mine Intel N5000. I get that doing deep learning on a N5000 is a bit of a stretch, but if the tensorflow is supported also by the RaspberryPi, I don’t see the problem.

Anyway, I installed the last version of TensorFlow (2.0) on my Intel N5000 by compiling it from source. It took 14 hours because I had to run the compilation on a single core, since it needs a lot of RAM and I have only 4Gb invited to the party.

I took inspiration from this guide here but the experience was far from smooth, there were constantly dependencies missing that I need to install and re-launch the compilation. And some other stuff too that I solved when the compilation crashed.

Have fun and thanks for the hassle. Providing through pip a binary already compiled for non-AVX was clearly too much to add in your continuous integration workflow

Thanks for all this solve my issue on Python 3.6

_ (tensorflow) naniny@Aspire-E5-573:~$ pip unistall tensorflow

_(tensorflow) naniny@Aspire-E5-573:~$ pip install tensorflow==1.5

_(tensorflow) naniny@Aspire-E5-573:~$ python

_ (tensorflow) naniny@Aspire-E5-573:~$ import tensorflow as tf

now works without any problem …

Same issue in tensorflow-gpu 1.8.0 as well

OS: Ubuntu 18.04 Python: 3.6.5

Same error on Linux Mint 19 with 2.0.0b1. Just installed with pip3 like instructed from the install page of the official site

This solved my issue: After installing NVIDIA driver, CUDA Toolkit, and CUDNN. First uninstall tensorflow-gpu:

$ pip uninstall tensorflow-gpu

Then install tensorflow-gpu using Anaconda:

$ conda create -n tensorflow
$ conda install tensorflow-gpu -n tensorflow

CUDA9.0+cuDNN7.1+Tensorflow-gpu1.5 is working

I solved it. Not by buying new CPU😏