elastalert: ImportError: failed to find libmagic. Check your installation

I believe this may be an issue with the magic package used by thehive4py/api.py

After installing elastalert v 0.1.3.6 I get the following stacktrace when I try to run an elastalert test with a folder of rules

Traceback (most recent call last):
  File "/home/elastalert/.local/bin/elastalert", line 11, in <module>
    load_entry_point('elastalert==0.1.36', 'console_scripts', 'elastalert')()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 480, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2691, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2322, in load
    return self.resolve()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2328, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/elastalert/.local/lib/python2.7/site-packages/elastalert/elastalert.py", line 21, in <module>
    from alerts import DebugAlerter
  File "/home/elastalert/.local/lib/python2.7/site-packages/elastalert/alerts.py", line 31, in <module>
    from thehive4py.api import TheHiveApi
  File "/home/elastalert/.local/lib/python2.7/site-packages/thehive4py/api.py", line 7, in <module>
    import magic
  File "/home/elastalert/.local/lib/python2.7/site-packages/magic.py", line 181, in <module>
    raise ImportError('failed to find libmagic.  Check your installation')
ImportError: failed to find libmagic.  Check your installation
RMTMACSIERRA01:infrastructure-cm lucas.rudd$ docker run --rm -it -e ES_HOST=search-pss-pf-prod-es-51-gcm4nbuhxjuo6telgbszkp3i7u.us-east-1.es.amazonaws.com -e AWS_PROFILE=asurion-soluto-nonprod.dev -v ~/.aws:/home/elastalert/.aws  -v ~/Desktop/rules/:/home/elastalert/rules --entrypoint elastalert asurionpss/elastalert:test --verbose --es_debug
Traceback (most recent call last):
  File "/home/elastalert/.local/bin/elastalert", line 11, in <module>
    load_entry_point('elastalert==0.1.36', 'console_scripts', 'elastalert')()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 480, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2691, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2322, in load
    return self.resolve()
  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2328, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/elastalert/.local/lib/python2.7/site-packages/elastalert/elastalert.py", line 21, in <module>
    from alerts import DebugAlerter
  File "/home/elastalert/.local/lib/python2.7/site-packages/elastalert/alerts.py", line 31, in <module>
    from thehive4py.api import TheHiveApi
  File "/home/elastalert/.local/lib/python2.7/site-packages/thehive4py/api.py", line 7, in <module>
    import magic
  File "/home/elastalert/.local/lib/python2.7/site-packages/magic.py", line 181, in <module>
    raise ImportError('failed to find libmagic.  Check your installation')
ImportError: failed to find libmagic.  Check your installation

After manually installing libmagic the issue persists.

Has anyone else experienced this issue?

About this issue

Original URL
State: closed
Created 6 years ago
Reactions: 30
Comments: 57 (1 by maintainers)

Commits related to this issue

Remedy bug https://github.com/Yelp/elastalert/issues/1927 — committed to engdan77/music_meta_manager by deleted user a year ago

Most upvoted comments

This worked for me:

pip uninstall python-magic pip install python-magic-bin==0.4.14

+732

synhershko on Sep 27, 2018

Solved it by running

brew install libmagic

+71

saginadir on Nov 3, 2019

This worked for me:

pip uninstall python-magic pip install python-magic-bin==0.4.14

+46

Samuel-Gong on Nov 2, 2018

@primeroz , Can you tell me that how do you solve this problem, just install libmagic when build docker image?

We’ve just added apk add libmagic.

+18

eliasbrange on Oct 23, 2018

Use the method by @synhershko i.e, 1. pip uninstall python-magic 2. pip install python-magic-bin Except, don’t include the python-magic-bin version!

+17

richardbalwane on Nov 26, 2019

My laptop is MacBook Pro M1, and I succeed with the following process:

brew install libmagic
pip install python-magic (since pip install python-magic-bin not working)

+16

kaiyikang on Mar 23, 2022

In my case (M1 MacBook), it was all about telling the system where to find the library files. So after obvious brew install libmagic (on M1s, homebrew installs libmagic to /opt/homebrew/Cellar/libmagic), it was necessary to create a symbolic link from the library files to the place where the system expects libmagic to be located, which is /usr/local/lib.

Therefore, this worked for me:

sudo ln -s /opt/homebrew/Cellar/libmagic/5.41/lib/libmagic.dylib /usr/local/lib/libmagic.dylib

FYI: my version of python-magic is 0.4.18 (and I wasn’t willing to downgrade as suggested above)

+15

jhavej on Feb 28, 2022

Solved it by running
brew install libmagic

In my case this is what helped (MacOS obviously…)

+14

vvvlad on Mar 4, 2020

Installation

The current stable version of python-magic is available on PyPI and can be installed by running pip install python-magic.

Other sources:

PyPI: http://pypi.python.org/pypi/python-magic/ GitHub: https://github.com/ahupp/python-magic This module is a simple wrapper around the libmagic C library, and that must be installed as well:

Debian/Ubuntu

$ sudo apt-get install libmagic1

Windows

You’ll need DLLs for libmagic. @julian-r has uploaded a version of this project that includes binaries to PyPI: https://pypi.python.org/pypi/python-magic-bin/0.4.14

Other sources of the libraries in the past have been File for Windows . You will need to copy the file magic out of [binary-zip]\share\misc, and pass its location to Magic(magic_file=...).

If you are using a 64-bit build of python, you’ll need 64-bit libmagic binaries which can be found here: https://github.com/pidydx/libmagicwin64. Newer version can be found here: https://github.com/nscaife/file-windows.

OSX

When using Homebrew: brew install libmagic When using macports: port install file

Ref:- https://pypi.org/project/python-magic/

+11

kushan-gunasekera on Sep 10, 2020

On my Macbook with M1 chip. This worked for me.

brew install libmagic
pip install python-magic-bin==0.4.14

Fuadeiza on Mar 23, 2022

I’m on MacOS m1 and I fix this issue by installing libmagic using Homebrew:

brew install libmagic

javadnikbakht on Nov 27, 2022

This worked for me: pip uninstall python-magic pip install python-magic-bin==0.4.14

it works thank you

efinal on Aug 20, 2019

pip install python-magic-bin==0.4.14
ERROR: Could not find a version that satisfies the requirement python-magic-bin==0.4.14 (from versions: none)
ERROR: No matching distribution found for python-magic-bin==0.4.14

Is not for Linux, the package is only for Widnows and MacOS https://pypi.org/project/python-magic-bin/0.4.14/#files

For me it works (22.04 Ubuntu):

pip uninstall python-magic
apt install libmagic1
pip install python-magic

ktaborowski on Aug 21, 2023

This worked for me:

pip uninstall python-magic pip install python-magic-bin==0.4.14

Viswahs on Aug 4, 2023

What is this for a crap? I get this error on Windows and Mac OS.

Lebostein on Oct 15, 2019

This worked for me:

pip uninstall python-magic pip install python-magic-bin==0.4.14

I love you.

Ogagnon on Jul 20, 2019

On my Macbook with M1 chip. This worked for me.

brew install libmagic

pip install python-magic-bin==0.4.14

I tried this by only doing brew install libmagic and having the python-magic already works

Thanks!!

gelodefaultbrain on Apr 4, 2023

This worked for me: pip uninstall python-magic pip install python-magic-bin==0.4.14

This works for my windows 11.

ZhengPeng7 on Mar 21, 2023

This worked for me:

pip uninstall python-magic pip install python-magic-bin==0.4.14

it really works!

ShuxunoO on Mar 1, 2023

When I run pip uninstall python-magic and then pip install python-magic-bin==0.4.14 I get the error message: ERROR: Could not find a version that satisfies the requirement python-magic-bin==0.4.14 (from versions: none) ERROR: No matching distribution found for python-magic-bin==0.4.14. Has anyone else seen this error?

cabeedelhertz on Mar 2, 2022

This worked for me:

pip uninstall python-magic pip install python-magic-bin==0.4.14

This worked for me too

gema-sanz on Jul 22, 2021

In my case (M1 MacBook), it was all about telling the system where to find the library files. So after obvious brew install libmagic (on M1s, homebrew installs libmagic to /opt/homebrew/Cellar/libmagic), it was necessary to create a symbolic link from the library files to the place where the system expects libmagic to be located, which is /usr/local/lib.

Therefore, this worked for me:

sudo ln -s /opt/homebrew/Cellar/libmagic/5.41/lib/libmagic.dylib /usr/local/lib/libmagic.dylib

FYI: my version of python-magic is 0.4.18 (and I wasn’t willing to downgrade as suggested above)

This worked for me (M2 pro). Just make sure you use the correct version inside the /opt/homebrew/Cellar/libmagic/.

Vishnu-Dhanabalan on Oct 27, 2023

pip uninstall python-magic
apt install libmagic1
pip install python-magic

Worked for me too.

danruggi on Sep 18, 2023

When I run pip uninstall python-magic and then pip install python-magic-bin==0.4.14 I get the error message: ERROR: Could not find a version that satisfies the requirement python-magic-bin==0.4.14 (from versions: none) ERROR: No matching distribution found for python-magic-bin==0.4.14. Has anyone else seen this error?

Happened to me, solved using python-magic-debian-bin==0.4.25

pgcd on Jul 7, 2022

We had this error on a popular Jupyter Notebook image, scipy-notebook. The fix ended up being to install libmagic with the Anaconda package manager:

conda install libmagic

Tracking down this error was a pain in the neck, so I hope this helps someone.

battlesysadmin on May 4, 2021

neither solution worked for me

roykingtree on Oct 9, 2019

I just installed libmagic and it fixed that error. I am running an alpine container

primeroz on Oct 1, 2018

@synhershko Your solution doesn’t work for me because python-magic requirement is >=0.4.15. Here is the error message:

pkg_resources.DistributionNotFound: The ‘python-magic>=0.4.15’ distribution was not found and is required by elastalert

lsoumille on Sep 27, 2018