llvmlite: Unable to install llvmlite in pip on Apple Silicon Mac (M1)

I was able to install llvmlite with pipenv on an m1 with the following steps:

• install cmake
• arch -arm64 brew install llvm@11
• LLVM_CONFIG="/opt/homebrew/Cellar/llvm@11/11.1.0_2/bin/llvm-config" arch -arm64 pipenv install llvmlite

It’s important to:

• specify version 11 of llvm (brew install llvm will get you version 12, and llvmlite specifically calls this out as a compatibility error).
• make sure you’re installing both with the same arch.
• double-check that your LLVM_CONFIG value is correct for your system (homebrew has recently changed where it puts things).

@thatandromeda 's workaround worked for me but I had to update the config path with /11.1.0_3/ and use my homebrew installed python instead:

LLVM_CONFIG="/opt/homebrew/Cellar/llvm@11/11.1.0_3/bin/llvm-config" arch -arm64 python3 -m pip install llvmlite

FWIW: work to officially start supporting the Numba stack on M1 has commenced now and I am actively working on this: https://github.com/numba/llvmlite/pull/813

@texadactyl thank you for commenting on this. I’d like to get this issue into shape, so there are three things I am thinking about:

• We need to distinguish between:
  • Making llvmlite work on M1 silicon in principle
  • Providing wheels for aarch64 platforms like Raspberry-Pi
  • Providing wheels that work on M1 silicon specifically.

This issue additionally mentions lack of 3.9 support. FWIW: the latest version 0.36.0 now does have 3.9 support!

In essence, I think we ought to clarify which bug/feature this ticket is asking for. How does that sound?

hi, you can use ‘conda install llvmlite’…

For me it worked simply with pip install llvmlite after making sure that /opt/homebrew/opt/llvm@11/bin is in PATH, but defining LLVM_CONFIG should also do the trick, like @thatandromeda mentioned. No other options are needed, as far as I can tell, it figures out the architecture correctly.

Are there any updates on this?

I’d opt to treat this issue as: “making llvmlite work on M1 silicon”

on Apple/M1:

~>brew install llvm@9
llvm@9: The x86_64 architecture is required for this software.
Error: An unsatisfied requirement failed this build.

On Mac M1, the following worked well for me.

# (note that llvmlite and numba can only be installed via conda-forge on mac M1)
conda install llvmlite
conda install numba

Note: I use conda-forge to manage conda environments.

FWIW: I will be looking into building binary wheels for the M1 platform .

llvmlite 0.38rc1 install went smoothly however as soon as I tried installing numba, pip tried to uninstall this version…

Installing collected packages: llvmlite, numba
  Attempting uninstall: llvmlite
    Found existing installation: llvmlite 0.38.0rc1
    Uninstalling llvmlite-0.38.0rc1:
      Successfully uninstalled llvmlite-0.38.0rc1
    Running setup.py install for llvmlite ... error
    ERROR: Command errored out with exit status 1:

Is there a way to force pip to keep this version?

I was looking to switch to python 3.10 but I will have to stay with 3.9 for now I guess…

I would recommend trying a Numba release candidate too, for example 0.55.0rc1 – it should work with the llvmlite RC.

xref: https://github.com/numba/numba/issues/6589#issuecomment-752226780 there’s packages available from conda-forge.

FYI, Requires-Python was not designed or intended to limit Python versions. It was added solely to allow old versions of Python to be nicely dropped. Numba was used as one of the examples of how this is currently not supported correctly when discussing the issue and three possible solutions (see https://discuss.python.org/t/requires-python-upper-limits/12663 ). TL;DR: solution 1 would make it a warning or error to set an upper limit (which is would require the least changes, since the current ecosystem doesn’t handle upper limits correctly), solution 2 would add an error if an upper cap was hit instead of “scrolling back” for older versions with looser/missing upper caps, and solution 3 would be to ignore upper caps. Core devs seemed to be liking solution 1, people writing packages seemed to like solution 2 (which, IMO, might be the worst solution, even though it’s giving people that they think they want). Also see https://iscinumpy.dev/post/bound-version-constraints/.

This is really bad for numba, since it gets an old version, which gets an old version of LLVMLite, which then breaks when compiling, instead of telling the user anything about Python being too new.

In the current system, the best solution is to remove the metadata cap (from both llvmlite and numba) - this will cause the setup.py to be run, and will produce the correct error message. You can also see the workaround SciPy is trying to implement, since they insist on incorrectly setting this metadata item.

This does also show an interesting feature for upper capping, though - in this case, solvers could solve forward for the RC version on Python 3.10, rather than looking back for an older uncapped version. (They don’t do this, and it would be complicated to add, but interesting thing not previously discussed).

* `LLVM_CONFIG="/opt/homebrew/Cellar/llvm@11/11.1.0_2/bin/llvm-config" arch -arm64 pipenv install llvmlite`

numba <0.54.0 requires llvmlite>0.36.0rc1,<0.37, and therefore manually installing llvmlite 0.37.0 means we can’t use numba<0.54.0, so in order to make numba work again, I had to upgrade it, since numba >=0.54 requires llvmlite<0.38,>=0.37.0rc1.

Numba 0.53.1 requires llvmlite 0.36 Numba 0.54.0 requires llvmlite 0.37

If you use Numba master you have a Numba 0.55.0dev0 and need llvmlite 0.38.0dev0.

(I hope that is correct, please do correct me if I made a mistake.)

You’re also absolutely right that llvmlite won’t build against LLVM 12:

RuntimeError: Building llvmlite requires LLVM 11.x.x, got '12.0.1'. 
Be sure to set LLVM_CONFIG to the right executable path.
Read the documentation at http://llvmlite.pydata.org/ for more information about building llvmlite.

Correct. LLVM 12 support for llvmlite will likely arrive in 2022 🤞

but hopefully, using LLVM 11 worked for me too.

Slowly but surely the Anaconda Distribution packages for the M1 silicon are coming out. Hopefully we can start shipping official support for that hardware soon so you won’t have to use hacks anymore to get it installed.

@hide-kawabata thank you for posting this, it looks useful. It may be worth trying to include this in the next llvmlite release. If you could, please submit a patch for this. You can use the conda-build system and simply add a patch for llvmdev here: https://github.com/numba/llvmlite/tree/master/conda-recipes Let me know if you need any further assistance with adding the patch.

@esc I just tried but I’m not sure if I did what I was expected. I wonder if you could check.

I’ve added this to my list and will revisit when I begin to tackle this issue, thank you!!

Also, it may be worth looking at the llvmdev from conda-forge which is here: https://github.com/conda-forge/llvmdev-feedstock

I’m afraid I’m not familiar with conda related things. I appreciate your help if I should do something.

No worries, I think that is sufficient for now and I’ll get back to you if anything is missing.

IIRC they do have Numba working on Apple Silicon.

I just tried conda for the first time and conda-forge of macOS ARM builds. https://conda-forge.org/blog/posts/2020-10-29-macos-arm64/ conda install llvmlite succeeded installing arm64 version without error…

Good, thank you for confirming this. Probably then using conda + the conda-forge channels is a good workaround for folks trying to use llvmlite on apple silicon right now.

@hide-kawabata thank you for posting this, it looks useful. It may be worth trying to include this in the next llvmlite release. If you could, please submit a patch for this. You can use the conda-build system and simply add a patch for llvmdev here:

https://github.com/numba/llvmlite/tree/master/conda-recipes

Let me know if you need any further assistance with adding the patch.

@esc I just tried but I’m not sure if I did what I was expected. I wonder if you could check.

Also, it may be worth looking at the llvmdev from conda-forge which is here:

https://github.com/conda-forge/llvmdev-feedstock

I’m afraid I’m not familiar with conda related things. I appreciate your help if I should do something.

IIRC they do have Numba working on Apple Silicon.

I just tried conda for the first time and conda-forge of macOS ARM builds. https://conda-forge.org/blog/posts/2020-10-29-macos-arm64/ conda install llvmlite succeeded installing arm64 version without error…