qiskit-aer: ImportError for Qiskit Aer built from source for MacOS arm64

@PhilipHierhager

That could be a number of things, but in my experience that is the bug that shows up when a version of llvm-openmp less than 12.0.1 is being used (thus why step 5 given above is necessary). For versions of llvm-openmp older than this, for some reason Conan builds for x86 under the covers rather than arm64, which results in the issues you are experiencing when you try to import modules in Aer. Are you using source code obtained from the main GitHub branch or the .tar.gz file from pypi?

NOTE: It looks like a lot of the manual file changes given in the steps above have been merged to the main GitHub branch. It is possible that pip installing straight from the GitHub branch (so long as you have all the build dependencies installed) should work, but this is something I will have to test myself.

EDIT: I tried this today and it worked for me. Instead of doing all of the manual file changes given above, you can run pip install "git+https://github.com/Qiskit/qiskit-aer.git". This will clone the latest GitHub main branch and pip install it for you, avoiding the need for the manual file changes.

@HilbertSpecs

Hi! It’s not completely clear to me from your post which steps you are stuck on. I think throughout the course of this discussion we figured out different ways to get Aer to build correctly, which probably makes it less straightforward to follow. Furthermore the last few posts pertain to cross-compilation using CI wheel build, which is relevant to the Qiskit team who have to automate the wheel-build process to publish binaries on pypi, but is not relevant if you are trying to build for M1 on an M1 machine since that is not cross-compilation. Here I will try to compile the relevant parts of the discussion into a straightforward set of steps that one should be able to get Aer to build successfully if they follow them. I will also try to elaborate on certain steps a bit more. My goal is to try and be as specific as possible so people with a wide range of experience levels should not have much problem following it.

1. The first step is to to set up an M1-native python virtual environment. That is, an environment which will automatically install M1-native packages from pypi or condo-forge if they are available and attempt to build for M1 if they are not available. A python virtual environment which is not set up in this way will default to installing Intel x86 packages and build for Intel CPUs. There are multiple ways to do this, but in my opinion the most straightforward way to do this is to install the M1-native version of miniconda3 at: https://github.com/conda-forge/miniforge You will want to install the macosx-arm64 one. This is a file that will install miniconda3 for M1 when you run it in the terminal. NOTE: There is not an M1-native version of Anaconda available at this time. Most people use Anaconda to use python and conda, but Anaconda environments will default to x86. You can get it to work with some extra steps, but using miniconda is easier.

2. With your M1-native conda environment set up, you can now start to install the Qiskit dependencies and build Aer from source. Start by running terminal command:

conda create --name your-env-name python=3.9. (You can also now use python 3.8 as well if you prefer.) Activate the environment by running the command conda activate your-env-name.

1. Now you need a copy of the Aer source code. You can do this by either downloading a Github clone or downloading the qiskit-aer-0.9.1.tar.gz file at https://pypi.org/project/qiskit-aer/#files and unzipping it. Both of these download a folder that contains the Aer source code, so as long as you know where this folder has been downloaded on your machine, either will work.

2. Now you need to install the build dependencies, which you do by running the following terminal commands:

pip install cmake scikit-build cython
brew install libomp
brew install openblas
xcode-select --install
conda install numpy scipy

1. Navigate to the folder where the Aer source code is located. There are a few lines in a couple of files in this folder that need to be changed. The first change that needs to be made is in the file /cmake/conan_utils.cmake on line 16. The line:

    set(REQUIREMENTS ${REQUIREMENTS} llvm-openmp/8.0.1)
   

needs to be changed to:

    set(REQUIREMENTS ${REQUIREMENTS} llvm-openmp/12.0.1)

1. Now a change need to be made in the file setup.py. On line 107, the line:

   cmake_args = ["-DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=10.9"]
   

needs to be changed to:

 cmake_args = []

If I remember correctly (It’s been a while, so hopefully this is correct.) this should be all you need to do as far as editing source code files is concerned.

1. Now with the qiskit Aer source code folder as your working directory, you can run the terminal command python setup.py bdist_wheel . This will build the wheel file which will be located in the folder dist. You can navigate to this folder by running the command cd dist. You can check the name of the wheel file by running the command ls. It should be something like: qiskit_aer-0.9.1-cp39-cp39-macosx_11_0_arm64.whl. (It might not be this exactly, I’m not sure.) Whatever the name of this file is, run the command:

   pip install file-name.whl

and this should install Aer native for M1. You can verify that everything has been done correctly by running any Qiskit code which utilizes Aer by opening the Activity Monitor app that comes pre-installed on every Mac. It should say “Apple” under the “Kind” column next to the process that corresponds to the python program. If it says “Intel”, then something because that indicates that it’s running non-natively through Rosetta 2. Let me know if this works or if there are some steps which are still not quite clear.

Thank you for posting detailed instructions, that is super helpful for everyone currently stuck on an m1 mac right now. Unfortunately, I don’t think that will be trivial to automate in CI since it involves coordinating modifying conan’s source cache with an assembly file pulled from git, which will be tricky to get right (or building openmp from source prior to the wheel builds and disabling conan). Especially since the aer 0.9.0 is pending as soon as #1344 merges I don’t think publishing arm64 wheels is viable for this release. But when @vvilpas’s conan pr merges and conan is hosting a version of openmp that works on arm64 mac we can prepare a patch release (0.9.1) that will publish arm64 (or universal2) wheels.

I can confirm that the instructions from @JoelHBierman also worked for me. Thanks a lot!

Since this was a while ago that I built Aer, I built it again from source just now. Here are the steps to reproduce it as complete as it occurs to me to make them. If there’s any other info you need, let me know. This is starting from a fresh Miniconda3 environment.

1. Install the macosx-arm64 version of miniforge here: https://github.com/conda-forge/miniforge
2. Run terminal command conda create --name your-env-name python=3.9
3. download the source file qiskit-aer-0.8.2.tar.gz from pypi and unzip it to get the folder qiskit-aer-0.8.2
4. Follow the instructions for building Aer at https://qiskit.org/documentation/getting_started.html by running the following terminal commands:

pip install cmake scikit-build cython brew install libomp brew install openblas xcode-select --install (I already had this installed)

1. Run terminal command conda install numpy scipy to get the arm64 version of these packages from conda-forge. (Although I think this is not strictly speaking necessary as numpy already has published macosx-arm64 wheels and I recall being able to build scipy from source with a bit of effort previously a couple of months ago.) Note: Running this command seems to install llvm-openmp 12.0.0.

2. Run terminal command cd qiskit-aer-0.8.2 to get to the source code folder you downloaded from pypi.

Now at this point I tried just running the command python setup.py bdist_wheel --plat-name macosx-11.0-arm64 to see if this new version of llvm-openmp made a difference. I still got this cmake error:

In particular we still get the error message that @willemrc mentions above:

so to get around this error, we can replace the z_Linux_asm.S installed in the directory /Users/yourusername/.conan/data/llvm-openmp/8.0.1/_/_/source/source_subfolder/runtime/src/ on your local machine with the one that’s included in the GitHub branch that @willemrc mentions above. Note: To reproduce this build just now, I did not re-download this file from the Github branch of llvm-openmp. I had it already saved to my M1 MacBook (which is the original machine on which I did the build) and copied it to my M1 Mac mini (which is the machine I am currently reproducing the build on and did not previously have this file on it as far as I am aware, which is why I ran into the CMake error again.) For complete thoroughness, we would have to either re-do this build experiment with the z_Linux_asm.S file that’s on the current Github branch or otherwise verify that they are identical.

After making this file replacement, I ran the terminal command python setup.py bdist_wheel --plat-name macosx-11.0-arm64 again and the build completes and I get the file qiskit_aer-0.8.2-cp39-cp39-macosx_11_0_arm64.whl in the dist folder of qiskit-aer-0.8.2.

1. Run terminal commands cd dist and pip install qiskit_aer-0.8.2-cp39-cp39-macosx_11_0_arm64.whl. It then downloads the wheel and seems to also install qiskit-terra amongst other things like tweedledum without any additional effort on my part.

I ran the following snippet of code just so I could verify on Activity Monitor that it is indeed running natively:

import numpy as np
from qiskit import QuantumCircuit, transpile
from qiskit.providers.aer import QasmSimulator

backend = QasmSimulator()

for n in range(100000):
    circ = QuantumCircuit(3)
    circ.h(0)
    circ.cx(0, 1)
    circ.cx(0, 2)

    qc_compiled = transpile(circ, backend)

And indeed it does appear to be running natively without issue. (This is just some simple code copied from one of the Qiskit tutorials, looped over enough times so that the program runs long enough for me to verify on Activity Monitor that it’s running natively and not through Rosetta 2.)

The one and only issue really seems to be that z_Linux_asm.S file on llvm-openmp. As far as I can tell, currently that’s the only thing that has to be done manually to build Aer from source. If it wasn’t for that one issue, you could probably just run pip install qiskit-aer in a proper macosx-arm64 miniconda environment and it would build from source without issue.

@vvilpas I have the .whl file for macosx-arm64 binaries for qiskit-aer 0.8.2 saved to my computer. After a few months of using it regularly to run qiskit-aer natively for this platform, I have not encountered any problems with it. The speed-up of running Aer natively as opposed to running it through Rosetta 2 is quite significant and I imagine this would be valuable for others to be able to use. I noticed that this binary is not published on pypi as of right now. I am relatively new to this sort of thing, so my question to you and the IBM team is this: Is there a proper way for me (or others, since I am definitely not the only or the first person to build qiskit-aer for this platform) to contribute to Qiskit by providing this binary file so that others do not have to go through this same annoying process? If so, what is the way to go about doing so? I imagine that simply posting the .whl file here for people to install using pip is not the way to do it for multiple reasons. (i.e. installing a file some stranger posted on GitHub seems like a poor security practice and a limited number of people would see it here anyways.)