grpc: "pip install grpcio" failure on MacOS 12 M1 and Python 3.10

Alright, this is what worked for at least two machines. Hopefully its just modifying the versions for you.

pyenv uninstall 3.8.13
pyenv install 3.8.13
pyenv local 3.8.13
pip install virtualenv
virtualenv test
source test/bin/activate
pip install grpcio

For whatever reason, that works for us. Remember to recreate the virutalenvs for your own project.

I was able to resolve this issue by adding these two environment variables to my shell before installing:

export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1

I believe the issues being reported in this thread are caused by toolchain issues. The ultimate fix is going to be publishing native M1 artifacts so that you don’t have to have a developer toolchain installed. In the meantime, if you are still having issues building from source, I would recommend using Rosetta in x86 compatibility mode.

I suspect that the users reporting that changing Python versions helped switched from a version for which they did not have Python headers installed to one where they did, enabling compilation. Most of the comments I see above don’t give me the full compiler logs, which reveal the actual cause of the failure. I’m going to close this particular issue out since there isn’t anything actionable left to deal with. If there are further build issues, please open a new issue and include full logs from the failure so that we can help debug.

closing a widely reported and experienced issue that has no resolution, while providing no help or information for users, “besides your doing it wrong”, is kinda of problematic, there are dozens of other issues in this repo wrt m1 python issues, some of which have helpful information re workarounds for the lack of actual support in released artifacts. ie. at the very least, link to the issue for m1 binary artifacts or a user workaround. given how many years it took to produce linux aarch64 artifacts, no one is holding their breath on binary artifacts.

grpcio 1.50 M1 mac python3.9 CFLAGS="-I/Library/Developer/CommandLineTools/usr/include/c++/v1 -I/opt/homebrew/opt/openssl/include" LDFLAGS="-L/opt/homebrew/opt/openssl/lib" pip3 install grpcio It works

<< Redacting the following: not working>> So, what i’ve discovered is that:

• pip install grpcio

Fails as documented above. But:

• conda install -c apple grpcio
• conda install -c apple tensorflow-deps
• pip install tensorflow-macos
• pip install tensorflow-serving-api

Will complete with only minor errors. << End redaction>>

Nevermind. It completed processing, but fails to import in Python. And if I go to an Ubuntu 22 instance, then pip install grpcio works on the first try. Sigh.

upgrading python version worked for me. Tested 3.10.6 and 3.9.14 and both worked fine.

I’m on Python 3.10.6 but did not work for me.

p.s. I’m using macOS Ventura 13.0 & M2 chip.

The latest version that I could install properly was 1.46.3 on Python 3.10.6 and Mac M1. The problem is the package that I need to install depends on the 1.48.1, it’s dbt-bigquery adapter. Any news on this?

The latest version that I could install properly was 1.46.3 on Python 3.10.6 and Mac M1. The problem is the package that I need to install depends on the 1.48.1, it’s dbt-bigquery adapter. Any news on this?

The following command well works in Python 3.9.15 and Mac M1 pip install --ignore-installed --no-binary grpcio grpcio

I have a few people with the same unqualified-id errors. Python 3.8.13. Still trying to find a fix for them.

@gnossen , No not at all… just updated my python. I noticed that with python 3.10.5, i can install grpcio 1.46.3 without problems. So I tried updating with the latest

Here is what I could learn during my try-and-error steps to find a working concept:

• Use the zlib-devel package provided by distribution
• Use a more recent C-Compiler (I began with 4.2 (which was the installed version), updated to 4.8, updated to 5.5, updated to 8.3), which I installed to /usr/local (without any prefix)
• Use a more recent BinUtils package (installed 2.20, updated to 2.29), which I installed in custom directory /opt/binutils2291
• Use at least OpenSSL 1.0.2 (configure -fPIC -shared), which I installed in custom directory /opt/openssl102

Then I played with the environment variables and found, these are finally build and install the package without an error:

PKG_CONFIG_PATH=/opt/openssl102/lib/pkgconfig:$PKG_CONFIG_PATH GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 CFLAGS="-I/opt/binutils2291/include -I/opt/openssl102/include -L/opt/openssl102/lib" LDFLAGS="-L/opt/openssl102/lib" pip install grpcio

I’m not sure why it is so difficult to install grpcio on an M1 Mac. Every time I open a new repo I have go through this again, and again and again. Sometimes, after a couple hours of trying, I get it work, but sometimes it doesnt. Like now. My primary interest is to install tensorflow-serving-api (2.11.0), which requires tensorflow, which tried to install grpcio and failed. I’ve tried several versions of each.

So, I’m unable to install grpcio on a MacBook Pro M1/Max with MacOS 12.2. I’m using Python V3.10.9 And I added these exports but it made no difference:

• export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1
• export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1

The error message is long enough that it filled the terminal buffer, so I attached it as a text file: grpcio-dump.txt

The last error trace is:

      Traceback (most recent call last):
        File "/private/var/folders/9m/ms8r01p55fd8cldfdwjbz6c40000gn/T/pip-install-tnssv36d/grpcio_79c2ad0e67da41d7ad661e1114c85053/src/python/grpcio/commands.py", line 280, in build_extensions
          build_ext.build_ext.build_extensions(self)
        File "/Users/kevin/tfserving_ai2/venv/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
          self._build_extensions_serial()
        File "/Users/kevin/tfserving_ai2/venv/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
          self.build_extension(ext)
        File "/Users/kevin/tfserving_ai2/venv/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
          _build_ext.build_extension(self, ext)
        File "/Users/kevin/tfserving_ai2/venv/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 547, in build_extension
          objects = self.compiler.compile(
        File "/private/var/folders/9m/ms8r01p55fd8cldfdwjbz6c40000gn/T/pip-install-tnssv36d/grpcio_79c2ad0e67da41d7ad661e1114c85053/src/python/grpcio/_parallel_compile_patch.py", line 58, in _parallel_compile
          multiprocessing.pool.ThreadPool(BUILD_EXT_COMPILER_JOBS).map(
        File "/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/pool.py", line 367, in map
          return self._map_async(func, iterable, mapstar, chunksize).get()
        File "/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/pool.py", line 774, in get
          raise self._value
        File "/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/pool.py", line 125, in worker
          result = (True, func(*args, **kwds))
        File "/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/pool.py", line 48, in mapstar
          return list(map(*args))
        File "/private/var/folders/9m/ms8r01p55fd8cldfdwjbz6c40000gn/T/pip-install-tnssv36d/grpcio_79c2ad0e67da41d7ad661e1114c85053/src/python/grpcio/_parallel_compile_patch.py", line 54, in _compile_single_file
          self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
        File "/private/var/folders/9m/ms8r01p55fd8cldfdwjbz6c40000gn/T/pip-install-tnssv36d/grpcio_79c2ad0e67da41d7ad661e1114c85053/src/python/grpcio/commands.py", line 263, in new_compile
          return old_compile(obj, src, ext, cc_args, extra_postargs,
        File "/Users/kevin/tfserving_ai2/venv/lib/python3.10/site-packages/setuptools/_distutils/unixccompiler.py", line 188, in _compile
          raise CompileError(msg)
      distutils.errors.CompileError: command '/usr/bin/clang' failed with exit code 1
      

For me a simple:

pyenv uninstall 3.8.10
pyenv install 3.8.10

did the trick.

I’m on macOS Monterey 12.6 (21G115), MBP M1 Max, clang version 14.0.0 (clang-1400.0.29.102), installing grpcio version 1.49.1.

Also having this issue, wondering if there is a quick fix. Running Python 3.9.7 and MacOS 12.5.1, M1 Pro

@gnossen , Just an update. I fixed this by updating my python to 3.10.6