core: Unable to upgrade grpcio to newer versions due to armv7 link error

The problem

grpcio is currently pinned to an older version because of a link error that appears on armv7. As a result, many other libraries that depend on grpcio are pinned to older versions, and we are starting to see that maintaining all the necessary pins is getting very difficult.

This was originally pinned in Sept 2020 in #40678 to help with #40148 and #40266

The pin was last reverted in #50605 but had to be added back in #51289 as reports like #51167 appeared again. Related #51198 and #51286

As the conflicting pins in downstream libraries come up, it shows up like #51290 with fixes like #55115 and #54239. Now also seeing #55687 and #55883 which probably can be fixed with more pins, but it seems better just to focus on the root issue and get to a newer grpcio.

This issue tracks getting things into a state where grpcio can be upgraded to a newer version since I don’t think we’re tracking it anywhere else.

What is version of Home Assistant Core has the issue?

core-2021.9.7

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Container

Integration causing the issue

No response

Link to integration documentation on our website

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

This can be reproduced in a development environment like this:

example.py

#!/usr/bin/python

from google_nest_sdm.google_nest import main

def main():
  print("Hello world")

if __name__ == "__main__":
  main()

requirements.txt

grpcio==1.38.0  # No links for 1.40.0
google-nest-sdm

Dockerfile

FROM ghcr.io/home-assistant/armv7-base-python:3.8-alpine3.12

ENV WHEELS_LINKS=https://wheels.home-assistant.io/alpine-3.12/armv7/

WORKDIR /code

COPY requirements.txt .
COPY example.py .

RUN python3 --version

RUN \
    pip3 install --no-cache-dir --no-index --only-binary=:all: --find-links ${WHEELS_LINKS} \
        -r requirements.txt

CMD python3 ./example.py

$ uname -a
Linux raspberrypi 5.10.17-v7l+ #1414 SMP Fri Apr 30 13:20:47 BST 2021 armv7l GNU/Linux

$ cat /proc/cpuinfo
processor	: 0
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 108.00
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd08
CPU revision	: 3
...
Hardware	: BCM2711
Revision	: b03111
Serial		: 10000000c466b355
Model		: Raspberry Pi 4 Model B Rev 1.1

$ docker build --tag example-wheel-image .
...

$ docker run example-wheel-image:latest
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
Traceback (most recent call last):
  File "./example.py", line 3, in <module>
    from google_nest_sdm.google_nest import main
  File "/usr/local/lib/python3.8/site-packages/google_nest_sdm/google_nest.py", line 36, in <module>
    from .google_nest_subscriber import GoogleNestSubscriber
  File "/usr/local/lib/python3.8/site-packages/google_nest_sdm/google_nest_subscriber.py", line 14, in <module>
    from google.cloud import pubsub_v1
  File "/usr/local/lib/python3.8/site-packages/google/cloud/pubsub_v1/__init__.py", line 17, in <module>
    from google.cloud.pubsub_v1 import types
  File "/usr/local/lib/python3.8/site-packages/google/cloud/pubsub_v1/types.py", line 36, in <module>
    from google.pubsub_v1.types import pubsub as pubsub_gapic_types
  File "/usr/local/lib/python3.8/site-packages/google/pubsub_v1/__init__.py", line 18, in <module>
    from .services.publisher import PublisherClient
  File "/usr/local/lib/python3.8/site-packages/google/pubsub_v1/services/publisher/__init__.py", line 18, in <module>
    from .client import PublisherClient
  File "/usr/local/lib/python3.8/site-packages/google/pubsub_v1/services/publisher/client.py", line 27, in <module>
    from google.api_core import gapic_v1  # type: ignore
  File "/usr/local/lib/python3.8/site-packages/google/api_core/gapic_v1/__init__.py", line 18, in <module>
    from google.api_core.gapic_v1 import config
  File "/usr/local/lib/python3.8/site-packages/google/api_core/gapic_v1/config.py", line 23, in <module>
    import grpc
  File "/usr/local/lib/python3.8/site-packages/grpc/__init__.py", line 23, in <module>
    from grpc._cython import cygrpc as _cygrpc
ImportError: Error loading shared library ld-linux-armhf.so.3: No such file or directory (needed by /usr/local/lib/python3.8/site-packages/grpc/_cython/cygrpc.cpython-38-arm-linux-gnueabihf.so)
[cmd] /bin/ash exited 1
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.

Here is an example image that works:

Dockerfile:

FROM arm32v7/python:3.8-buster

WORKDIR /code

WORKDIR /code

COPY requirements.txt .
COPY example.py .

RUN python3 --version

RUN \
    pip3 install --no-cache-dir -r requirements.txt

CMD python3 ./example.py

$ docker build --tag example-image .
$ docker run example-image:latest
Hello world