core: ESPHome UnhandledAPIConnectionError after 2023.10.4 on 32-bit systems using noise encryption

The problem

ESPHome device unavailable after updating today.

Looks like the error is Associated data too long. Max 2**32 bytes (UnhandledAPIConnectionError)

Errors in the logs:

Logger: aioesphomeapi.reconnect_logic
Source: /usr/local/lib/python3.11/site-packages/aioesphomeapi/reconnect_logic.py:146
First occurred: 17:33:50 (6 occurrences)
Last logged: 17:34:55

Can't connect to ESPHome API for airgradient-pro @ 192.168.18.7: Unexpected error while connecting to airgradient-pro @ 192.168.18.7: Associated data too long. Max 2**32 bytes (UnhandledAPIConnectionError)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aioesphomeapi/client.py", line 374, in finish_connection
    await self._connection.finish_connection(login=login)
  File "aioesphomeapi/connection.py", line 564, in finish_connection
  File "aioesphomeapi/connection.py", line 551, in aioesphomeapi.connection.APIConnection.finish_connection
  File "aioesphomeapi/connection.py", line 552, in aioesphomeapi.connection.APIConnection.finish_connection
  File "aioesphomeapi/connection.py", line 526, in _do_finish_connect
  File "aioesphomeapi/connection.py", line 366, in _connect_init_frame_helper
  File "aioesphomeapi/_frame_helper/noise.py", line 138, in perform_handshake
  File "aioesphomeapi/_frame_helper/noise.py", line 179, in aioesphomeapi._frame_helper.noise.APINoiseFrameHelper._send_hello_handshake
  File "/usr/local/lib/python3.11/site-packages/noise/connection.py", line 110, in write_message
    result = self.noise_protocol.handshake_state.write_message(payload, buffer)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/noise/state.py", line 345, in write_message
    message_buffer += self.symmetric_state.encrypt_and_hash(payload)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/noise/state.py", line 172, in encrypt_and_hash
    ciphertext = self.cipher_state.encrypt_with_ad(self.h, plaintext)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/noise/state.py", line 55, in encrypt_with_ad
    ciphertext = self.cipher.encrypt(self.k, self.n, ad, plaintext)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/noise/backends/default/ciphers.py", line 10, in encrypt
    return self.cipher.encrypt(nonce=self.format_nonce(n), data=plaintext, associated_data=ad)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "src/chacha20poly1305_reuseable/__init__.py", line 102, in chacha20poly1305_reuseable.ChaCha20Poly1305Reusable.encrypt
  File "src/chacha20poly1305_reuseable/__init__.py", line 128, in chacha20poly1305_reuseable.ChaCha20Poly1305Reusable.encrypt
OverflowError: Associated data too long. Max 2**32 bytes

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aioesphomeapi/reconnect_logic.py", line 173, in _try_connect
    await self._cli.finish_connection(login=True)
  File "/usr/local/lib/python3.11/site-packages/aioesphomeapi/client.py", line 380, in finish_connection
    raise UnhandledAPIConnectionError(
aioesphomeapi.core.UnhandledAPIConnectionError: Unexpected error while connecting to airgradient-pro @ 192.168.18.7: Associated data too long. Max 2**32 bytes

What version of Home Assistant Core has the issue?

core-2023.10.4

What was the last working version of Home Assistant Core?

core-2023.10.3

What type of installation are you running?

Home Assistant Core

Integration causing the issue

ESPHome

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

About this issue

Original URL
State: closed
Created 8 months ago
Comments: 29 (18 by maintainers)

Commits related to this issue

fix: remove cython type for MAX_SIZE While it works fine and tests fine, This seems to have a comparision problem on some platforms: https://github.com/home-assistant/core/issues/102474 https://githu... — committed to bdraco/chacha20poly1305-reuseable by bdraco 8 months ago

Most upvoted comments

The wheels are up and can be installed using the community ssh addon with protected mode disable by running:

docker exec -it homeassistant /bin/bash
pip3 install --upgrade chacha20poly1305-reuseable==0.10.2 --find-links=https://wheels.home-assistant.io/musllinux/

Than restart

bdraco on Oct 21, 2023

Ok so likely a 32bit problem

bdraco on Oct 21, 2023

Either way the new improved error reporting in aioesphomeapi works great as it really helped to identify the issue quickly

bdraco on Oct 21, 2023

I saw it before on the test PI I just built, but gone now as well. 🤞

TBH surprised that could fix it

Pleasantly surprised. I have no more issues to report

cvandesande on Oct 21, 2023

BTW, I have another weird issue. My browser keeps loosing it’s socket connection. Also since this latest update. Do you think this is related or should I open a separate issue?

In the logs I have

Logger: homeassistant.components.websocket_api.http.connection
Source: components/hassio/handler.py:561
Integration: Home Assistant WebSocket API ([documentation](https://www.home-assistant.io/integrations/websocket_api), [issues]([https://github.com/home-assistant/core/issues?q=is%3Aissue+is%3Aopen+label%3A%22integration%3A+websocket_api%22](https://github.com/home-assistant/core/issues?q=is%3Aissue+is%3Aopen+label%3A%22integration%3A+websocket_api%22+sort%3Aupdated-desc)))
First occurred: 18:15:53 (37 occurrences)
Last logged: 18:35:16

    [2451647272] Error handling message: Unknown error (unknown_error) Chris from 192.168.16.1 (Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/118.0)
    [2417752296] Error handling message: Unknown error (unknown_error) Chris from 192.168.16.1 (Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/118.0)
    [2448037736] Error handling message: Unknown error (unknown_error) Chris from 192.168.16.1 (Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/118.0)
    [2417620648] Error handling message: Unknown error (unknown_error) Chris from 192.168.16.1 (Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/118.0)
    [2447942248] Error handling message: Unknown error (unknown_error) Chris from 192.168.16.1 (Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/118.0)

RuntimeError: File descriptor 59 is used by transport <_SelectorSocketTransport fd=59 read=polling write=<idle, bufsize=0>>

I’d open a new issue for that as its doesn’t seem related

bdraco on Oct 21, 2023

New build to follow since the first one got canceled https://github.com/home-assistant/core/actions/runs/6598698115

bdraco on Oct 21, 2023

HA wheels are building here https://github.com/home-assistant/core/actions/runs/6598661986

I’ll give you a command to update once its done.

In the mean time, please install the community ssh addon, as you’ll need to be able to access the container without protection mode turned on to update the library

bdraco on Oct 21, 2023

https://github.com/bdraco/chacha20poly1305-reuseable/actions/runs/6598620732/job/17926787396 is building with a prospective fix.
I’ll get some wheels up for HA shortly so you can try installing them in your container

bdraco on Oct 21, 2023