core: Illegal instruction trying to start hass

The problem

I am trying to upgrade to 2023.6.0 using the core installation. I have been running all previous installs via core with no issues.

I was already running 2023.5.4 on python3.11

What version of Home Assistant Core has the issue?

2023.6.0

What was the last working version of Home Assistant Core?

2023.5.4

What type of installation are you running?

Home Assistant Core

Integration causing the issue

No response

Link to integration documentation on our website

No response

Diagnostics information

This is the contents of home-assistant.log.fault

Fatal Python error: Illegal instruction

Current thread 0x00007f664bfff700 (most recent call first): File “/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/util/json.py”, line 68 in load_json File “/usr/lib/python3.11/concurrent/futures/thread.py”, line 58 in run File “/usr/lib/python3.11/concurrent/futures/thread.py”, line 83 in _worker File “/usr/lib/python3.11/threading.py”, line 975 in run File “/usr/lib/python3.11/threading.py”, line 1038 in _bootstrap_inner File “/usr/lib/python3.11/threading.py”, line 995 in _bootstrap

Thread 0x00007f6650bb2700 (most recent call first): File “/usr/lib/python3.11/logging/handlers.py”, line 1528 in dequeue File “/usr/lib/python3.11/logging/handlers.py”, line 1579 in _monitor File “/usr/lib/python3.11/threading.py”, line 975 in run File “/usr/lib/python3.11/threading.py”, line 1038 in _bootstrap_inner File “/usr/lib/python3.11/threading.py”, line 995 in _bootstrap

Thread 0x00007f6654011740 (most recent call first): File “/usr/lib/python3.11/selectors.py”, line 468 in select File “/usr/lib/python3.11/asyncio/base_events.py”, line 1884 in _run_once File “/usr/lib/python3.11/asyncio/base_events.py”, line 607 in run_forever File “/usr/lib/python3.11/asyncio/base_events.py”, line 640 in run_until_complete File “/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/runner.py”, line 179 in run File “/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/main.py”, line 214 in main File “/srv/homeassistant/.virtualenv/bin/hass”, line 8 in <module>

Extension modules: _cffi_backend, multidict._multidict, yarl._quoting_c, ciso8601, aiohttp._helpers, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket, charset_normalizer.md, frozenlist._frozenlist, ulid_transform._ulid_impl, markupsafe._speedups, yaml._yaml (total: 13)

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Fatal Python error: Illegal instruction

Current thread 0x00007f664bfff700 (most recent call first):
  File "/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/util/json.py", line 68 in load_json
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58 in run
  File "/usr/lib/python3.11/concurrent/futures/thread.py", line 83 in _worker
  File "/usr/lib/python3.11/threading.py", line 975 in run
  File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f6650bb2700 (most recent call first):
  File "/usr/lib/python3.11/logging/handlers.py", line 1528 in dequeue
  File "/usr/lib/python3.11/logging/handlers.py", line 1579 in _monitor
  File "/usr/lib/python3.11/threading.py", line 975 in run
  File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
  File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap

Thread 0x00007f6654011740 (most recent call first):
  File "/usr/lib/python3.11/selectors.py", line 468 in select
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1884 in _run_once
  File "/usr/lib/python3.11/asyncio/base_events.py", line 607 in run_forever
  File "/usr/lib/python3.11/asyncio/base_events.py", line 640 in run_until_complete
  File "/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/runner.py", line 179 in run
  File "/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/__main__.py", line 214 in main
  File "/srv/homeassistant/.virtualenv/bin/hass", line 8 in <module>

Extension modules: _cffi_backend, multidict._multidict, yarl._quoting_c, ciso8601, aiohttp._helpers, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket, charset_normalizer.md, frozenlist._frozenlist, ulid_transform._ulid_impl, markupsafe._speedups, yaml._yaml (total: 13)

Additional information

I can rollback to 2023.5.4 no issues

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Reactions: 1
  • Comments: 24 (3 by maintainers)

Most upvoted comments

Same problem for me and upgrading manually to orjson 3.8.14 fix it ! python3 -m pip install orjson==3.8.14

I might have an answer on why we get this error.

In HA 2023.6.0 the orjson dependency is updated from 3.8.10 to 3.8.12 and it seems that orjson changed the wheel in 3.8.11 to require x86-64-v2. My (old) CPU only supports x86-64-v1 and other people also had that issue with orjson so they reverted that requirement in 3.8.14. See here for more info.

I was getting the “Illegal instruction” error too. I spent the last few days debugging and finally got it working. I’m guessing other people getting this error are finding this thread, so I’ll summarize what I learned and what worked.

Some context: I was updating the homeassistant python package from 2022.6.7 to 2023.1.7 (I wasn’t trying the latest HA version at this point, but later in this post I switch to 2023.6.3). I’m using an old RPi 1 B+. I was running Python 3.9.

The issue: running the following code

source /srv/homeassistant/bin/activate
hass

Produces the error Illegal instruction

I tried a bunch of things (which didn’t work) so I started digging into the source code to figure out where this error message is coming from. Eventually I determined the error is produced upon running this line (of python code):

import jwt

which is from the pyjwt python package.

Key takeaway so far: if you’re getting the “Illegal instruction” error, try running python -c "import jwt". If there’s no output, then jwt is fine, but if it says “Illegal instruction” then you know that pyjwt is the source of the problem.

I had version 2.5 of pyjwt. I tried installing a bunch of different (earlier and later) versions, but they all produced the same error. Then I tried a different python virtual environment and found that pyjwt worked there, so I decided to make a whole new virtual environment. Since I was running Python 3.9 before, I decided it was a good time to upgrade to 3.10. I used https://www.python.org/ftp/python/3.10.12/Python-3.10.12.tgz (I’ll skip installation instructions; you can look them up elsewhere). Then, to test: in the new virtual environment I ran pip install wheel pyjwt==2.5.0 and python -c "import jwt" and it worked.

Then I tried pip install homeassistant (version 2023.6.3) and got an error about orjson. It seems people in this thread also have issues with orjson, so I’ll report what worked for me. The error message was:

Collecting orjson==3.8.12 Downloading orjson-3.8.12.tar.gz (669 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 669.2/669.2 kB 551.8 kB/s eta 0:00:00 Installing build dependencies: started Installing build dependencies: still running… Installing build dependencies: finished with status ‘done’ Getting requirements to build wheel: started Getting requirements to build wheel: finished with status ‘done’ Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status ‘error’ error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [13 lines of output] error: failed to parse manifest at /tmp/pip-install-b00ndvyy/orjson_7591cd510ded4c3aa8758373a4b30229/Cargo.toml

Caused by: failed to parse the edition key

Caused by: supported edition values are 2015 or 2018, but 2021 is unknown 💥 maturin failed Caused by: Cargo metadata failed. Does your crate compile with cargo build? Caused by: cargo metadata exited with an error: Checking for Rust toolchain… Running maturin pep517 write-dist-info --metadata-directory /tmp/pip-modern-metadata-yjauqncr --interpreter /srv/testenv/bin/python3.10

Since maturin is a tool for building and publishing Rust-based Python packages, I wanted to try upgrading rust but I couldn’t figure out how to upgrade the existing one on my rpi. So instead I installed a new one. I don’t know much about rust, but apparently “rust” and “rustup” both install the rust compiler but rustup is more developer-oriented. Following standard installation procedure, I used curl https://sh.rustup.rs -sSf | sh (using my “homeassistant” OS user). It said “error: cannot install while Rust is installed” but offered to continue anyway so I said “yes” (I suppose I have 2 versions installed now, so I’ll have to be wary of that going forward). It installed itself to /home/homeassistant/.cargo/bin.

Finally, running pip install orjson==3.8.12 was successful now. Likewise, I was able to do pip install homeassistant and everything worked.

Final takeaways:

  • if you get the Illegal instruction error, try running python -c "import jwt". If there’s no output, then jwt is fine, but if it says “Illegal instruction” then you know that pyjwt is the source of the problem. My solution was a new virtual environment.
  • if you get the maturin failed error message (when installing orjson), my solution was to install rustup (i.e. get the latest version the rust compiler).

Hope this helps someone!

It did it for me too thanks a lot doudz

python3 -m pip install orjson==3.8.14 have to leave the line in , in start script as follow

pulseaudio --start ( need for audio running sox) source bin/activate python3 -m pip install orjson==3.8.14 ( needed otherwise HomeAssistant kept on replacing with orjson=3.8.12 ) hass

Not really surprising, but disappointing, that it will auto update after restart. Let’s hope that they can update the home assistant dependency of orjson to at least 3.8.14 or higher soon.

I can confirm that if I upgrade Home Assistant Core to 2023.6.1 and I manually upgrade orjson to the latest version available, hass no longer crashes. However, it has numerous other problems. I should have manually updated orjson to 3.8.14 rather than latest to test this. I’ll try that at some point today…

This helps, but after restarting hass, orjson is updated to version 3.8.12 again and hass starts again with an error.

I can confirm that if I upgrade Home Assistant Core to 2023.6.1 and I manually upgrade orjson to the latest version available, hass no longer crashes. However, it has numerous other problems. I should have manually updated orjson to 3.8.14 rather than latest to test this. I’ll try that at some point today…