core: Illegal instruction trying to start hass
The problem
I am trying to upgrade to 2023.6.0 using the core installation. I have been running all previous installs via core with no issues.
I was already running 2023.5.4 on python3.11
What version of Home Assistant Core has the issue?
2023.6.0
What was the last working version of Home Assistant Core?
2023.5.4
What type of installation are you running?
Home Assistant Core
Integration causing the issue
No response
Link to integration documentation on our website
Diagnostics information
This is the contents of home-assistant.log.fault
Fatal Python error: Illegal instruction
Current thread 0x00007f664bfff700 (most recent call first): File “/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/util/json.py”, line 68 in load_json File “/usr/lib/python3.11/concurrent/futures/thread.py”, line 58 in run File “/usr/lib/python3.11/concurrent/futures/thread.py”, line 83 in _worker File “/usr/lib/python3.11/threading.py”, line 975 in run File “/usr/lib/python3.11/threading.py”, line 1038 in _bootstrap_inner File “/usr/lib/python3.11/threading.py”, line 995 in _bootstrap
Thread 0x00007f6650bb2700 (most recent call first): File “/usr/lib/python3.11/logging/handlers.py”, line 1528 in dequeue File “/usr/lib/python3.11/logging/handlers.py”, line 1579 in _monitor File “/usr/lib/python3.11/threading.py”, line 975 in run File “/usr/lib/python3.11/threading.py”, line 1038 in _bootstrap_inner File “/usr/lib/python3.11/threading.py”, line 995 in _bootstrap
Thread 0x00007f6654011740 (most recent call first): File “/usr/lib/python3.11/selectors.py”, line 468 in select File “/usr/lib/python3.11/asyncio/base_events.py”, line 1884 in _run_once File “/usr/lib/python3.11/asyncio/base_events.py”, line 607 in run_forever File “/usr/lib/python3.11/asyncio/base_events.py”, line 640 in run_until_complete File “/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/runner.py”, line 179 in run File “/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/main.py”, line 214 in main File “/srv/homeassistant/.virtualenv/bin/hass”, line 8 in <module>
Extension modules: _cffi_backend, multidict._multidict, yarl._quoting_c, ciso8601, aiohttp._helpers, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket, charset_normalizer.md, frozenlist._frozenlist, ulid_transform._ulid_impl, markupsafe._speedups, yaml._yaml (total: 13)
Example YAML snippet
No response
Anything in the logs that might be useful for us?
Fatal Python error: Illegal instruction
Current thread 0x00007f664bfff700 (most recent call first):
File "/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/util/json.py", line 68 in load_json
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58 in run
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 83 in _worker
File "/usr/lib/python3.11/threading.py", line 975 in run
File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap
Thread 0x00007f6650bb2700 (most recent call first):
File "/usr/lib/python3.11/logging/handlers.py", line 1528 in dequeue
File "/usr/lib/python3.11/logging/handlers.py", line 1579 in _monitor
File "/usr/lib/python3.11/threading.py", line 975 in run
File "/usr/lib/python3.11/threading.py", line 1038 in _bootstrap_inner
File "/usr/lib/python3.11/threading.py", line 995 in _bootstrap
Thread 0x00007f6654011740 (most recent call first):
File "/usr/lib/python3.11/selectors.py", line 468 in select
File "/usr/lib/python3.11/asyncio/base_events.py", line 1884 in _run_once
File "/usr/lib/python3.11/asyncio/base_events.py", line 607 in run_forever
File "/usr/lib/python3.11/asyncio/base_events.py", line 640 in run_until_complete
File "/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/runner.py", line 179 in run
File "/srv/homeassistant/.virtualenv/lib/python3.11/site-packages/homeassistant/__main__.py", line 214 in main
File "/srv/homeassistant/.virtualenv/bin/hass", line 8 in <module>
Extension modules: _cffi_backend, multidict._multidict, yarl._quoting_c, ciso8601, aiohttp._helpers, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket, charset_normalizer.md, frozenlist._frozenlist, ulid_transform._ulid_impl, markupsafe._speedups, yaml._yaml (total: 13)
Additional information
I can rollback to 2023.5.4 no issues
About this issue
- Original URL
- State: closed
- Created a year ago
- Reactions: 1
- Comments: 24 (3 by maintainers)
Same problem for me and upgrading manually to orjson 3.8.14 fix it !
python3 -m pip install orjson==3.8.14
I might have an answer on why we get this error.
In HA 2023.6.0 the orjson dependency is updated from 3.8.10 to 3.8.12 and it seems that orjson changed the wheel in 3.8.11 to require x86-64-v2. My (old) CPU only supports x86-64-v1 and other people also had that issue with orjson so they reverted that requirement in 3.8.14. See here for more info.
I was getting the “Illegal instruction” error too. I spent the last few days debugging and finally got it working. I’m guessing other people getting this error are finding this thread, so I’ll summarize what I learned and what worked.
Some context: I was updating the
homeassistant
python package from 2022.6.7 to 2023.1.7 (I wasn’t trying the latest HA version at this point, but later in this post I switch to 2023.6.3). I’m using an old RPi 1 B+. I was running Python 3.9.The issue: running the following code
Produces the error
Illegal instruction
I tried a bunch of things (which didn’t work) so I started digging into the source code to figure out where this error message is coming from. Eventually I determined the error is produced upon running this line (of python code):
which is from the
pyjwt
python package.Key takeaway so far: if you’re getting the “Illegal instruction” error, try running
python -c "import jwt"
. If there’s no output, then jwt is fine, but if it says “Illegal instruction” then you know that pyjwt is the source of the problem.I had version 2.5 of pyjwt. I tried installing a bunch of different (earlier and later) versions, but they all produced the same error. Then I tried a different python virtual environment and found that pyjwt worked there, so I decided to make a whole new virtual environment. Since I was running Python 3.9 before, I decided it was a good time to upgrade to 3.10. I used
https://www.python.org/ftp/python/3.10.12/Python-3.10.12.tgz
(I’ll skip installation instructions; you can look them up elsewhere). Then, to test: in the new virtual environment I ranpip install wheel pyjwt==2.5.0
andpython -c "import jwt"
and it worked.Then I tried
pip install homeassistant
(version2023.6.3
) and got an error aboutorjson
. It seems people in this thread also have issues with orjson, so I’ll report what worked for me. The error message was:Since maturin is a tool for building and publishing Rust-based Python packages, I wanted to try upgrading rust but I couldn’t figure out how to upgrade the existing one on my rpi. So instead I installed a new one. I don’t know much about rust, but apparently “rust” and “rustup” both install the rust compiler but rustup is more developer-oriented. Following standard installation procedure, I used
curl https://sh.rustup.rs -sSf | sh
(using my “homeassistant” OS user). It said “error: cannot install while Rust is installed” but offered to continue anyway so I said “yes” (I suppose I have 2 versions installed now, so I’ll have to be wary of that going forward). It installed itself to/home/homeassistant/.cargo/bin
.Finally, running
pip install orjson==3.8.12
was successful now. Likewise, I was able to dopip install homeassistant
and everything worked.Final takeaways:
Illegal instruction
error, try runningpython -c "import jwt"
. If there’s no output, then jwt is fine, but if it says “Illegal instruction” then you know that pyjwt is the source of the problem. My solution was a new virtual environment.maturin failed
error message (when installingorjson
), my solution was to installrustup
(i.e. get the latest version the rust compiler).Hope this helps someone!
It did it for me too thanks a lot doudz
pulseaudio --start ( need for audio running sox) source bin/activate python3 -m pip install orjson==3.8.14 ( needed otherwise HomeAssistant kept on replacing with orjson=3.8.12 ) hass
Not really surprising, but disappointing, that it will auto update after restart. Let’s hope that they can update the home assistant dependency of orjson to at least 3.8.14 or higher soon.
This helps, but after restarting hass, orjson is updated to version 3.8.12 again and hass starts again with an error.
I can confirm that if I upgrade Home Assistant Core to 2023.6.1 and I manually upgrade orjson to the latest version available, hass no longer crashes. However, it has numerous other problems. I should have manually updated orjson to 3.8.14 rather than latest to test this. I’ll try that at some point today…