operating-system: Bluetooth failing after upgrade to HAOS 10 on Raspberry Pi

Describe the issue you are experiencing

I’m running a HA Yellow with a BT/WiFi CM4. After a non consistant uptime the internal Bluetooth adapter is stopping to work and the integrations fail. 6 devices have been integrated, but even after reducing to 3 devices the problem occurs. Unlike on HAOS 9.5, Bluetooth doesn’t seem to recover from intermittend issues.

What operating system image do you use?

yellow (Home Assistant Yellow)

What version of Home Assistant Operating System is installed?

10

Did you upgrade the Operating System.

Yes

Steps to reproduce the issue

  1. Upgrade to HAOS 10
  2. Reboot
  3. Wait for failing BT integrations

Anything in the Supervisor logs that might be useful for us?

No error messages in Supervisor logs

Anything in the Host logs that might be useful for us?

Apr 19 09:13:10 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:13:15 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:13:15 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:13:15 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:13:15 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:13:15 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:13:15 homeassistant kernel: Bluetooth: hci0: stop background scanning failed: -16
Apr 19 09:13:35 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:13:35 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:13:40 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:13:40 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:13:40 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:13:40 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:13:40 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:13:40 homeassistant kernel: Bluetooth: hci0: stop background scanning failed: -16
Apr 19 09:13:40 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:13:40 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:13:45 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:13:45 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:13:45 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:13:45 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:13:45 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:13:45 homeassistant kernel: Bluetooth: hci0: stop background scanning failed: -16
Apr 19 09:14:05 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:14:05 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:14:10 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:14:10 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:14:10 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:14:10 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:14:10 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:14:10 homeassistant kernel: Bluetooth: hci0: stop background scanning failed: -16
Apr 19 09:14:10 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:14:10 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:14:15 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:14:15 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:14:15 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:14:15 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:14:15 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:14:15 homeassistant kernel: Bluetooth: hci0: stop background scanning failed: -16
Apr 19 09:14:35 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:14:35 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:14:40 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:14:40 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:14:40 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:14:40 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:14:40 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:14:40 homeassistant kernel: Bluetooth: hci0: stop background scanning failed: -16
Apr 19 09:14:40 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:14:40 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:14:45 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:14:45 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:14:45 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:14:45 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:14:45 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:14:45 homeassistant kernel: Bluetooth: hci0: stop background scanning failed: -16
Apr 19 09:15:05 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:15:05 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:15:10 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:15:10 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:15:10 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:15:10 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:15:10 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:15:10 homeassistant kernel: Bluetooth: hci0: stop background scanning failed: -16
Apr 19 09:15:10 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:15:10 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:15:15 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:15:15 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:15:15 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:15:15 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:15:15 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:15:15 homeassistant kernel: Bluetooth: hci0: stop background scanning failed: -16
Apr 19 09:15:35 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:15:35 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:15:40 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:15:40 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:15:40 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:15:40 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:15:40 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:15:40 homeassistant kernel: Bluetooth: hci0: stop background scanning failed: -16
Apr 19 09:15:40 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:15:40 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:15:45 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:15:45 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:15:45 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:15:45 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -16
Apr 19 09:15:45 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -16
Apr 19 09:15:45 homeassistant kernel: Bluetooth: hci0: stop background scanning failed: -16

System information

System Information

version core-2023.4.5
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.10.10
os_name Linux
os_version 6.1.21-v8
arch aarch64
timezone Europe/Berlin
config_dir /config
Home Assistant Community Store
GitHub API ok
GitHub Content ok
GitHub Web ok
GitHub API Calls Remaining 5000
Installed Version 1.32.1
Stage running
Available Repositories 1346
Downloaded Repositories 64
HACS Data ok
Home Assistant Cloud
logged_in true
subscription_expiration 17. Mai 2023 um 02:00
relayer_connected true
relayer_region eu-central-1
remote_enabled true
remote_connected true
alexa_enabled true
google_enabled true
remote_server eu-central-1-4.ui.nabu.casa
can_reach_cert_server ok
can_reach_cloud_auth ok
can_reach_cloud ok
Home Assistant Supervisor
host_os Home Assistant OS 10.0
update_channel stable
supervisor_version supervisor-2023.04.0
agent_version 1.5.1
docker_version 23.0.3
disk_total 916.9 GB
disk_used 48.8 GB
healthy true
supported true
board yellow
supervisor_api ok
version_api ok
installed_addons SSH & Web Terminal (13.1.0), Studio Code Server (5.5.6), MariaDB (2.5.2), AirCast (3.5.2), RPC Shutdown (2.2), Samba share (10.0.0), AdGuard Home (4.8.6), AppDaemon (0.12.1), Network UPS Tools (0.11.2), phpMyAdmin (0.8.5), ESPHome (2023.3.2), Node-RED (14.1.4), Vaultwarden (Bitwarden) (0.19.3), Spotify Connect (0.12.3), Samba Backup (5.2.0), Matter Server (4.3.1), Mosquitto broker (6.2.0), MQTT Explorer (browser-1.0.1), VLC (0.1.3), NetDaemon V3.1 (.NET 7) (23.15.0), Nginx Proxy Manager (0.12.3), Z-Wave JS (0.1.77), OpenThread Border Router (1.2.0)
Dashboards
dashboards 5
resources 22
views 29
mode storage
Recorder
oldest_recorder_run 9. April 2023 um 09:07
current_recorder_run 19. April 2023 um 00:54
estimated_db_size 964.77 MiB
database_engine mysql
database_version 10.6.10
Spotify
api_endpoint_reachable ok

Additional information

Core Logs: `Logger: bluetooth_auto_recovery.recover Source: components/bluetooth/util.py:78 First occurred: 01:03:46 (2437 occurrences) Last logged: 11:13:15

Bluetooth adapter hci0 [E4:5F:01:6F:FF:84] could not be reset due to timeout`

`Logger: homeassistant.components.bluetooth.scanner Source: components/bluetooth/scanner.py:305 Integration: Bluetooth (documentation, issues) First occurred: 01:03:47 (1219 occurrences) Last logged: 11:13:15

hci0 (E4:5F:01:6F:FF:84): Failed to restart Bluetooth scanner: hci0 (E4:5F:01:6F:FF:84): Failed to start Bluetooth: [org.bluez.Error.InProgress] Operation already in progress Traceback (most recent call last): File “/usr/src/homeassistant/homeassistant/components/bluetooth/scanner.py”, line 223, in _async_start await self.scanner.start() # type: ignore[no-untyped-call] File “/usr/local/lib/python3.10/site-packages/bleak/init.py”, line 182, in start await self._backend.start() File “/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/scanner.py”, line 191, in start self._stop = await manager.active_scan( File “/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/manager.py”, line 368, in active_scan assert_reply(reply) File “/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/utils.py”, line 22, in assert_reply raise BleakDBusError(reply.error_name, reply.body) bleak.exc.BleakDBusError: [org.bluez.Error.InProgress] Operation already in progress

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File “/usr/src/homeassistant/homeassistant/components/bluetooth/scanner.py”, line 350, in _async_restart_scanner await self._async_start() File “/usr/src/homeassistant/homeassistant/components/bluetooth/scanner.py”, line 305, in _async_start raise ScannerStartError( homeassistant.components.bluetooth.scanner.ScannerStartError: hci0 (E4:5F:01:6F:FF:84): Failed to start Bluetooth: [org.bluez.Error.InProgress] Operation already in progress`

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Reactions: 15
  • Comments: 164 (35 by maintainers)

Commits related to this issue

Most upvoted comments

Is it the core you downgraded? Or how do you downgrade to 9.5?

With the command: ha os update --version 9.5

With the command: ha os update --version 9.5

Thank you! I’ve been struggling with this for days and have only just come across this issue.

To the HA team:

This is NOT fixed in HA OS 10.3 or 10.4 for me:

Screenshot 2023-08-16 at 20 05 03

Screenshot 2023-08-16 at 20 05 33

Screenshot 2023-08-16 at 20 13 57

I downgraded to 9.5… works 👍 back to 10.4, it breaks 👎

This whole issue is yet another massive HA goof!.. you should rename it Home ‘it breaks weekly’ Assistant.

The reason I run HA in Proxmox is because of the high likelihood of breakages from updates and changes. I can just snap it and revert. One such goof that springs to mind was that database upgrade failure that caused boot loops requiring manual file deletion and loss of history when you made the initial startup faster. NOT fun!

Perhaps the team need to take a step back from the new features and interface beautifying and look at what really matters: STABILITY.

It’s no damn fun when something that runs your entire house has constant issues after updates. April 19th this issue was opened and for me it’s still not fixed.

If HA wasn’t the most compatible solution (ie. device support) I’d have ditched it long ago for OpenHAB.

Sorry for the rant but this kind of stuff has to stop! I’ve recommended HA to a couple of people recently and they ditched it because of issues like this! Stop trying to add new features and release updates at the rate of Apple, and focus on making the damn thing not break!

Same here with HAOS 10 on RPi3. Bluetooth (used for Xiaomi BLE LYWSD03MMC temperature) works for a few hours, then fails and doesn’t come back. In 9.5 this sometimes also happened (once every few days), but after a few minutes it automatically came back. Now only a full system reboot helps

It means the adapter is locking up/crashing, but there is a regression in the reset code that tries to recovery/power cycle the adapter in 2023.9.1 that is fixed in 2023.9.2 via https://github.com/home-assistant/core/pull/99979

You can manually unplug/power cycle the adapter in the mean time or replace it with an adapter that doesn’t lock up.

Installed 10.3 two hours ago, Bluetooth did not break! (yet)

Those are good indications.

Can you turn back in 3 to 5 days and give a “long-term” report from your side? @thomasvochten @Rossaluss

I’m still wondering which change might have fixed this… nothing specific/related in 10.3 release notes afaics.

Will report back in two days 👌

[Update] After day 1: still working

@agners My appologies - I’ve deleted my comment. FWIW though, I do feel bobmarley2021’s pain. Issues with HA are not few and far between.

Those are good indications.

Can you turn back in 3 to 5 days and give a “long-term” report from your side? @thomasvochten @Rossaluss

I’m still wondering which change might have fixed this… nothing specific/related in 10.3 release notes afaics.

It’s not only with Raspberry Pi, I have INTEL NUC and have almost the same problems 😃 It’s another topic.

From what we understand currently, the issue is in the Raspberry Pi provided Linux kernel. The issue hasn’t been addressed so far, in fact there is still the same kernel in HAOS 10.2, so I don’t expect 10.2 to make a difference here, unfortunately 😢

I’m not seeing Bluetooth failure after upgrading to 2023.5.3

Home Assistant 2023.5.3
Supervisor 2023.04.1
Operating System 10.1
Frontend 20230503.3 - latest

In general, folks with root shell access on the OS (e.g. on the system console using login or via ssh port 22222), can you try this commands after Bluetooth failed and see if this recovers the problem in your case?

hciconfig hci0 down
hciconfig hci0 up

I experience the same issue on an Intel NUC with HAOS 10, worked fine before the upgrade to HAOS 10.

Same issue here after upgrading to 10.0 running on a Proxmox VM on a Intel Nuc from the 8th generation.

I am very sorry for the troubles you have.

Unfortunately, that regression is not a problem directly caused by changes on our side. Instead, this is a problem in the upstream Linux kernel. Between Linux 5.15 and 6.1 larger parts of the Bluetooth stack got reworked, and this seems to cause havoc especially on our use case.

Large part of the regression we’ve been able to solve by reverting changes made in the latest version of Linux (see this mailing list post and the many reports who say thigns work now for them, in this issue as well as in #2535). FWIW, I’ve tried to understand and fix the issues but the Bluetooth stack is not trivial, and the problem seems to be related to a race condition, which are especially hard to find and fix. Currently we do live with the work around, and it seems to have fixed many installations, including the unit I had here where I was able to reproduce the problem.

Unfortunately there seems more to it, and your installation has troubles still 😢

@bobmarley2021 BUT: It realllly doesn’t help to comment on a issue, which is not related to your board (this is about RPi), and where the symptoms look different! If you want us to have a real look into your particular issue, please open a new issue, along with all the relevant information (such as type of Bluetooth hardware).

Unfortunately, we work with very very diverse type of hardware. Heck we found bugs in x86-64 which even large distros like Ubuntu didn’t catch, just because HAOS users run on very old x86 machines at times. We simply can’t guarantee stability on every platform in the world. That is exactly why we have the option to downgrade the operating system, to bridge the time until the issue has been addressed.

If HA wasn’t the most compatible solution (ie. device support) I’d have ditched it long ago for OpenHAB.

I can pretty much guarantee you that if OpenHAB would have that Bluetooth support, you’d run into the same issue, since the issue is actually in the Linux kernel.

@ellnic please share such generic rants in the community forum or somewhere else, it is really not helpful in an issue tracker.

I believe this issue follows up on the problem for other board vendors and VMs https://github.com/home-assistant/operating-system/issues/2612. At least the symptoms looks the same.

@almirus the logs are from Home Assistant Core, please report the issue in the Core repository.

If the OS indeed has a problem with that particular dongle, please open a new issue since this issue is about the built-in Bluetooth support.

On my end too, the Raspberry Pi 3 where I’ve previously (see https://github.com/home-assistant/operating-system/issues/2485#issuecomment-1533031702) was able to reproduce didn’t show the problem in 24h+. There are still Bluetooth: hci0: Frame reassembly failed (-84) errors (and the overrun counter on the tty counts up), but it seems with the fix added with PR #2598 indeed addresses the Raspberry Pi issue as well.

I consider this issue closed. If you have Bluetooth issues still, please open a new issue.

@bcutter Actually there was this change, as discussed few posts above:

Add patch to fix Bluetooth LE advertisement stall (https://github.com/home-assistant/operating-system/issues/2535) (https://github.com/home-assistant/operating-system/pull/2598)

Since many of the BT devices are reporting sensor data through BLE advertisements, there is significant chance most of the issues could be resolved with this patch.

@bdraco Indeed - but it seems there are more issues mixed in this ticket. The issues with passive BLE sensors could be resolved by the fix added to 10.3, it’s not Intel specific and it manifested without any errors in the kernel log. But the unavailability of the Bluetooth integration wasn’t something we saw on Intel, but maybe it could be a side-effect of the bluetooth_auto_recovery triggered originally by the BLE scan bug? I guess we will see once 10.3 rolls out, let’s give it a chance.

@chertvl @onlize @DemianTinkiel do you see Bluetooth: hci0: Frame reassembly failed (-84) errors?

If not, that is the issue tracking the x86-64 issue: https://github.com/home-assistant/operating-system/issues/2535

I’m on rpi4 4gb with ssd boot using the onboard bluetooth, and passive ble monitor integration for a single xiaomi ble temperature sensor.

After months of stable operation and no changes, I was then getting freezing and lockups at least daily which seemed to coincide with the upgrade past HA OS 10 Since then I have completely changed hardware, different rpi, different ssd and adapter, different official power supply I am currently on 2023.5.3 and 10.1 Downgrading to 9.5 on my previous hardware seemed to be more stable but still with crahes, and there was differences in CPU usage so I think I’m dealing with more than 1 issue anyway.

I got about 24 hours out of it then my log started getting spammed with homeassistant kernel: Bluetooth: hci0: Frame reassembly failed (-84) This goes on several times a second over a few minutes. By the time I notice and SSH in I can see CPU usage using TOP is high (but no PID listed shows high cpu), the supervisor is unresponsive and the terminal is very very slow.

This is in my HA Host logs:

May 28 07:40:39 homeassistant kernel: Bluetooth: hci0: Frame reassembly failed (-84)
May 28 07:40:39 homeassistant kernel: Bluetooth: hci0: Frame reassembly failed (-84)
May 28 07:40:39 homeassistant NetworkManager[447]: <info>  [1685259639.2429] device (wlan0): supplicant interface state: disconnected -> scanning
May 28 07:40:39 homeassistant NetworkManager[447]: <info>  [1685259639.2476] device (p2p-dev-wlan0): supplicant management interface state: disconnected -> scanning
May 28 07:40:39 homeassistant kernel: Bluetooth: hci0: Frame reassembly failed (-84)
May 28 07:40:39 homeassistant kernel: ------------[ cut here ]------------
May 28 07:40:39 homeassistant kernel: Firmware transaction timeout
May 28 07:40:39 homeassistant kernel: WARNING: CPU: 0 PID: 249859 at drivers/firmware/raspberrypi.c:67 rpi_firmware_property_list+0x1e4/0x250
May 28 07:40:39 homeassistant kernel: Modules linked in: rfcomm nft_chain_nat nf_conntrack_netlink xfrm_user xfrm_algo nft_compat nf_tables nfnetlink cmac algif_hash aes_arm64 aes_generic algif_skcipher af_alg bnep hci_uart btbcm bluetooth ecdh_generic ecc libaes sch_fq_codel vc4 snd_soc_hdmi_codec drm_display_helper cec drm_dma_helper brcmfmac bcm2835_v4l2(C) drm_kms_helper bcm2835_codec(C) bcm2835_isp(C) rpivid_hevc(C) bcm2835_mmal_vchiq(C) brcmutil v4l2_mem2mem videobuf2_vmalloc videobuf2_dma_contig snd_soc_core v3d snd_compress videobuf2_memops cfg80211 snd_pcm_dmaengine snd_bcm2835(C) gpu_sched drm_shmem_helper rfkill snd_pcm videobuf2_v4l2 videobuf2_common raspberrypi_hwmon vc_sm_cma(C) snd_timer syscopyarea videodev snd sysfillrect cp210x sysimgblt fb_sys_fops nvmem_rmem uio_pdrv_genirq uio drm fuse drm_panel_orientation_quirks backlight
May 28 07:40:39 homeassistant kernel: CPU: 0 PID: 249859 Comm: kworker/0:3 Tainted: G         C         6.1.21-v8 #1
May 28 07:40:39 homeassistant kernel: Hardware name: Raspberry Pi 4 Model B Rev 1.2 (DT)
May 28 07:40:39 homeassistant kernel: Workqueue: events dbs_work_handler
May 28 07:40:39 homeassistant kernel: pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
May 28 07:40:39 homeassistant kernel: pc : rpi_firmware_property_list+0x1e4/0x250
May 28 07:40:39 homeassistant kernel: lr : rpi_firmware_property_list+0x1e4/0x250
May 28 07:40:39 homeassistant kernel: sp : ffffffc00bafb970
May 28 07:40:39 homeassistant kernel: x29: ffffffc00bafb970 x28: ffffff80409d4dc0 x27: ffffffdec6863960
May 28 07:40:39 homeassistant kernel: x26: ffffffdec6863880 x25: ffffffc00846e008 x24: ffffff804936a480
May 28 07:40:39 homeassistant kernel: x23: 0000000000001000 x22: 00000000ffffff92 x21: ffffff80409d4d80
May 28 07:40:39 homeassistant kernel: x20: 0000000000000018 x19: ffffffc00846e000 x18: 0000000000000000
May 28 07:40:39 homeassistant kernel: x17: 0000000000000000 x16: 0000000000000015 x15: ffffffdec5ecbb30
May 28 07:40:39 homeassistant kernel: x14: ffffffdec6742e60 x13: 74756f656d697420 x12: 6e6f69746361736e
May 28 07:40:39 homeassistant kernel: x11: ffffffdec66ea238 x10: ffffffdec66ea248 x9 : ffffffdec52f8f34
May 28 07:40:39 homeassistant kernel: x8 : 00000000ffffefff x7 : ffffffdec6742248 x6 : 00000000000022f8
May 28 07:40:39 homeassistant kernel: x5 : 000000000000bff4 x4 : 0000000000000000 x3 : 0000000000000027
May 28 07:40:39 homeassistant kernel: x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffffff8000101ec0
May 28 07:40:39 homeassistant kernel: Call trace:
May 28 07:40:39 homeassistant kernel:  rpi_firmware_property_list+0x1e4/0x250
May 28 07:40:39 homeassistant kernel:  rpi_firmware_property+0x78/0x120
May 28 07:40:39 homeassistant kernel:  raspberrypi_fw_set_rate+0x5c/0xe0
May 28 07:40:39 homeassistant kernel:  clk_change_rate+0xe0/0x500
May 28 07:40:39 homeassistant kernel:  clk_core_set_rate_nolock+0x158/0x270
May 28 07:40:39 homeassistant kernel:  clk_set_rate+0x40/0x190
May 28 07:40:39 homeassistant kernel:  _opp_config_clk_single+0x38/0xa0
May 28 07:40:39 homeassistant kernel:  _set_opp+0xec/0x3b0
May 28 07:40:39 homeassistant kernel:  dev_pm_opp_set_rate+0x118/0x280
May 28 07:40:39 homeassistant kernel:  set_target+0x38/0x50
May 28 07:40:39 homeassistant kernel:  __cpufreq_driver_target+0x31c/0x6b0
May 28 07:40:39 homeassistant kernel:  od_dbs_update+0xc4/0x1c0
May 28 07:40:39 homeassistant kernel:  dbs_work_handler+0x48/0x90
May 28 07:40:39 homeassistant kernel:  process_one_work+0x1dc/0x450
May 28 07:40:39 homeassistant kernel:  worker_thread+0x154/0x450
May 28 07:40:39 homeassistant kernel:  kthread+0x104/0x110
May 28 07:40:39 homeassistant kernel:  ret_from_fork+0x10/0x20
May 28 07:40:39 homeassistant kernel: ---[ end trace 0000000000000000 ]---
May 28 07:40:39 homeassistant kernel: raspberrypi-clk soc:firmware:clocks: Failed to change fw-clk-arm frequency: -110
May 28 07:40:40 homeassistant kernel: Bluetooth: hci0: Frame reassembly failed (-84)
May 28 07:40:40 homeassistant kernel: Bluetooth: hci0: Frame reassembly failed (-84)
May 28 07:40:40 homeassistant kernel: Bluetooth: hci0: unexpected subevent 0x05 length: 75 > 12
May 28 07:40:41 homeassistant kernel: Bluetooth: hci0: Frame reassembly failed (-84)
May 28 07:40:41 homeassistant kernel: Bluetooth: hci0: Frame reassembly failed (-84)
May 28 07:40:41 homeassistant kernel: Bluetooth: hci0: Frame reassembly failed (-84)
May 28 07:40:42 homeassistant kernel: Bluetooth: hci0: Frame reassembly failed (-84)
May 28 07:40:42 homeassistant kernel: Bluetooth: hci0: Frame reassembly failed (-84)
May 28 07:40:42 homeassistant kernel: Bluetooth: hci0: Frame reassembly failed (-84)
May 28 07:40:42 homeassistant kernel: raspberrypi-exp-gpio soc:firmware:gpio: Failed to set GPIO 6 state (-110 86)
May 28 07:40:43 homeassistant kernel: Bluetooth: hci0: Frame reassembly failed (-84)
May 28 07:40:43 homeassistant kernel: Bluetooth: hci0: Frame reassembly failed (-84)

`

May 28 07:42:44 homeassistant kernel: Bluetooth: hci0: unknown advertising packet type: 0x1f
May 28 07:42:44 homeassistant kernel: Bluetooth: hci0: Frame reassembly failed (-84)
May 28 07:42:44 homeassistant kernel: Bluetooth: hci0: Frame reassembly failed (-84)
May 28 07:42:44 homeassistant kernel: Bluetooth: hci0: unexpected subevent 0x01 length: 75 > 18

Eventually it stops here

May 28 07:50:31 homeassistant kernel: Bluetooth: hci0: Opcode 0x200c failed: -110
May 28 07:50:31 homeassistant kernel: Bluetooth: hci0: Unable to disable scanning: -110
May 28 07:50:31 homeassistant kernel: Bluetooth: hci0: command 0x200c tx timeout
May 28 07:46:24 homeassistant NetworkManager[447]: <info>  [1685259984.2467] manager: (veth11a807d): new Veth device (/org/freedesktop/NetworkManager/Devices/44)
May 28 07:47:20 homeassistant NetworkManager[447]: <info>  [1685260040.2107] manager: (vethbd1c07d): new Veth device (/org/freedesktop/NetworkManager/Devices/45)
May 28 07:47:59 homeassistant NetworkManager[447]: <info>  [1685260079.5805] manager: (veth041f83c): new Veth device (/org/freedesktop/NetworkManager/Devices/46)
May 28 07:50:38 homeassistant kernel: INFO: task kworker/0:3:249859 blocked for more than 483 seconds.
May 28 07:50:38 homeassistant kernel:       Tainted: G        WC         6.1.21-v8 #1
May 28 07:50:38 homeassistant kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
May 28 07:50:38 homeassistant kernel: task:kworker/0:3     state:D stack:0     pid:249859 ppid:2      flags:0x00000208
May 28 07:50:38 homeassistant kernel: Workqueue: events dbs_work_handler
May 28 07:50:38 homeassistant kernel: Call trace:
May 28 07:50:38 homeassistant kernel:  __switch_to+0xf4/0x170
May 28 07:50:38 homeassistant kernel:  __schedule+0x2d8/0x9d0
May 28 07:50:38 homeassistant kernel:  schedule+0x60/0x100
May 28 07:50:39 homeassistant kernel:  schedule_timeout+0xa0/0x1c0
May 28 07:50:39 homeassistant kernel:  wait_for_completion_timeout+0x7c/0x150
May 28 07:50:39 homeassistant kernel:  mbox_send_message+0xa8/0x130
May 28 07:50:39 homeassistant kernel:  rpi_firmware_property_list+0xf0/0x250
May 28 07:50:39 homeassistant kernel:  rpi_firmware_property+0x78/0x120
May 28 07:50:40 homeassistant kernel:  raspberrypi_fw_get_rate+0x4c/0x80
May 28 07:50:40 homeassistant kernel:  clk_recalc+0x40/0xb0
May 28 07:50:40 homeassistant kernel:  clk_change_rate+0xf4/0x500
May 28 07:50:40 homeassistant kernel:  clk_core_set_rate_nolock+0x158/0x270
May 28 07:50:41 homeassistant kernel:  clk_set_rate+0x40/0x190
May 28 07:50:41 homeassistant kernel:  _opp_config_clk_single+0x38/0xa0
May 28 07:50:41 homeassistant kernel:  _set_opp+0xec/0x3b0
May 28 07:50:41 homeassistant kernel:  dev_pm_opp_set_rate+0x118/0x280
May 28 07:50:41 homeassistant kernel:  set_target+0x38/0x50
May 28 07:50:42 homeassistant kernel:  __cpufreq_driver_target+0x31c/0x6b0
May 28 07:50:42 homeassistant kernel:  od_dbs_update+0xc4/0x1c0
May 28 07:50:42 homeassistant kernel:  dbs_work_handler+0x48/0x90
May 28 07:50:42 homeassistant kernel:  process_one_work+0x1dc/0x450
May 28 07:50:42 homeassistant kernel:  worker_thread+0x154/0x450
May 28 07:50:42 homeassistant kernel:  kthread+0x104/0x110
May 28 07:50:42 homeassistant kernel:  ret_from_fork+0x10/0x20

Terminal Saved Output bluetooth hci copy.txt

I have the same issue with HA yellow, CM4 wifi + bt 8gb. After updating to haos 10, my Bluetooth devices became unstable. But installing BLE monitor custom integration as @Bobbik1 mentioned fixes the issues and my devices are now responsive.

2023.5.3 has a workaround to bounce the bluetooth adapter when it gets stuck.

While it won’t solve the underlying issues here, it should be able to auto recover from a few more stuck states.

Baseboard: Home Assistant Yellow PoE (version with JP2 instead of SW3) CM4: CM4108032 (8GB RAM, 32GB eMMC, WiFi/BT)

config.txt changes (additional to the stock settings): dtparam=eth_led0=4 dtparam=eth_led1=4 dtparam=ant2

Problem also occurred without the dtparam=ant2 change for the external antenna.

Same, Raspberry Pi 4 CM + Home assistant Yellow: Screenshot_20230429_140312_Home Assistant Screenshot_20230429_175256_Home Assistant

{
  "home_assistant": {
    "installation_type": "Home Assistant OS",
    "version": "2023.4.6",
    "dev": false,
    "hassio": true,
    "virtualenv": false,
    "python_version": "3.10.10",
    "docker": true,
    "arch": "aarch64",
    "timezone": "America/New_York",
    "os_name": "Linux",
    "os_version": "6.1.21-v8",
    "supervisor": "2023.04.1",
    "host_os": "Home Assistant OS 10.1",
    "docker_version": "23.0.3",
    "chassis": "embedded",
    "run_as_root": true
  },
  "custom_components": {},
  "integration_manifest": {
    "domain": "bluetooth",
    "name": "Bluetooth",
    "after_dependencies": [
      "hassio"
    ],
    "codeowners": [
      "@bdraco"
    ],
    "config_flow": true,
    "dependencies": [
      "usb"
    ],
    "documentation": "https://www.home-assistant.io/integrations/bluetooth",
    "iot_class": "local_push",
    "loggers": [
      "btsocket",
      "bleak_retry_connector",
      "bluetooth_adapters",
      "bluetooth_auto_recovery"
    ],
    "quality_scale": "internal",
    "requirements": [
      "bleak==0.20.1",
      "bleak-retry-connector==3.0.2",
      "bluetooth-adapters==0.15.3",
      "bluetooth-auto-recovery==1.0.3",
      "bluetooth-data-tools==0.3.1",
      "dbus-fast==1.84.2"
    ],
    "is_built_in": true
  },
  "data": {
    "manager": {
      "adapters": {
        "hci0": {
          "address": "E4:5F:01:DF:00:00",
          "sw_version": "",
          "hw_version": "usb:v1D6Bp0246d0542",
          "passive_scan": true,
          "manufacturer": "Raspberry Pi Trading Ltd",
          "product": null,
          "vendor_id": null,
          "product_id": null
        }
      },
      "slot_manager": {
        "manager": true,
        "adapter_slots": {
          "hci0": 5
        },
        "allocations_by_adapter": {
          "hci0": []
        }
      },
      "scanners": [
        {
          "name": "hci0 (E4:5F:01:DF:00:00)",
          "start_time": 95.064099602,
          "source": "E4:5F:01:DF:00:00",
          "scanning": false,
          "type": "HaScanner",
          "last_detection": 76578.779684603,
          "monotonic_time": 142357.682742187,
          "discovered_devices_and_advertisement_data": [],
          "adapter": "hci0"
        }
      ],
      "connectable_history": [],
      "all_history": [],
      "advertisement_tracker": {
        "intervals": {},
        "sources": {},
        "timings": {}
      }
    },
    "adapters": {
      "hci0": {
        "address": "E4:5F:01:DF:00:00",
        "sw_version": "",
        "hw_version": "usb:v1D6Bp0246d0542",
        "passive_scan": true,
        "manufacturer": "Raspberry Pi Trading Ltd",
        "product": null,
        "vendor_id": null,
        "product_id": null
      }
    },
    "dbus": {
      "/org/bluez": {
        "org.freedesktop.DBus.Introspectable": {},
        "org.bluez.AgentManager1": {},
        "org.bluez.ProfileManager1": {}
      },
      "/org/bluez/hci0": {
        "org.freedesktop.DBus.Introspectable": {},
        "org.bluez.Adapter1": {
          "Address": "E4:5F:01:DF:00:00",
          "AddressType": "public",
          "Name": "",
          "Alias": "",
          "Class": 2883584,
          "Powered": true,
          "PowerState": "on",
          "Discoverable": false,
          "DiscoverableTimeout": 180,
          "Pairable": false,
          "PairableTimeout": 0,
          "Discovering": false,
          "UUIDs": [
            "0000110e-0000-1000-8000-00805f9b34fb",
            "00001200-0000-1000-8000-00805f9b34fb",
            "0000110a-0000-1000-8000-00805f9b34fb",
            "0000110b-0000-1000-8000-00805f9b34fb",
            "00001108-0000-1000-8000-00805f9b34fb",
            "0000110c-0000-1000-8000-00805f9b34fb",
            "00001800-0000-1000-8000-00805f9b34fb",
            "00001801-0000-1000-8000-00805f9b34fb",
            "00001844-0000-1000-8000-00805f9b34fb",
            "0000180a-0000-1000-8000-00805f9b34fb",
            "00001112-0000-1000-8000-00805f9b34fb"
          ],
          "Modalias": "usb:v1D6Bp0246d0542",
          "Roles": [
            "central",
            "peripheral"
          ]
        },
        "org.freedesktop.DBus.Properties": {},
        "org.bluez.BatteryProviderManager1": {},
        "org.bluez.GattManager1": {},
        "org.bluez.AdvertisementMonitorManager1": {
          "SupportedMonitorTypes": [
            "or_patterns"
          ],
          "SupportedFeatures": []
        },
        "org.bluez.Media1": {
          "SupportedUUIDs": [
            "0000110a-0000-1000-8000-00805f9b34fb",
            "0000110b-0000-1000-8000-00805f9b34fb"
          ]
        },
        "org.bluez.NetworkServer1": {},
        "org.bluez.LEAdvertisingManager1": {
          "ActiveInstances": 0,
          "SupportedInstances": 5,
          "SupportedIncludes": [
            "tx-power",
            "appearance",
            "local-name"
          ],
          "SupportedFeatures": [],
          "SupportedCapabilities": {
            "MaxAdvLen": 31,
            "MaxScnRspLen": 31
          }
        }
      }
    }
  }
}

Reboot the device(complete hard reset instead of restart home assistant system) temporarily restore Bluetooth functionality then crash after a day or two.

Hi Greg, where did you get version 10.1? Still working OK for you? Thanks.

The latest stable Linux kernel 6.1.25 has some changes which might address this problem: https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-6.1.25

In particular:

  • Bluetooth: Fix printing errors if LE Connection times out
  • Bluetooth: hci_conn: Fix not cleaning up on LE Connection failure

So far I was not able to reproduce the failures described here, hence I cannot test the new kernel myself. The last nightly development build of Home Assistant OS comes with the 6.1.25. Can someone who can easily reproduce the problem give this a try maybe? Ideally you should take a full backup and download it, just to be safe 😄

Updating to the latest development builds would involve:

ha supervisor options --channel=dev
ha supervisor reload
ha supervisor update
ha os info

At this point there should be a OS update offered (11.0.dev20230420). You can install it from the CLI as well

ha os update

After the update, switch back to the stable or beta channel:

ha supervisor options --channel=stable
ha supervisor reload

Downgrading back to HAOS 10 (or 9.5) is fairly safe and can be done using:

ha os update --version=9.5

Issues here as well. Running HA in a VM in Proxmox. I have 15 Xiaomi sensors and two Inkbird. A restart of HA bring some back but looses others. I have not managed to get all back. Reverting to OS v9.5 fixes the problems.

Same issue here, running on Intel NUC 7th gen with Proxmox (latest version).

Based on the above feedback, it looks like something is very seriously broken with Bluetooth drivers in HAOS 10.

Same issue, Odroid N2+, my switchbots are mostly not responding.