operating-system: Host (Raspberry Pi) crashes and goes offline

Hardware Environment

• Raspberry Pi [1/2/3/4] - Pi 4 4GB RAM
• ODROID [C2/C4/N2(+)/XU4]
• ASUS Tinker [S]
• Generic x86-64 (like Intel NUC)
• OVA (Open Virtualization Appliance, on Intel NUC or any other hardware, please add the Hypervisor you are using)

Home Assistant OS release:

• Fresh installation of release x.y
• Updated from version 2021.6 (installed on 29th June 2021)
• Additional information (if accessible):

System Health

version	core-2021.7.3
installation_type	Home Assistant OS
dev	false
hassio	true
docker	true
virtualenv	false
python_version	3.9.5
os_name	Linux
os_version	5.10.17-v8
arch	aarch64
timezone	Europe/Rome

Home Assistant Community Store

GitHub API	ok
Github API Calls Remaining	4751
Installed Version	1.13.2
Stage	running
Available Repositories	847
Installed Repositories	13

Home Assistant Cloud

logged_in	false
can_reach_cert_server	ok
can_reach_cloud_auth	ok
can_reach_cloud	ok

Home Assistant Supervisor

host_os	Home Assistant OS 6.1
update_channel	stable
supervisor_version	supervisor-2021.06.8
docker_version	20.10.6
disk_total	109.3 GB
disk_used	6.1 GB
healthy	true
supported	true
board	rpi4-64
supervisor_api	ok
version_api	ok
installed_addons	Home Assistant Google Drive Backup (0.104.3), deCONZ (6.9.0), SSH & Web Terminal (9.0.0), Visual Studio Code (3.6.0), File editor (5.3.2), Check Home Assistant configuration (3.8.0), Let’s Encrypt (4.11.0), NGINX Home Assistant SSL proxy (3.0.1)

Lovelace

dashboards	1
resources	6
views	2
mode	storage

Spotify

api_endpoint_reachable	ok

Supervisor logs: N/A

Journal logs: These are the logs at the moment of the crash which I retrieved with journalctl -b <id_previous_boot> Just a couple of minutes before to the end of the file.

Jul 16 03:34:23 homeassistant b27c96514c39[466]: 05:34:22:981 dev /dev/ttyS0
Jul 16 03:34:23 homeassistant b27c96514c39[466]: 05:34:22:982 GW firmware version: 0x26690700
Jul 16 03:34:32 homeassistant b27c96514c39[466]: 05:34:32:980 dev /dev/ttyS0
Jul 16 03:34:32 homeassistant b27c96514c39[466]: 05:34:32:981 GW firmware version: 0x26690700
Jul 16 03:34:42 homeassistant b27c96514c39[466]: 05:34:42:979 dev /dev/ttyS0
Jul 16 03:34:42 homeassistant b27c96514c39[466]: 05:34:42:980 GW firmware version: 0x26690700
Jul 16 03:34:43 homeassistant 446376b1bc31[466]: 21-07-16 05:34:43 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.RUNNING
Jul 16 03:34:43 homeassistant hassos-supervisor[962]: 21-07-16 05:34:43 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.RUNNING
Jul 16 03:34:43 homeassistant hassos-supervisor[962]: 21-07-16 05:34:43 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.FREE_SPACE/ContextType.SYSTEM
Jul 16 03:34:43 homeassistant hassos-supervisor[962]: 21-07-16 05:34:43 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.SECURITY/ContextType.CORE
Jul 16 03:34:43 homeassistant 446376b1bc31[466]: 21-07-16 05:34:43 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.FREE_SPACE/ContextType.SYSTEM
Jul 16 03:34:43 homeassistant 446376b1bc31[466]: 21-07-16 05:34:43 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.SECURITY/ContextType.CORE
Jul 16 03:34:43 homeassistant 446376b1bc31[466]: 21-07-16 05:34:43 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.PWNED/ContextType.ADDON
Jul 16 03:34:43 homeassistant hassos-supervisor[962]: 21-07-16 05:34:43 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.PWNED/ContextType.ADDON
Jul 16 03:34:43 homeassistant 446376b1bc31[466]: 21-07-16 05:34:43 INFO (MainThread) [supervisor.resolution.check] System checks complete
Jul 16 03:34:43 homeassistant hassos-supervisor[962]: 21-07-16 05:34:43 INFO (MainThread) [supervisor.resolution.check] System checks complete
Jul 16 03:34:43 homeassistant hassos-supervisor[962]: 21-07-16 05:34:43 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.RUNNING
Jul 16 03:34:43 homeassistant 446376b1bc31[466]: 21-07-16 05:34:43 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.RUNNING
Jul 16 03:34:44 homeassistant 446376b1bc31[466]: 21-07-16 05:34:44 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
Jul 16 03:34:44 homeassistant 446376b1bc31[466]: 21-07-16 05:34:44 INFO (MainThread) [supervisor.resolution.fixup] Starting system autofix at state CoreState.RUNNING
Jul 16 03:34:44 homeassistant hassos-supervisor[962]: 21-07-16 05:34:44 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
Jul 16 03:34:44 homeassistant hassos-supervisor[962]: 21-07-16 05:34:44 INFO (MainThread) [supervisor.resolution.fixup] Starting system autofix at state CoreState.RUNNING
Jul 16 03:34:44 homeassistant hassos-supervisor[962]: 21-07-16 05:34:44 INFO (MainThread) [supervisor.resolution.fixup] System autofix complete
Jul 16 03:34:44 homeassistant 446376b1bc31[466]: 21-07-16 05:34:44 INFO (MainThread) [supervisor.resolution.fixup] System autofix complete
Jul 16 03:34:52 homeassistant b27c96514c39[466]: 05:34:52:984 dev /dev/ttyS0
Jul 16 03:34:52 homeassistant b27c96514c39[466]: 05:34:52:985 GW firmware version: 0x26690700
Jul 16 03:35:02 homeassistant b27c96514c39[466]: 05:35:02:986 dev /dev/ttyS0
Jul 16 03:35:02 homeassistant b27c96514c39[466]: 05:35:02:987 GW firmware version: 0x26690700
Jul 16 03:36:17 homeassistant NetworkManager[357]: <warn>  [1626406577.9421] sup-iface[0x221eb110,wlan0]: could not get scan request result: Timeout was reached
Jul 16 03:36:54 homeassistant systemd[1]: systemd-logind.service: Watchdog timeout (limit 3min)!
Jul 16 03:35:16 homeassistant audit[373]: ANOM_ABEND auid=4294967295 uid=0 gid=0 ses=4294967295 subj==unconfined pid=373 comm="systemd-logind" exe="/usr/lib/systemd/systemd-logind  " sig=6 res=1
Jul 16 03:36:54 homeassistant systemd[1]: systemd-logind.service: Killing process 373 (systemd-logind) with signal SIGABRT.
Jul 16 03:36:54 homeassistant kernel: audit: type=1701 audit(1626406516.169:839): auid=4294967295 uid=0 gid=0 ses=4294967295 subj==unconfined pid=373 comm="systemd-logind" exe="/u  sr/lib/systemd/systemd-logind" sig=6 res=1
Jul 16 03:37:36 homeassistant systemd[1]: systemd-resolved.service: Watchdog timeout (limit 3min)!
Jul 16 03:37:36 homeassistant systemd[1]: systemd-resolved.service: Killing process 353 (systemd-resolve) with signal SIGABRT.
Jul 16 03:38:25 homeassistant systemd[1]: systemd-logind.service: State 'stop-watchdog' timed out. Killing.
Jul 16 03:38:25 homeassistant systemd[1]: systemd-logind.service: Killing process 373 (systemd-logind) with signal SIGKILL.

Kernel logs: Retrieved with journalctl -k -b <id_previous_boot> Not much going on here.

Jul 16 02:40:06 homeassistant kernel: hassio: port 7(vethbee281e) entered disabled state
Jul 16 02:40:06 homeassistant kernel: device vethbee281e entered promiscuous mode
Jul 16 02:40:06 homeassistant kernel: audit: type=1700 audit(1626403206.849:820): dev=vethbee281e prom=256 old_prom=0 auid=4294967295 uid=0 gid=0 ses=4294967295
Jul 16 02:40:07 homeassistant kernel: audit: type=1325 audit(1626403207.385:821): table=nat family=2 entries=0 op=xt_register pid=265877 subj==unconfined comm="iptables"
Jul 16 02:40:07 homeassistant kernel: audit: type=1325 audit(1626403207.417:822): table=filter family=2 entries=0 op=xt_register pid=265878 subj==unconfined comm="iptables"
Jul 16 02:40:07 homeassistant kernel: audit: type=1325 audit(1626403207.465:823): table=nat family=2 entries=5 op=xt_replace pid=265881 subj==unconfined comm="iptables"
Jul 16 02:40:07 homeassistant kernel: audit: type=1325 audit(1626403207.489:824): table=nat family=2 entries=7 op=xt_replace pid=265882 subj==unconfined comm="iptables"
Jul 16 02:40:07 homeassistant kernel: audit: type=1325 audit(1626403207.517:825): table=nat family=2 entries=8 op=xt_replace pid=265884 subj==unconfined comm="iptables"
Jul 16 02:40:07 homeassistant kernel: audit: type=1325 audit(1626403207.541:826): table=nat family=2 entries=10 op=xt_replace pid=265885 subj==unconfined comm="iptables"
Jul 16 02:40:07 homeassistant kernel: audit: type=1325 audit(1626403207.573:827): table=nat family=2 entries=11 op=xt_replace pid=265886 subj==unconfined comm="iptables"
Jul 16 02:40:07 homeassistant kernel: audit: type=1325 audit(1626403207.613:828): table=nat family=2 entries=12 op=xt_replace pid=265887 subj==unconfined comm="iptables"
Jul 16 02:40:07 homeassistant kernel: audit: type=1325 audit(1626403207.649:829): table=nat family=2 entries=13 op=xt_replace pid=265888 subj==unconfined comm="iptables"
Jul 16 02:40:07 homeassistant kernel: eth0: renamed from veth42f5046
Jul 16 02:40:07 homeassistant kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethbee281e: link becomes ready
Jul 16 02:40:07 homeassistant kernel: hassio: port 7(vethbee281e) entered blocking state
Jul 16 02:40:07 homeassistant kernel: hassio: port 7(vethbee281e) entered forwarding state
Jul 16 03:09:40 homeassistant kernel: kauditd_printk_skb: 1 callbacks suppressed
Jul 16 03:09:40 homeassistant kernel: audit: type=1334 audit(1626404980.617:831): prog-id=321 op=LOAD
Jul 16 03:09:40 homeassistant kernel: audit: type=1334 audit(1626404980.621:832): prog-id=322 op=LOAD
Jul 16 03:09:40 homeassistant kernel: audit: type=1334 audit(1626404980.965:833): prog-id=323 op=LOAD
Jul 16 03:09:40 homeassistant kernel: audit: type=1334 audit(1626404980.965:834): prog-id=324 op=LOAD
Jul 16 03:10:10 homeassistant kernel: audit: type=1334 audit(1626405010.981:835): prog-id=322 op=UNLOAD
Jul 16 03:10:10 homeassistant kernel: audit: type=1334 audit(1626405010.981:836): prog-id=321 op=UNLOAD
Jul 16 03:10:11 homeassistant kernel: audit: type=1334 audit(1626405011.273:837): prog-id=324 op=UNLOAD
Jul 16 03:10:11 homeassistant kernel: audit: type=1334 audit(1626405011.273:838): prog-id=323 op=UNLOAD
Jul 16 03:36:54 homeassistant kernel: audit: type=1701 audit(1626406516.169:839): auid=4294967295 uid=0 gid=0 ses=4294967295 subj==unconfined pid=373 comm="systemd-logind" exe="/usr/lib/systemd/systemd-logind" sig=6 res=1

Description of problem: Every 3 to 5 days, my host crashes. This time happened during the night, with no load whatsoever or user changes. HASSIO is installed on an external M2 SSD connected via USB. When the OS crashes, I’m unable to ping the host, all Pi ligths are on (status, ethernet) but the SSD light goes off. I use Healthchecks to tell when the system crashed, last ping was 03:35 (logs time)

I’ve found similar issues but don’t know if they are relevant (and/or not resolved) #1336 #1119