core: Shelly Integration switch changing to unavailable constantly

We switched from aiocoap library to our own implementation. Code is merged on dev branch currently.

So you can wait 0.118.0 or install a development build.

Thank you for your patience,

Simone

Please open new issue and post there debug log for shelly component.

This issue should be closed.

I installed 0.118 beta via Docker and I can confirm that the Shellys no longer flip to unavailable! Great job guys!

@gigatexel @meme1337 you can also copy code from the current branch into custom components. @chemelli74 this should also work, right?

No, because DataUpdateCoordinator in HA had to be modified to to allow updating data from multicast listen. You must use HA 0.118dev for this new version of Shelly integration to work.

I’d just like to thank all of you that have worked on the resolution of this issue: Thank you!

Pardon my ignorance but is there a release schedule for HA (118) or is it simply ”when ready” - it doesnt matter, just interested (;

Go to https://developers.home-assistant.io/ and on the right, there is the Upcoming Events section 🙂 BTW it is planned to be released at 18 November

Are you willing to test the new version of the Shelly integration? The integration is based on the new version of the aioshelly library (great work @chemelli74), which already has its own COAP support and doesn’t use external packages, which should eliminate most communication errors (entity unavailability). The library also supports listening to multicast, but so far it is not implemented in the integration.

Repo: https://github.com/bieniu/ha-shelly-own-coap

The package is prepared as a custom component so the shelly directory should be copied to /config/custom_components and restart HA is needed after that.

Thanks for testing.

The updated device just became unavailable for 5 seconds and then became available again 👎

Please check with a continuous ping as latest hotfix is for WI-FI disconnect. Integration issue is expected to be something else.

Simone

The updated device just became unavailable for 5 seconds and then became available again 👎

I have just installed the FW on one of my shelly2.5 devices and it seems stable. I will let it run for a few hours and report back

@RobBie1221 I’m still testing this change. If there are no negative aspects of this, I will open PR tomorrow.

does the integration rescan every x minutes? for me when a shelly goes unavailable it stays unavailable. I have configured static ips for all my shellys (shelly 2.5). A few minutes ago I moved one of the shellys to another access point with a different ssid name. It never came back in homeassistant. Only restarting homeassistant let it be discovered again

Manually updated xxx data in the log means integration get data via multicast listener. Multicast doesn’t work well in your network for some reason.

@t0bse The log indicates that integration doesn’t always receive data from devices via multicast listening.Try to enable “IGMP snooping” for 2,4 GHz WiFi network in the router/AP configuration.

i installed the beta today but it didn‘t fix the bug for me.

If you used test version I posted make sure you remove the Shelly directory from /config/custom_components If problem still exists please post debug log for Shelly component.

@kenlasko wrote just four comments above that 0.118 helped in his case. If you have errors please post them here. Maybe they can be fixed as part of 0.118.

117.5 Timeout fetching shellyswitch25-68C63AF9E4C1 data 11:39:15 AM – Shelly (ERROR) - message first occurred at November 11, 2020, 8:54:33 PM and shows up 1072 times Constant timeouts

We switched from aiocoap library to our own implementation. Code is merged on dev branch currently.

So you can wait 0.118.0 or install a development build.

Thank you for your patience,

Simone

!!!

117.5 Timeout fetching shellyswitch25-68C63AF9E4C1 data 11:39:15 AM – Shelly (ERROR) - message first occurred at November 11, 2020, 8:54:33 PM and shows up 1072 times Constant timeouts

@maurizioandreotti60 You wrote that You are using 0.117.1. The new version will be released in 0.118 so wait for that version or run dev from Github

@Misiu I’m using the library from Gitub Repo: https://github.com/bieniu/ha-shelly-own-coap Are you referring to that one?

@maurizioandreotti60 You wrote that You are using 0.117.1. The new version will be released in 0.118 so wait for that version or run dev from Github

@bieniu , I’ve just tested this new version. I confirm that it is much better now! Great job!

Here are the details. I still have a few timeout errors:

2020-11-05 18:17:18 ERROR (MainThread) [custom_components.shelly] Timeout fetching shellyswitch25-2f-bath data

But there were just 11 errors in 3 hours. They could be related to some network issues.

And entities do not become unavailable anymore. This is really great result. Congrats and thank you!

Using the new library has no effect on my environment. Shellis continue to become unavailable for 5s periodically. Anyway you say 11 errors in 3 hours that are errors 88 errors in 24 hours. That is not good. IMHO.

Are you willing to test the new version of the Shelly integration? The integration is based on the new version of the aioshelly library (great work @chemelli74), which already has its own COAP support and doesn’t use external packages, which should eliminate most communication errors (entity unavailability). The library also supports listening to multicast, but so far it is not implemented in the integration.

Repo: https://github.com/bieniu/ha-shelly-own-coap

The package is prepared as a custom component so the shelly directory should be copied to /config/custom_components and restart HA is needed after that.

Thanks for testing.

I have to say that the issue persists using the new library and shellis continues to become unavailable periodically for 5 seconds. Here the errors on the log:

2020-11-06 16:47:47 ERROR (MainThread) [custom_components.shelly] Timeout fetching shelly1-98Fxxxxxx data 2020-11-06 16:47:55 ERROR (MainThread) [custom_components.shelly] Timeout fetching shelly1-77xxxx data 2020-11-06 16:48:17 ERROR (MainThread) [custom_components.shelly] Timeout fetching shelly1-98yyyyyyy data

Actually I have HA 0.117.1

You should wait for 0.118.

@tobi-blue Battery powered devices are still not fully supported. Overall it is much better than it was before because by listening to multicast, integration gets data from the device when device wakes up for a second or two. Previously this was not possible.

The updated device just became unavailable for 5 seconds and then became available again 👎

Please check with a continuous ping as latest hotfix is for WI-FI disconnect. Integration issue is expected to be something else. Simone

Device has became unavailable for 5 seconds (it’s always exactly 5 seconds!) between 16:34:51 and 15:34:56

Here are the ping results from the host that runs HA to the Shelly device:

Mon Oct 12 16:34:47 +03 2020: 64 bytes from 10.0.0.207: icmp_seq=339 ttl=255 time=90.5 ms Mon Oct 12 16:34:48 +03 2020: 64 bytes from 10.0.0.207: icmp_seq=340 ttl=255 time=32.8 ms Mon Oct 12 16:34:49 +03 2020: 64 bytes from 10.0.0.207: icmp_seq=341 ttl=255 time=27.6 ms Mon Oct 12 16:34:50 +03 2020: 64 bytes from 10.0.0.207: icmp_seq=342 ttl=255 time=10.6 ms Mon Oct 12 16:34:51 +03 2020: 64 bytes from 10.0.0.207: icmp_seq=343 ttl=255 time=28.7 ms Mon Oct 12 16:34:52 +03 2020: 64 bytes from 10.0.0.207: icmp_seq=344 ttl=255 time=32.0 ms Mon Oct 12 16:34:53 +03 2020: 64 bytes from 10.0.0.207: icmp_seq=345 ttl=255 time=29.2 ms Mon Oct 12 16:34:54 +03 2020: 64 bytes from 10.0.0.207: icmp_seq=346 ttl=255 time=24.8 ms Mon Oct 12 16:34:55 +03 2020: 64 bytes from 10.0.0.207: icmp_seq=347 ttl=255 time=19.4 ms Mon Oct 12 16:34:56 +03 2020: 64 bytes from 10.0.0.207: icmp_seq=348 ttl=255 time=17.1 ms Mon Oct 12 16:34:57 +03 2020: 64 bytes from 10.0.0.207: icmp_seq=349 ttl=255 time=16.3 ms Mon Oct 12 16:34:58 +03 2020: 64 bytes from 10.0.0.207: icmp_seq=350 ttl=255 time=16.7 ms Mon Oct 12 16:34:59 +03 2020: 64 bytes from 10.0.0.207: icmp_seq=351 ttl=255 time=14.3 ms

I also have constant disconnects that lasts exactly five seconds: 2020-10-26 15:26:00 ERROR (MainThread) [homeassistant.components.shelly] Timeout fetching Belysning uppe data 2020-10-26 15:26:05 INFO (MainThread) [homeassistant.components.shelly] Fetching Belysning uppe data recovered 2020-10-26 15:26:15 ERROR (MainThread) [homeassistant.components.shelly] Timeout fetching Belysning uppe data 2020-10-26 15:26:20 INFO (MainThread) [homeassistant.components.shelly] Fetching Belysning uppe data recovered

I have two shellys, powered from mains. Connected over 2,4Ghz to Ubiquiti Wireless Access Point. DHCP. Running HA 0.116.2 from a docker container in Linux. Firmware of Shellys is the latest that was available when I installed a month ago. Using the Shelly integration in HA.

I don’t know if the disconnects are visible in the image but there is a small thin grey line every 20-40 minutes.

The Ubiquiti logs does not mention any problems. The clients are listed as connected for many days and no warnings/errors related to the relevant IP addresses.

I have just installed the FW on one of my shelly2.5 devices and it seems stable. I will let it run for a few hours and report back

Just a thought… I trust/hope that the Shelly will discover/upgrade as per usual when there is an official/new version … like for example 1.9.*

There is an API call you need to run to force it to update from the default URL.

Ah, so you mean that if I upgrade my Shelly through the API (/ota?url=…) to get the hotfix I then need to update manually to the latest stable release to get it back on track?

Yes. You should use (/ota?update=true)

@alundstroem I was wrong… when looking at the shelly app it shows me that there is a pending firmware update to the latest version.

I have just installed the FW on one of my shelly2.5 devices and it seems stable. I will let it run for a few hours and report back

Just a thought… I trust/hope that the Shelly will discover/upgrade as per usual when there is an official/new version … like for example 1.9.*

There is an API call you need to run to force it to update from the default URL.

Ah, so you mean that if I upgrade my Shelly through the API (/ota?url=…) to get the hotfix I then need to update manually to the latest stable release to get it back on track?

Yes. You should use (/ota?update=true)

Ah, great - I’ll just wait for you to report back then ^^

(I cannot test it at the moment, regardless)

I found https://shelly-api-docs.shelly.cloud/, for future reference

@bieniu I’ve retrieved the directory, placed it into config/custom_components and restarted HA… all shellies are accounted for now (: I’ll keep an eye on it.

Please test new version of the aioshelly library. You have to change manifest.json in shelly folder. Line: "requirements": ["aioshelly==0.3.4"], to: "requirements": ["pip install git+https://github.com/bieniu/aioshelly.git@semaphore#aioshelly==0.3.5"],

@RobBie1221 @t0bse

Sure, will test it this evening

@bieniu Many thanks indeed. This seems to work, I ran the debugger on the DataUpdateCoordinator and where it kept giving timeouts before with this version it picks up after devices come online.

Can someone test new version of the aioshelly library? If yes:

• remove old version: pip uninstall aioshelly
• install new version: pip install git+https://github.com/bieniu/aioshelly.git@message#aioshelly==0.3.3

@bieniu first test looks good. Powered off my Shelly 1 for 5 minutes and i comes back available 😃. I will add more Shelly devcies to the HA integration now.

Many thanks for looking into this issue.

Can someone test new version of the aioshelly library? If yes:

• remove old version: pip uninstall aioshelly
• install new version: pip install git+https://github.com/bieniu/aioshelly.git@message#aioshelly==0.3.3

@bieniu I’ve installed aioshelly 0.3.3 and will test it out on 1 of my switches. I will report back in 24 hours. Thank you!

Didn’t know that, thanks.

I mean this integration https://www.home-assistant.io/integrations/ping/#binary-sensor Use Shelly device hostname/IP address as host in the configuration.

I have the same problem. However, it seems like reloading the entity (in the integrations section) makes it show online again, so no need to reboot.

I have similar issues and reloading the integration works. Will attach logs next time it happens.

@vassilis-panos OSError is already fixed #40393 You have to wait for HA 0.115.3 release.

Released just now 😉

Simone

Having the same issue. mostly it fixes itself when i reboot Home-Assistant. (not really what you want) When a Shelly device is unavailable in home-assistant i’am able to control it through the web ui or with Domoticz. Only Home-Assistant seems to have this issue.

Can you please share your home-assistant.log ? Thank you,

Simone

@ollimagic Please post HA log with entries from shelly, coap, aiocoap.

home-assistant-shelly-coap-aiocoap.log