node-zwave-js: šŸš§ META-Issue: Problems with 700 series (healing, delays, neighbors, ...) šŸš§

It seems that 700-series sticks (including the currently latest firmware 7.17) have some problems which mostly appear on networks that:

  • are relatively busy (lots of unsolicited reports like motion sensor, power meters)
  • are large and/or
  • contain some battery-powered devices

When lots of reports reach the controller in a short time, the 700-series sticks are not able to send any message. It looks like the stick is somehow blocked and simply doesnā€™t send anything, maybe not even the protocol level acknowledgements for receiving the messages, causing end nodes to repeat their messages over and over, making the situation even worse.

šŸ‘·šŸ»ā€ā™‚ļø EDIT: Fix available, see below for direct links to the updated firmware 7.17.2

šŸ”„ Bug in NVM conversion routine, potentially causing connectivity issues. Details see below

šŸ—³ If youā€™ve updated, please take part in the survey so we can see if the update helps.


We believe the following symptoms are all caused by this:

  • Huge delays (up to 60 seconds) when sending messages
  • Failure to send anything (TransmitStatus: Fail)
  • Floods of incoming reports that are transmitted over and over
  • Incorrect neighbor information (e.g. controller doesnā€™t list some/any nodes as neighbors, etc.)
  • Failure to heal the network or individual nodes, especially in busy situations

Additional background info: https://forums.homeseer.com/forum/homeseer-products-services/homeseer-z-wave-products/smartstick/1483440-does-anyone-have-a-solid-working-g3-system-at-this-time/page4#post1510687


A workaround until this is fixed is migrating back to a 500 series stick, using the migration tool 700<->500 series. Description: https://github.com/zwave-js/node-zwave-js/issues/3906#issuecomment-997484466

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 32
  • Comments: 467 (213 by maintainers)

Most upvoted comments

Letā€™s do a little poll and gather feedback!

Everyone who has updated to 7.17.1, please use the reactions on this post to indicate whether the issue is solved (šŸŽ‰) or still persists (šŸ˜•).

Edit: if the issue (seems to) persist(s), please comment and/or open a new issue referencing this, so we can check if thatā€™s really the same problem.

Actually I did get a message from them today:

The protocol team is working on this issue currently. Iā€™ll update soon with a fix.

Alright, folks. Iā€™m going to close this issue with the upcoming release. 7.17.2 seems to (largely?) resolve the problem. In current logs, Iā€™m only seeing the Fail status for a short time when the controller is actually flooded and should wait a couple of ms before attempting communication again. This will be tracked in https://github.com/zwave-js/node-zwave-js/issues/4191.

Next release is going to add a notification/warning about the firmware upgrade in supporting UIs and later the config DB browser.

I just got word from Silabs. There should be a new release coming out this friday. Iā€™m guessing thatā€™s going to be 7.17.1 or 7.18.0. Whoever has problems, please test that and report back.

Please refrain from posting off topic posts and ā€œme tooā€ posts as alcalzone said. There are plenty of us that subscribed to this for news about the actual post topic who are being bombarded by these posts that are off-topic Thank you.

On Tue, Jan 25, 2022, 7:40 AM AlCalzone @.***> wrote:

Iā€™ll answer anyways, since this is somewhat relevant hereā€¦

Can you explain the advantage/choice of going with zwavejs2mqtt over the flagship zwavejs integration?

zwavejs2mqtt gives you more control at the cost of a less direct integration. This includes:

  • configurable associations
  • custom config parameters (get/set by number/size/value without the need of a pre-existing config file)
  • and most importantly NVM backup/restore, which is necessary for migration between 500 and 700 series without re-including every single node.

ā€” Reply to this email directly, view it on GitHub https://github.com/zwave-js/node-zwave-js/issues/3906#issuecomment-1021193158, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE34CSPZWFEA726H6SXA7RTUX2R5VANCNFSM5KD7ZG7Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

Letā€™s please remember a lot of people are subscribed to this thread for updates, and try to stay more on topic.

To make it easier on future readers, weā€™re going to clean this issue up a bit and remove comments that donā€™t add to the overall issue.

If Silicon Labs does release the new firmware on GitHub, Iā€™m sure links to the files will be posted here.

The easy way to download is to use the GitHub file filter. Navigate to the gecko_sdk project, click the ā€œGo to fileā€ button near the top, and type the prefix of the file, ZW_SerialAPI_Controller, and it will show only the matching files. Then you can just click on the one for your region and chipset.

image

If itā€™s not in the list, then refine it further based on the version listed and chipset and region.

image

Which chipset is yours?

EFR32ZG14:

  • Aeotec Z-Pi 7
  • Aeotec Z-Stick 7
  • Silicon Labs UZB7 (SLUSB001A)

ZGM130S:

  • Zooz ZST10 700

I believe the Razberry 7 and HomeSeer G3 controllers also use the ZGM130S, but I donā€™t know for sure. From my experience using the wrong firmware file will just fail the update process with no harm done.

For what itā€™s worth, the 7.17.2.0 release notes include a new ā€œfixed issueā€: 750117 Large network becomes unresponsive using 700.

Edit: zwavejs2mqtt 6.3.0 has built-in support for this now. Just restore a backup of the source stick onto the target stick and youā€™re good to go.


If anyone wants to take a little risk and try the migration back to the 500 series (requires Node.js and npm to be installed):

  1. make an NVM backup of the current 700 series stick
  2. make an NVM backup of the target 500 series stick
  3. execute the convert command here: https://github.com/zwave-js/node-zwave-js/tree/master/packages/nvmedit#convert-one-nvm-to-be-compatible-with-another-one
  4. Restore the resulting NVM file on the target stick

ā—ā—ā— Disclaimer: I have quite a few unit tests ensuring the correct format but I havenā€™t actually tested restoring the resulting files on a stick, so Iā€™m not 100% certain if it will work (both the backup and the stick šŸ˜¬). If it doesnā€™t work, you should be able to hard-reset the target stick to get it working again, but Iā€™m not making guarantees here. Try at your own risk!

Quote:

Please try the release coming out this Friday and let me know if it fixes things.

Iā€™ll answer anyways, since this is somewhat relevant hereā€¦

Can you explain the advantage/choice of going with zwavejs2mqtt over the flagship zwavejs integration?

zwavejs2mqtt gives you more control at the cost of a less direct integration. This includes:

  • configurable associations
  • custom config parameters (get/set by number/size/value without the need of a pre-existing config file)
  • and most importantly NVM backup/restore, which is necessary for migration between 500 and 700 series without re-including every single node.

Also, despite the name it does not require MQTT, it just enables you to use it. It uses the same websocket connection that the official integration does.

Does anyone happen to know where this firmware file will exist for direct download? Iā€™d rather use the CLI upgrade method than spin up a VM for Simplicity Studio.

Iā€™m hoping it will be published on GitHub. Iā€™ll take this chance to plug my own upgrade instructions. šŸ˜‰

Please refrain from posting ā€œme tooā€ and similar comments here. It does not add any value and notify everyone whoā€™s subscribed to the issue unnecessarily. Reactions are meant for this.

This is not here to spam, so feel free to erase the message - i just want to lift my hat up for you AlCalzone! You do a tremendous work with zwave-js - and I hope more of us start sponsring you šŸ˜ƒ

Failure to heal the network or individual nodes, especially in busy situations

If any message gets sent in the network while healing it will fail. If the healed node is not in direct range from the controller, it will always fail. Right now I gave up coding anything zwave because of this, it makes it completely useless and unusable.I hope they can manage to find a solution soon, VERY soon.

Have you updated firmware on the switches? I had a lot of improvement with 7.17.2 on the stick and then again a boost in stability after updating all Zooz switches. ZEN34 are now on 1.3, ZEN71/72 on 10.0, etc.

I upgraded my Zooz ZST10-700 USB stick to 7.17.2, which helped some. Are the Zooz switch firmwares publicly available? I have ZEN22, ZEN24 and ZEN74 switches, and they still go dead, occasionally.

Youā€™ll have to request firmware from Zooz support

Things on 7.17.2 are a lot better for me, still buggy tho. The outermost edge light switch pretty much always drops off despite having plenty of switches in range of it, all battery powered devices I have to pair essentially on top of the controller, I have to constantly keep them awake otherwise they wonā€™t fully interview, about 40% of them I have to exclude and re-add due to incomplete interviews, there is still a good amount of lag, but at least itā€™s usable now. I have a fairly large network (60 devices now, still have about 15 more to install) in a ~2400 sq ft four story townhome, controller is on a 10ft USB extension cable on one side of the second level.

Is there any reason why zwavejs2mqtt doesnā€™t show the full firmware version?

Its coming: https://github.com/zwave-js/node-zwave-js/pull/4355 šŸ˜ƒ

They donā€™t have their own firmware. The file they provide has the exact same hash as the one on GitHub. All 700 firmware is the same (one for each of the two chipsets), except Zwave.meā€™s.

Same as @justindthomas, abandoned my 500 stick and upgraded my 700 stick. (Zooz, S2 700) THen replaced my stick with the 700.

I had a few nodes that had moved or otherwise needed to be updated. I tried to exclude - re-add but was having a terrible time. So I went on a tear manually repairing all the nodes between the coordinator and the stuff I needed to re-add. After about 5 or 6, the node added successfully. Everything started coming online and I left it to sit over dinner.

When I came back, I fired off a full network heal (and prayed.) 5 minutes in and itā€™s already successfully healed through node 11 and trucking along nicely. Should have the entire network minus the battery devices in about an hour. Duplicate sends, and the controller timeout errors have all but vanished, and Iā€™m not seeing devices drop outā€¦ The system is quiteā€¦ Snappy.

Iā€™m cautiously optimistic.

EDIT: (~+1hr) All powered nodes (~65+) healed successfully; Iā€™m not running around finding all the blind remotes I never use to force them to heal.) cleared a longstanding problem where there was HEAVY (meaning 80% of my devices) all routing through one node Network ā€˜feelsā€™ downright fastā€¦

Can you explain the advantage/choice of going with zwavejs2mqtt over the flagship zwavejs integration?

This thread is getting way off topic again. This would be a better question to ask elsewhere.

The latest release no longer has any references to the broken packages. Also, zwavejs2mqtt 6.3.0 has built-in support now.

Hope to fix for 700 series . I donā€™t wanna buy an ā€žoldā€œ stick ā€¦

I unknowingly ran into this issue (ping fixes dead nodes) and the folks at the hass community forum pointed me here. I am trying to setup a simple compensating automation, but having some problems. If anyone has any ideas take a peek here? https://community.home-assistant.io/t/automate-zwavejs-ping-dead-nodes/374307/7

I have setup a way that automatically pings dead nodes - working 99% of the time - replied in https://community.home-assistant.io/t/automate-zwavejs-ping-dead-nodes/374307/7 so you can see my solution there.

Try skipping the nvmedit steps and just use Z2M directly, e.g. restore the 500.bin file.

Not sure how I did thatā€¦ Maybe I need some glasses. I got the backup loaded and will report back if I have any further issues with dead nodes. Everything seems to be good for now. Thanks again.

Just FYI that the 7.17.2 firmware is it now.

Letā€™s do a little poll and gather feedback!

Everyone who has updated to 7.17.1, please use the reactions on this post to indicate whether the issue is solved (šŸŽ‰) or still persists (šŸ˜•).

Edit: if the issue (seems to) persist(s), please comment and/or open a new issue referencing this, so we can check if thatā€™s really the same problem.

I updated my Zooz S2 700 stick to 7.17 without issue, but my problems mutated. Where before the update I was having trouble with random wired devices being marked as dead after 1-3 commands in short succession, I am now having problems with my battery powered devices suddenly seeming to fail to update their new status. The device wakes up, seemingly works fine (a door lock being unlocked with a pin code, for instance), but then flashes a generic wireless error message. I lock the door behind me, and Home Assistant continues to say the lock is unlocked. I refresh values, it says the node responded to waking up, but then says the node timed out on the refresh. If I reboot the zwavejs (w/ mqtt) container, suddenly it works and understands my locks are locked. If I stand there and flick the lock open/closed for a while (3-4 lock/unlock cycles) sometimes it will un-stick itself and realize the lock is locked.

When I back up and restore to my 500 series controller, all problems vanish, so I am convinced this is still a 700-series controller issue.

Sadly, my symptoms persist: intermittent failed commands, very slow response times,

I can take a look at this if you open a separate issue. Failed commands may very well be a symptom of something else.

I was able to successfully update the firmware on my Zooz ZST10 700 using the Serial app. Based on the Aeotec instructions. I sent the 2 binary strings as text to the device, then uploaded the file, then pressed ā€œRunā€ (2) on the menu from the device.

I think you could probably do this with the lrzsz package, but after hours of trying to get that to work it took 3 mins with Serial.

I just used the tool to successfully move back from a Zooz ZST-10 500 series to a Zooz ZST-10 700 series. Thank you for getting the migration tool sorted!

Update: The upcoming v8.11.4 (out in a few minutes) should fix the remaining issues preventing a 500-to-700 series migration. Until this is available in zwavejs2mqtt, you can manually convert the backups using

npx @zwave-js/nvmedit@8.11.4 convert --source /path/to/500_backup.nvm --target /path/to/700_backup.nvm --out /path/to/converted.nvm

and restore the converted backup onto the stick.

My firmware is 7.0 on the device

It wonā€™t get worse than this.

But itā€™s odd that it looks like it had 0.05 seconds to respond to the [REQ] [AssignSUCReturnRoute] before being proclaimed dead.

WTF. If you have more complete logs, I could look at it in a separate issue. It feels like weā€™re missing some context here.

~3 days now and seems to be ok on Zooz stick. Required a full heal after upgrade and starting up zwavejs2mqtt. Heal took ~12 hours to fully complete due to battery devices, otherwise mains nodes finished in minutes. No further action since, no dead nodes since. Thank you to all who helped action this firmware update!

Yes. It wonā€™t mess up the naming if you follow these steps. There was a bug until the most recent patch release that could make you lose it if the device went unavailable while the integration was connected. These steps work both before and after the bug was patched.

You literally follow these exact steps. You donā€™t need to switch back but you can by reversing them. In zjs2mqtt you just backup the one nvm and restore to the other, I believe. Iā€™ve never done it.

Switching does not require renaming your devices.

Disable the Z-Wave JS integration. Do not remove the Z-Wave JS integration or you will lose all device and entity naming. This will automatically stop the official Z-Wave JS add-on.

Note your network security keys from the official add-on.

Install and configure the Z-Wave JS to MQTT add-on, including setting the location of your Z-Wave device and the network security keys.

Add the Z-Wave JS integration again (even though it is still installed), and uncheck the ā€œUse the Z-Wave JS Supervisor add-onā€. Enter the correct address for the community add-on in the URL field in the next step.

Uninstall the official Z-Wave JS add-on.

Enable the Z-Wave JS integration.

IC. I suggest you open a new issue to troubleshoot that, as Iā€™m not convinced itā€™s related to this firmware or the original bug.

I saw the same thing. Working Aeotec 500 nvm backup restored to UZB7 inside of zjs2mqtt. UZB7 restore completed, but nothing would ping or interview after it did. Rebooted, and saw the same thing. It was communicating with the controller itself though / its status was green.

Likely not firmware related, but rather conversion related. For now I just went back to the Aeotec 500.

Iā€™ll try again tomorrow and see if I can get some logs, and post on the other issue or start a new one.

For me the instability primarily manifested as battery nodes not being able to identify half its neighbors. There would be 3 mains repeaters within 10 feet and it would not see them as neighbors, but would register a node clear on the other side of the house as a neighbor. It was also listed as one of the symptoms at the top of this thread. Now that a bunch of people have done heals, which should re-calculate neighbors, does anyone see an improvement in that area?

For me, until now I can say, it works !!! Night mode - no dead nodes anymoreā€¦ā€¦ fingers crosses. THX @AlCalzone thx silabs šŸ˜‰

Upgraded Zooz ZST10 700 using pc controller without issue. Typically see 4-5 dead nodes per day, will report back in a bit.

Looks like 7.17.1.344 just dropped. Iā€™ll be updating my stick shortly and bedding in the update. Hopefully all is fixed.

Will we have to do a direct firmware update on the 700 series controllers, and are there instructions for this? Or will the update be in HA, and mean that as part of an HA update, the 700 series controllers are expecting to work better?

I would think you need to do this on the controller, instructions on the Aeontec Gen7 is here https://help.aeotec.com/support/solutions/articles/6000252296-update-z-stick-7-with-windows

I think Iā€™ve reached resolution with the HA folks to change the recommendation. Iā€™ve also proposed adding an alert: https://github.com/home-assistant/alerts.home-assistant.io/pull/474

SiLabs doesnā€™t release new firmware that often - even after they identify an issue. Just my opinion, but I would expect this to drag out for quite a while. Not trying to be negative, just trying to set expectations.

Thanks for the description. Thatā€™s all I need I hope. The migration was only merged on master, but in the future youā€™ll just be able to restore a backup from a different stick and it will automatically be converted.

I stiill see these messages occasionally

Iā€™d need additional debug-level logs to diagnose this, but thatā€™s OT.

I wonder whether the issue could be less about overall volume and more about multiple messages arriving very close together?

Short bursts definitely seem enough to trigger this. Large, chatty networks are just more reliable at triggering the issue.

Iā€™ve gotten a Zniffer log from another user and the general sequence (in his case) goes like this:

  • short, (random?) RF issue causes some ACKs to get lost and causes one device to re-transmit 2-3 times (a matter of milliseconds)
  • since the packets are routed, the repeaters also re-transmit
  • This compounds to ~10-15 messages in a short time, causing the stick to stop responding (including the routed ACKs)
  • While it is not responsive, 1-2 other nodes send messages too and donā€™t get the expected ACKs and also re-transmit, including their repeaters - you get where this is goingā€¦
  • After everything dies down, things go back to normal.

That would be the correct firmware for the Aeotec Gen5 Plus Z-Stick, not the versions previous to the Plus.

Actually, it is for the Gen5 (without ā€œ+ā€): ā€œThis firmware update will update the Z-Stick Gen5 to the same firmware Z-Stick Gen5+ uses (Z-Wave device firmware 6.07 or Firmware version V1.02 built with Z-Wave SDK 6.81.06).ā€

Another option would be a Zooz ZST10. S2 they appear to have some ā€œopen boxā€ I have found their used devices work as good as new and have the same support.

At the last minute I delayed moving to my ZST10-700 S2 and am staying on my older 500 Series ZST10 S2

I migrated my setup from a Zooz ZST-10 700 S2 to the Zooz ZST-10 S2 today using the command line NVMedit tool for the conversion. The process completed without issue and the restored stick came up right up.

What they released means nothing because they use firmware from SiLabs. Download SiLabs PC Controller and it will fetch latest firmware. Either way it wonā€™t help because latest version didnā€™t fix the issue.

@rice1204 make sure not to use an old locally installed version of @zwave-js/nvmedit, but the latest available one. Old versions had a bug where the radio settings could get messed up during the migration.

@rice1204 thatā€™s strange. After I restored the backup to gen 7 all my existing nodes was displayed as being interviewed in the zwavejs UI. Some battery devices was displayed as dead, since they where asleep, but after manually waking them up, they performed interview as well.

Do we need a new poll? 7.17.2 has solved my issues, havenā€™t tried 7.17.1

I am having trouble with dead devices (switches, dimmer, but not battery powered devices) after rebooting the HA VM or the host machine. I upgraded to 1.17.2 and things stayed the same.

The workaround in another thread to have an automation that triggers when a zwave device goes dead to ping it works for me, after adding a ā€˜homeassistant startā€™ trigger.

Iā€™m also getting the object not found error when trying to convert my 700 backup to json.

Please send me the backup via email.

@joel-bourquard You need to use zwavejs-9 tag (if you are using docker) as that change is coming with zwavejs-9 that is on beta

@Botched1 From your description, you have probably disabled the driver logs or are logging to a file, so you see only the zwavejs2mqtt logs in the debug window.

Just installed the 7.17.2 into my Zooz 700. Been fighting with the random dead nodes for a few weeks now so fingers crossed.

Besides the logs question, is there a way I can determine if Iā€™m on build .2 ?

The PC Controller software shows the firmware versions in the node info tab of each node. I canā€™t remember if it shows the revision part though (.02). You can get the firmware version from zwavejs2mqtt using a driver function:

const {Message, MessageType, MessagePriority} = this.require('zwave-js')
const GetProtocolVersion = new Message(driver, { type: MessageType.Request, functionType: 0x9 });
await driver.sendMessage(GetProtocolVersion, { priority: MessagePriority.Controller });

After executing that code, in the driver logs youā€™ll see something like:

2022-03-12T23:47:25.662Z DRIVER Ā« [RES] [UNKNOWN_FUNC_UNKNOWN_0x09]
                                    payload: 0x00071101015830303030303030303030303030303030

The relevant version values are bytes 2-4 of the payload, which are 071101. These are hex numbers, so 0x07 = 7, 0x11 = 17, and 0x01 = 1, in other words 7.17.01.

let us know when you test it.

@lappat I have received my replacement ZST-10 700. It came out of the box with 7.15 so my first task was to upgrade it to 7.17 using the file downloaded from the Zooz website and Zwave PC Controller, the process had no hiccups and only took about 3 minutes. I was able to do an NVM backup from ZwaveJS2MQTT, unplug the old stick, plug in the new stick, and do an NVM Restore and all of my devices started working within seconds.

I did lose some custom entity names inside Home Assistant, but thatā€™s way better than having to go through the house and exclude/include devices. All the names I had saved inside ZwaveJS were still there. Over all it was a very seamless process!

And the fantastic news is my Zwave (not Zwave+) deadbolt that was a hop away from the controller included and started working with minimal fuss, I had spent 3 days trying to get that going on firmware 7.15. I am a very happy camper now! šŸ˜

I have 90 zwave devices and one (zwave dimmer I have many of) in particular tends to be marked dead. A simple ping usually revives it and the others that do the same.

If you can capture both in a log and post it in another issue, I can take a look if this is related or just a suboptimal connection of that one particular device.

I updated my Zooz 700 series stick five days ago with this patch. So far, so good. My zwave was losing nodes several times a day before the patch. Since I applied the patch I have had no issues.

Just for anyone else having issues, Zooz has posted an official set of update instructions on their website.

https://www.support.getzooz.com/kb/article/931-how-to-perform-an-ota-firmware-update-on-your-zst10-700-z-wave-stick/

The file on the site has the same MD5 hash as the one I have downloaded from https://github.com/SiliconLabs/gecko_sdk/blob/gsdk_4.0/protocol/z-wave/Apps/bin/gbl/ZW_SerialAPI_Controller_7_17_1_344_ZGM130S_REGION_US.gbl?raw=true

Iā€™m 13 emails deep with support right now and havenā€™t been able to get my stick updated, but maybe these instructions will be helpful for others.

FWIW My Zooz stick upgraded fine with PC Controller on Windows.

Yes - same situation here with the stick showing up differentlyā€¦

I moved the antenna and placed it on a 1m extension, but Iā€™m still having the same issuesā€¦ bummer

Also - no change in RF performance. A heal results in very few 1-hop routes, and most being 2-hop routes. This is the exact opposite of the network map with the 500 series (mostly 1-hops and a smaller number of 2-hops)

Iā€™ve finally managed to find some time to upgrade my Z-Pi 7 to 7.17.1 and run a full heal: works like a charm, thanks.

Iā€™ll try the heal first and open another ticket if it recurs. Thanks.

UPDATE: Healing revealed some additional weirdness with LZW31 devices. Ticket here: https://github.com/zwave-js/node-zwave-js/issues/4145

Iā€™d try that. Letā€™s do this. Please open a new issue for this device. I think we should split off any new issues into a separate issue (referencing this one) so that we can troubleshoot each without messages for different issues crossing. If it ends up relating to this, we can update in here.

Please make the issue title like: [Investigate 700 Bug] Whatever Is Happening.

For others, you are likely to see similar single-device issues that need to be solved. The 700 series bug causes the controller to misbehave on a busy mesh. That is hopefully fixed, but there is still the issue of why your mesh is so busy in the first place. Often that is due to misconfigurations or misbehaving devices.

Nodes do die sometimes for legitimate reasons. I wouldnā€™t read a single one dying as being this problem absent a log file. Do you have a log? Try pinging the device to see if it comes back to life. And for the love of God, all of you, please turn on debug-level logging to file in zjs2mqtt so that we can start to capture some logs of these things happening.

The new 700 series FW seems to work well with 47 devices: I upgraded my UZB7 from 7.15 to the new 7.17, and left it in the same physical location, with the same software driving it: zwjs v8.10.1, zwjs-ui v6.10. UZB 7 is on a usb cable several feet away from the host What Iā€™ve noticed so far:

  • Heavy spam of duplicate reports from individual devices has stopped, very nice. Oops, a 4-in-1 just reported a slew of identical temperature readings of 57.59F, after first reporting once on that change from the prior value. I will say that the amount of this spam is vastly reduced.
  • A network heal completed all the way through. That hasnā€™t happened before
  • However, OTA FW updates of the (mains powered) devices that have continually failed to update before, still fail. So far, only 2 of 6 upgrades even started. One of those quit part way through, and the other is proceeding very slowly so far. This is after the successful network heal.
  • Iā€™ll monitor for dead nodes. Iā€™ve regularly had a handful report out as dead until I ping them.

Sounds like the same issue as #3992.

Thatā€™s going to be difficult to diagnose as itā€™s too many moving parts. Wouldnā€™t the 700 stick still have been setup?

People testing: please try to refrain changing anything other than the fw version between a non-working stick and the test case.

All my nodes showed up dead. Converted NVM from 700 to 500 back to 700 again (added more nodes since downgrading) after upgrading firmware. Maybe I did something wrong?

@alexruffell loglevels less than debug arenā€™t helpful to diagnose anything.

Because the UZB7 does not use that chip. Scroll up, the correct files and names of files have been posted numerous times.

First test - 20 Qubinos Flush dimmer at the same time - no dead nodes or stucking.

before it ends with dead Nodes and stucking šŸ‘

Go with non 255 version. That one is just for testing purposes.

Thatā€™s wrong. They have begun releasing the firmware images on GitHub as of a couple of versions ago.

If it makes any of you feel better that have purchased a second stick, you can convert those sticks to zniffers to help diagnose future issues. Itā€™s generally a one-way conversion though, so make sure this is fixed first.

Does anyone happen to know where this firmware file will exist for direct download? Iā€™d rather use the CLI upgrade method than spin up a VM for Simplicity Studio.

Is there a command-by-command walkthrough of how to do it

Its linked to in the very first post: https://github.com/zwave-js/node-zwave-js/issues/3906#issuecomment-997484466

Firmware upgrade is described on Aeotecā€™s support pages.

Iā€™m going to FUBAR my setup worse than it already is

Since youā€™re only working with a backup of the current stick, you canā€™t make it worse.

I removed the recommendation already. The alert shows at the top of the zwavejs integration docs. Both are already live.

Has Nabu Casa/HA retracted their recommendation for users to get a 700 series stick? If so, glad that they did.

I read that recommendation weeks ago, and went straight out and bought a 700 series stick. Only to be riddled with headaches.

@darkbasic , I have 2 Razberry 7 Proā€™s that I just got last week and heā€™s full of s&^t if heā€™s saying that itā€™s immune to the issue. I started noticing issues (delays and reception) as I was building the network from scratch and assumed it was me. I moved over to zwavejs2mqtt after a couple of days and the network was closer to completion. If I had to guess, heā€™s taking care of the dead nodes issues in the background with z-way server, but the trifecta of dead nodes, delay, neighbor, and reception related issues are very obvious in z2m.

I got a Zooz stick in the mail today (clearance deal on smarthouse 1 day shipping, a aeotec gen5+ stick wonā€™t arrive till next next week), restored the NVM through z2m, healed the network and itā€™s been wonderful.

The second box is in the guest house, but thatā€™s only a network of 6 or 7 devices and no problems that I can see. But Iā€™ll probably throw the Aeotec stick on there when it arrives just to be safe.

Now I can get on with the rest of my migration (automation through node-red from reactor) with some peace of mind.

-Iā€™ve posted my issues over at the z-wave.me board, so weā€™ll see if any one else replies.

We have contact with them as well. They are using a custom firmware. I just emailed them to be sure they understand the issue and that thatā€™s accurate.

Here are a couple of known bugs. Perhaps some of the 700 Series issue is related to S2 security?

https://nvd.nist.gov/vuln/detail/CVE-2020-10137

https://nvd.nist.gov/vuln/detail/CVE-2020-9061

EU , try this https://github.com/SiliconLabs/gecko_sdk/blob/991121c706578c9a2135b6f75cc88856e8c64bdc/protocol/z-wave/Apps/bin/gbl/ZW_SerialAPI_Controller_7_17_0_284_EFR32ZG14_REGION_EU.gbl

Fra: fisch55 @.> Sendt: mandag 17. januar 2022 21:24 Til: zwave-js/node-zwave-js @.> Kopi: Snapperud @.>; Manual @.> Emne: Re: [zwave-js/node-zwave-js] šŸš§ META-Issue: Problems with 700 series (healing, delays, neighbors, ā€¦) šŸš§ (Issue #3906)

Ok , but on GitHub I found only 7.17 US - maybe no EU Version available.

ā€” Reply to this email directly, view it on GitHubhttps://github.com/zwave-js/node-zwave-js/issues/3906#issuecomment-1014863857, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ANNT5JFQRBJLBZEPAIVXWFTUWR3FTANCNFSM5KD7ZG7Q. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you are subscribed to this thread.Message ID: @.@.>>

Iā€™ve just emailed zooz and silabs to feedback this and another problem with the 700 series (I also ran into a problem upgrading firmware on some inovelli switches through PC controller - changed to a 500 series and the firmware update went through instantly).

Iā€™ve been working with zwave for many years now as an end user, so I know how infrequently they release updates.

There are fragments of information everywhere from confused users describing the problems above, but they are not aware of the cause. This seems to be the only area where information on the issue with the Silabs 700 series firmware can be found. Might be helpful to make more noise from the community. Instead of ā€œWeā€™re working with Silabs to get to the bottom of this. Please donā€™t open any further issues.ā€ Is it possible to change the announcement at the top to say something along the lines of:

ā€œWe need Silabs to update their firmware. Please email Silabs to let them know you are having zwave network problems so they are aware of the severityā€

I raised that issue, it was suggested by Frenck that the upstream project would probably have to withdraw its support for the device for HA to stop recommending it.

I can talk to him. We do support these sticks, many users have no problems with them, but I wouldnā€™t recommend them currently.

Got a link or a reference?

@dwaleke Iā€™ve opened a separate issue for that - nice catch!

Timeout while waiting for a response from the controller (ZW0200)

This is the controller having an issue. Maybe it didnā€™t restart properly after the restore. Try re-plugging it and restarting z2m.

Dumb questionā€¦ After migrating my NVM from UZB7 to Aeon Zstick Gen5, everything is working just fine - the convert was easy, and after swapping out USB sticks and restarting the container all of my devices control and update fine (including S2 paired ones).

However, my controller still shows up as the old UZB in zwavejs2mqtt. Is there something I can do to ā€œrefreshā€ the controller manufacturer/product/product code?

This is my zwavejs2mqtt controller entry now that Iā€™m running on the Aeotec Zstick: image

Restart zwavejs2mqtt. I think thatā€™s what did it for me.

I confirmed with an Aoetec Gen5 (non-plus) firmware 1.1 it fails, but after following https://help.aeotec.com/support/solutions/articles/6000252294-z-stick-gen5-v1-02-firmware-update

The firmware now shows 1.2 and the convert works perfectly.

That would be the correct firmware for the Aeotec Gen5 Plus Z-Stick, not the versions previous to the Plus.

Actually, it is for the Gen5 (without ā€œ+ā€): ā€œThis firmware update will update the Z-Stick Gen5 to the same firmware Z-Stick Gen5+ uses (Z-Wave device firmware 6.07 or Firmware version V1.02 built with Z-Wave SDK 6.81.06).ā€

Now that I have re-read the support article, I agree with you. I went through to many iterations with different controllers and got myself confused. šŸ˜ƒ

Just received my Aeotec Gen5+ stick from amazon about five minutes ago. Two NVM dumps, then used the npx @zwave-js/nvmedit convert --source <source> --target <target> --out <output> command from this page

Everything came up immediately on the 500 series stick, IMMEDIATELY responsive.

@johanschelin I can personally confirm that it works using the guidance in this thread to downgrade from an Aeotec Z-Stick 7 to an Aeotec Z-Stick 5+. My house has been running on the migrated network for the last 10 days.

@AlCalzone Has there been any update from Silicon Labs in regards to this issue? If thereā€™s anything we can do to help troubleshoot/escalate this issue, please donā€™t hesitate to ask. Either way, thank you for all the time youā€™ve put into getting to the bottom of this!

I edited the original post above. 8.9.0-beta.4-pr-3789-5d3ea84 has the node ID change. If all goes well I can release it officially soon.

bingo! it was change number (2) alone. ā€œnodeIdā€: 0

all devices came straight to live šŸ˜ƒ automations work.

awesome, many thanks @AlCalzone

I will now monitor and will let you know if I bump into any post migration issues.

I have finally managed to restore my converted NVM on Zstick 5+. All the nodes appear to be dead same as for @justindthomas.

home id: 0; home hex: 0x0

I am sending you the 500 NVMs (original and restore) and the driver log.