zha-device-handlers: [BUG] Custom quirks not working in HA 2023.03
Recently in HA v2023.03, the ZHA integration has stopped working due to issues with some user-installed custom quirks.
The issue is caused by some changes made to Tuya’s MCU device implementations. These changes (necessary and brilliantly implemented) could have been deployed in a different way to avoid the impact on integration and the noise generated. This could be fixed with PR #2242, but it only mitigates the effect of possible changes.
It should be said that the use of custom quirks implies (or should imply) some technical knowledge about what is being done and about the analysis of the incidents that can be generated. On many occasions the problem has been detected and corrected by the users, but not all users are the same.
My intention is to collect here the problem that occurred, the possible solutions and lessons learned.
The first thing is to identify the origin (or origins) of the problem. I have detected mainly 2:
- PR #2043 that removes the
dp_typefrom theDPToAttributeMappingclass - PR #2017 that implements the autodetection of the Tuya data type from the value type
Nothing to reproach the changes. I gave them my approval myself.
But it is clear that we underestimate the impact of change. Although all the code implemented in the library was modified according to the new implementation, the custom quirks hosted in the user installations have not, and these have affected the integration. In order to mitigate this type of problem, it would have been convenient to grant a deprecation time to the implemented changes. It will not always be achievable, but at least we should considerate it.
Let it be clear that this issue is only addressed to myself and that he only seeks to collect my comments on it.
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 15 (5 by maintainers)
Yeah, warnings won’t help as much as they should. I deprecated
zigpy.zcl.foundation.Commandin favor ofzigpy.zcl.foundation.GeneralCommandin December of 2021. When I finally removed it in December of 2022 (over a year later!), it still broke a ton of installs and generated a lot of issues because many custom quirk users are non-technical and do not read warnings to begin with.There isn’t really much that can be done other than merging the custom quirks into the main repo, or letting things break (either explicitly or silently). I also agree with @TheJulianJES regarding custom quirks integrating with the frontend: they’re 100% meant for development and quick testing, not for general use.
I vote for merging #2242 to at least prevent blocking startup and upgrading the warnings to errors.
Wish full thinking from my side 😉
So ZHA throwing warning in the log is the only realistic option.
Hmm, I don’t think a PR with those changes would be accepted. Custom quirks are considered to be similar to custom integrations. See: https://github.com/home-assistant/home-assistant.io/pull/23884#discussion_r955050091
I am sure that any measure will not cover all cases, but if the avalanche of “ZHA breaks in version XX” incidents can at least be avoided, it is worth it. (And on a personal level I will know that I have tried to avoid it).
In my head it would be easy if we warn users were it bother more, in the HA notifications panel:
I think that is a interesting approach for all these ‘hit & run’ users, but not enough time or enough knowledge to explore that approach and do it myself.
I don’t like the idea of quirks failing ‘silently’ but totally against that a quirk (and even worse, a custom quirk) breaking the ZHA integration.
Thanks for your words. I really appreciate it.
The reason for implanting custom quirk is for easy testing quirk for devs and users. Its not made for making local quirk and not putting on PR for adding it and its no guarantee that ZHA / Zigpy is braking them like with my tuya TRV that is not PRed then no have writing the test for it and its the same for most of this devices and its have being broken.
The most impotent is knowing and informing users then making braking changes also for custom quirk for getting it working well also then the device is not officiously supported then the quirk is not PRed. Also making good instructions so user can updating there quirk ten it happening so they can getting there devices working OK a gen.
Braking thins is always not welcome but is always happening and its importing fixing them like the large reworking of ZCL R7 that was braking very mush bit it wars very needed and we is getting good things from it all the time after its was done and i think its the same with the tuya MCU updates (and converting most tuya devices toEnchantedDevice that also is very needed and can braking some devices).
Understanding and learning then things is not going well it is the definition of one intelligent animal and here is most very intelligent 😃)