core: Riemann Sum integration uses broken defaults
The problem
The trapezoidal
method (the default) of the Reimann Sum integration is completely incompatible with the way Home Assistant stores values.
See:
- https://github.com/home-assistant/core/issues/85040
- https://github.com/home-assistant/core/issues/74434
- https://github.com/home-assistant/core/issues/70827
I also see the issue come up on Reddit every now and then.
Every time the workaround is “use the left method”. But it’s not just those specific cases that need a different method - the default of trapezoidal
is simply wrong for practically any use case. With the way Home Assistant does not store or report updates to states with the same value, it will give inaccurate results for any sensor that could have repeated values. And that would be pretty-much every sensor out there.
The current warning in the documentation is not enough - it is not just in some specific cases that left
should be used - it should be used in practically all cases. And even the “Energy” example does not use left
, and that is where we most often see the issue come up.
I’d recommend changing the default to left
. Even though that could be considered a “breaking change”, I’d estimate it would fix many more cases than it would break.
Alternatively, require always setting the method, and make it clear in the documentation that trapezoid
should only be used in some specific cases. This requirement could be phased in over time, by first showing a deprecation message for config that doesn’t specify the method.
The right
method would probably be even worse, but at least most users should not attempt to use it.
Another related issue, but more difficult to solve:
What version of Home Assistant Core has the issue?
core-2023.4.6
What was the last working version of Home Assistant Core?
No response
What type of installation are you running?
Home Assistant OS
Integration causing the issue
Riemann sum integral
Link to integration documentation on our website
https://www.home-assistant.io/integrations/integration/
Diagnostics information
No response
Example YAML snippet
No response
Anything in the logs that might be useful for us?
No response
Additional information
No response
About this issue
- Original URL
- State: open
- Created a year ago
- Reactions: 7
- Comments: 15 (2 by maintainers)
The point the OP makes is valid, not just because HA does not store unchanged values. Battery sensors will typically only send a new value when the value changes significantly. So left Riemann is best for that too. Any sensor that updates on change - left is best.
Only on time based sampling trapezoid is better, but that is rarely the case. One could argue that time based sampling on something that requires a Riemann integral is by itself a bad choice unless the samples are very frequent.
Any sensor that has long constant values in HA need left. That is basically any sensor that can be 0 for a significant duration, such as wattage on devices, solar energy… again, most of what HA uses it for. And even if trapezoid would work, I would argue left would work pretty well too. The reverse is definitely not the case.
And did any one mention all graphs in HA also display the left Riemann equivalent, not trapezoid? That is not without reason.
The Problem still exists
For
left
, it still only updates when the underlying value changes, but when it does update it will have the correct value.For the issue of the update only happening when the value changes, see https://github.com/home-assistant/core/issues/88940. That is a more difficult one to solve - likely only by changing the implementation to be timer-based instead of / in addition to event-based.
With
trapezoid
, there is no way for the value to be correct without using a timer-based implementation. So having it as the default without fixing that is a big issue.related question, why isn’t it a timer-based implementation? only updating when the value changes introduces confusion and problems such as this one, numbers not changing until it receives a different value and the whole sensor becoming “unknown” when it receives a value of 0.
This is what is happening on my side:
this is the implementation:
the entity in the picture is “daily_smartplugs_rounded”, but the smartplugs_consumption (not rounded) has exactly the same saw-like pattern and value. though it get displayed correctly (i.e. positive) on the energy dashboard.
Don’t know, maybe i’ts a templating error or something, Tried to set daily_smartplugs_rounded to state_class: ‘total’ or state_class: ‘total_increment’ (and remove last_reset, as per docs), but is still have this weird behaviour i did not have before 2023.5.
It would update on 1st sensor change