core: History disappears for entries that don't change state
The problem
If an entities state does not change over the period that the recorder retains data, then it is no longer displayed on the history tab even if that entity is specified under the include: section of the config.
eg. I have my recorder set to retain 2 days worth of data. When the system starts up, a light entity is ‘off’. If the light is never turned on, then after 2 days it disappears from the history graph until either it is turned on, or the core is restarted.
What version of Home Assistant Core has the issue?
core-2022.4.5
What was the last working version of Home Assistant Core?
No response
What type of installation are you running?
Home Assistant OS
Integration causing the issue
History
Link to integration documentation on our website
https://www.home-assistant.io/integrations/history/
Diagnostics information
No response
Example YAML snippet
No response
Anything in the logs that might be useful for us?
No, this problem causes no errors or warnings
Additional information
No response
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 2
- Comments: 24 (10 by maintainers)
Commits related to this issue
- Fix history not include start time state We need to use run_information_with_session in this case instead of run_information_from_instance since run_information_with_session will look back in the dat... — committed to bdraco/home-assistant by bdraco 2 years ago
- Fix history not include start time state We need to use run_information_with_session in this case instead of run_information_from_instance since run_information_with_session will look back in the dat... — committed to bdraco/home-assistant by bdraco 2 years ago
- Refactor tracking of the recorder run history #70346 showed that the current design is error prone All of the run_information functions have been replaced with Recorder.run_history.get so there is ne... — committed to bdraco/home-assistant by bdraco 2 years ago
I found the issue.
I’m going to do a small fix for 2022.4 and then refactor this a bit as the way it works is far to confusing to maintain this going forward
I think it is very important to make clear that this has nothing to do with a DB purge. It happens at every time scale, even with data that is entirely inside the purge_keep_days period.
Consider the following example. You have an entity with the following 4 recorded states, ignoring time zones for sake of simplicity:
Now you issue a query for the day, so from 2022-04-22 00:00 to 2022-04-22 23:59. The exact timestamps don’t really matter, the important thing is that the first state, the ‘on’ at 2022-04-21 18:00 is not included in the query (but it is in the DB !). The result prior to 2022.4 looked like this:
Notice how the first initial state is ‘virtual’. It is generated at the beginning of the query period from the previously recorded state on 2022-04-21 18:00, so to give you an initial value on your query. You will get the same query results in 2022.4, unless you have rebooted HA anytime after 2022-04-21 18:00. In this case, the result you get is this:
No more generated initial state.
This is clearly a regression from 2022.3.x.