core: Synology DSM: State of camera entity not updated after state change in Synology Surveillance Station

The problem

The states of my camera entities (added via Synology_DSM integration) do not update following changes within Synology Surveillance Station. However, after reloading the integration, the states in Home Assistant change to appropriate values.

For example: In Surveillance Station, my camera is at idle. The state of the camera.synology_* entity is idle in Home Assistant, which is correct. If I start a recording within Surveillance Station, the state of the camera.synology_* entity is still idle in Home assistant. After reloading the Synology_DSM integration while the camera is still recording, the state of the camera.synology_* entity is recording.

I tried to understand how entities are updated. After monitoring states changes via the Template tab of the developer-tools page, I noticed the following:

  • The sensors linked with cpu and network seem to update according to the "scan_interval": 1 integration setting.
  • the switch.synology_home_mode entity is correctly updated following changes within Surveillance Station.
  • For other entities (whose domains are sensor,binary_sensor and camera), perhaps a status change within Synology Surveillance Station is necessary for the update to be visible within Home Assistant.

In any case, the homeassistant.update_entity does not seem to work.

Environment

  • Home Assistant Core release with the issue: 0.118
  • Last working Home Assistant Core release (if known): No idea as the problem is dificult to detect
  • Operating environment (OS/Container/Supervised/Core): Docker on Synology NAS, using host network
  • Integration causing this issue: Synology DSM
  • Link to integration documentation on our website: https://www.home-assistant.io/integrations/synology_dsm/

Problem-relevant configuration

Configuration via UI (with starts instead of specific parameters)

            {
                "entry_id": "******",
                "version": 1,
                "domain": "synology_dsm",
                "title": "******",
                "data": {
                    "host": "******",
                    "port": "******",
                    "ssl": true,
                    "verify_ssl": false,
                    "username": "******",
                    "password": "******",
                    "mac": [
                        "******"
                    ]
                },
                "options": {
                    "scan_interval": 1,
                    "timeout": 10
                },
                "system_options": {
                    "disable_new_entities": false
                },
                "source": "user",
                "connection_class": "cloud_poll",
                "unique_id": "******"
            }

Additional information

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 58 (24 by maintainers)

Most upvoted comments

The issue I’m talking about is unavailability: 2021-02-07T22:06:49.824462+00:00>, new_state=<state camera.nasdomowy_d_link_dcs_2670l=unavailable It’s possible that’s due to the timeout you’ve noticed. Yes, I did an overall debug, but they may not be performance issues as I’m turning for tests the HA on my Core i7-9750H 16GB RAM laptop. Sorry I can’t do the tests for a longer period because I’m switching off my main HA installation to perform the tests. I really hope someone else do some tests and we will enjoy your version in the future HA release.

Edit, It was only the first recording that wasn’t detected, it seams OK now! image

My installation is in docker on an old MSI wind laptop, but I can install it for test the python version in a debian VM on my PC

Hi everybody,

Please find below logs with the following options: 4 minutes scan_interval 30 seconds timeout

2021-02-08 01:03:55 DEBUG (MainThread) [homeassistant.components.synology_dsm] SynoAPI.async_update()
2021-02-08 01:03:59 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_central data in 4.706 seconds
2021-02-08 01:04:00 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_cameras data in 0.923 seconds
2021-02-08 01:04:03 DEBUG (SyncWorker_3) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:04:03 DEBUG (SyncWorker_0) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:04:13 DEBUG (SyncWorker_2) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:04:13 DEBUG (SyncWorker_6) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:04:23 DEBUG (SyncWorker_1) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:04:23 DEBUG (SyncWorker_6) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:04:23 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_switches data in 0.261 seconds
2021-02-08 01:04:30 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_cameras data in 0.920 seconds
2021-02-08 01:04:33 DEBUG (SyncWorker_2) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:04:33 DEBUG (SyncWorker_0) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:04:43 DEBUG (SyncWorker_1) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:04:43 DEBUG (SyncWorker_3) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:04:53 DEBUG (SyncWorker_2) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:04:53 DEBUG (SyncWorker_4) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:04:53 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_switches data in 0.148 seconds
2021-02-08 01:05:00 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_cameras data in 0.875 seconds
2021-02-08 01:05:03 DEBUG (SyncWorker_5) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:05:03 DEBUG (SyncWorker_6) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:05:13 DEBUG (SyncWorker_2) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:05:13 DEBUG (SyncWorker_1) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:05:23 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_switches data in 0.126 seconds
2021-02-08 01:05:23 DEBUG (SyncWorker_3) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:05:23 DEBUG (SyncWorker_2) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:05:30 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_cameras data in 0.928 seconds
2021-02-08 01:05:33 DEBUG (SyncWorker_1) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:05:33 DEBUG (SyncWorker_2) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:05:43 DEBUG (SyncWorker_4) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:05:43 DEBUG (SyncWorker_0) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:05:53 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_switches data in 0.096 seconds
2021-02-08 01:05:53 DEBUG (SyncWorker_1) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:05:53 DEBUG (SyncWorker_6) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:06:00 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_cameras data in 0.843 seconds
2021-02-08 01:06:03 DEBUG (SyncWorker_3) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:06:03 DEBUG (SyncWorker_0) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:06:12 DEBUG (SyncWorker_1) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:06:12 DEBUG (SyncWorker_4) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:06:13 DEBUG (SyncWorker_3) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:06:13 DEBUG (SyncWorker_5) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:06:23 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_switches data in 0.098 seconds
2021-02-08 01:06:23 DEBUG (SyncWorker_1) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:06:23 DEBUG (SyncWorker_5) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:06:30 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_cameras data in 0.893 seconds
2021-02-08 01:06:33 DEBUG (SyncWorker_3) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:06:33 DEBUG (SyncWorker_0) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:06:43 DEBUG (SyncWorker_1) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:06:43 DEBUG (SyncWorker_6) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:06:53 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_switches data in 0.140 seconds
2021-02-08 01:06:53 DEBUG (SyncWorker_3) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:06:53 DEBUG (SyncWorker_2) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:07:00 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_cameras data in 0.866 seconds
2021-02-08 01:07:03 DEBUG (SyncWorker_5) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:07:03 DEBUG (SyncWorker_6) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:07:13 DEBUG (SyncWorker_3) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:07:13 DEBUG (SyncWorker_0) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:07:23 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_switches data in 0.107 seconds
2021-02-08 01:07:23 DEBUG (SyncWorker_1) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:07:23 DEBUG (SyncWorker_0) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:07:30 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_cameras data in 0.894 seconds
2021-02-08 01:07:33 DEBUG (SyncWorker_6) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:07:33 DEBUG (SyncWorker_0) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:07:43 DEBUG (SyncWorker_4) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:07:43 DEBUG (SyncWorker_5) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:07:53 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1780PDN165410_switches data in 0.135 seconds
2021-02-08 01:07:53 DEBUG (SyncWorker_0) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2CD2463G0-IW)
2021-02-08 01:07:53 DEBUG (SyncWorker_6) [homeassistant.components.synology_dsm.camera] SynoDSMCamera.camera_image(HIKVISION - DS-2DE2A404IW-DE3)
2021-02-08 01:07:59 DEBUG (MainThread) [homeassistant.components.synology_dsm] SynoAPI.async_update()

No problem on my side since last night. Good job! Any chance to give the opportunity to reduce the 30s with an option (as per scan_interval or timeout) ?

Thanks again

Go, go,go to prod. You can always adjust something if needed later! Thank you for your great work Michael.

Yeah, you are right - the unavailableis caused by the timeout, but will be back to available when next Finished fetching 1660NNN173914_cameras was successful again.

2021-02-07 16:28:57 ERROR (MainThread) [homeassistant.components.synology_dsm] Timeout fetching 1660NNN173914_cameras data
2021-02-07 16:28:57 DEBUG (MainThread) [homeassistant.components.synology_dsm] Finished fetching 1660NNN173914_cameras data in 10.001 seconds

OK, an i7-9750H should be powerful enough 😄


Depending on this and your previously noted results I assume it works now as designed

In general I have better results as yesterday with one exception that the state change wasn’t update at all: 9,9,27,1,2,not at all, 29, 19

I will check again tonight. Regarding the measures, I trigger the timer once a recording detected in Synology and stop it when the state changes on the camera view in HA. It’s not 100% accurate but, let’s say there may be 1sec error. If somebody else can check, that would be great.

Hi @akunia please replace the branch name by synology-dsm_fix-service

Thanks for update @mib1185 . Good luck 😉

Hi,

the synology_dsm integration needs a little bit more love to change the internal mechanic for fetching data (see https://github.com/home-assistant/core/pull/43683#issuecomment-766385217). @akunia the error you are getting now, are related to the unfinished rebuild of the internal mechanic for fetching data.

I’m looking forward, that my personal life and my job does not exhaust me to much this week, so that I can finish the rebuild next weekend. 🤞

Hi @mib1185, Thanks for he update, this is good news!

With the news fix : as far as I can tell, the only thing I notice is that camera entities are still updated every 5 minutes regardless of the scan_interval parameter. However, home_mode and camera entities are updated in HA within seconds after state changes in Synology Surveillance Station. Nothing in the logs (despite some due to time out, corrected after setting the time_out parameter to 30sec). This fix looks promising !!

Do not hesitate if you have any specific tasks you want me to do for testing. I am still monitoring anyway, and will keep you informed of any problem that could appear. Good job 👍

Hi all,

based on #43683 f469caf I have created new docker images containing improvements for camera data update. They can be found here https://hub.docker.com/r/mib85/homeassistant

It would be great, if some of you could test it on your environment.

thanks in advanced.

Hi @akunia unless the issue with throwing tracebacks like

File "/home/debian/homeassistant/lib/python3.8/site-packages/synology_dsm/api/surveillance_station/init.py", line 34, in update
self._cameras_by_id[camera_id].update_motion_detection(
KeyError: 19

File "/home/debian/homeassistant/lib/python3.8/site-packages/synology_dsm/api/surveillance_station/init.py", line 34, in update
self._cameras_by_id[camera_id].update_motion_detection(
KeyError: 17

are not solved, I would not recommend that the pending pull request should be merged.

@Quentame do you have the possibility to have a look on this or do know some dev how has the possibility to have a look into this issue?

Hi @akunia

at the moment, @Shkrum31 is testing the bugfix #43683 for me, because I do not have any real cameras available. After the tests are successful, the bugfix should be merged and become released in one of the next versions.

Hi @akunia

Thank you very much for proposing your support for testing 😃

I can test but sorry I don’t know how.

It depends on how your current HA is installed (HassOS, Docker or python venv) and if have the possibilities to install HA from source following this instructions https://www.home-assistant.io/docs/installation/virtualenv/#install. But replace under 4. Install Home Assistant the line python3 -m pip install homeassistant with pip3 install --upgrade git+git://github.com/mib1185/homeassistant-core.git@synology-dsm_fix-camera-update afterwards you need to setup the Synology DSM integration again on this HA instance.

But always … Backup your current running HA installation, before you are playing around with dev versions

Hi @mib1185,

Thanks a lot for taking the time to help solve this problem.

* Was there a version before without having this issue?

I think that the camera entities were properly updated with the previous synology integration which is now deprecated. I am sorry I cannot help about the new synology_dsm integration as no recording had been previously triggered before I noticed the problem…

* Are there something strange entries shown in home-assistant.log?

Unfortunately no…

* Is the issue always reproduceable?

Yes: as long as I do not reload the integration, the states of camera entities never change, even if the entities are updated.

* Which DSM version and model you are using?

DS918+ DSM 6.2.3-25426 Update 2 Surveillance Station 8.2.8-6335 Device Pack 5.8.1-3056

* Please could you increase the timeout from 10 to 30 seconds and try again to reproduce the issue?

No difference in behaviour after changing the Timeout parameter.

In order to try to help you better understand the problem, I built the following sensor templates, to log when entities change:

  • {{ states.sensor.synology_cpu_load_1_min.last_updated }}
  • {{ states.sensor.synology_memory_available_swap.last_updated}}
  • {{ states.sensor.synology_network_up.last_updated}}
  • {{ states.camera.synology_hikvision_ds_2de2a404iw_de3.last_updated }}
  • {{ states.switch.synology_home_mode.last_updated }}
  • {{ states.sensor.synology_drive_1_temperature.last_updated }}

Please find below my findings:

  • The entities linked with cpu, memory, network, and drives seem to update correctly according to the scan_interval parameter, if, and only if, the entity changes its state. There is a perfect sync: if several states change, they change together.
  • The switch.synology_home_mode entity updates do not depend on the scan_interval parameter. It is completely out sync compare to other entities (i.e. the state changes just after any change within Surveillance Station).
  • The camera.synology_hikvision_ds_2de2a404iw_de3 entity is always updated every 5 minutes (irrespective of the scan_interval parameter), and I do not understand why. Indeed, the Last updated field associated with this entity changes while the associated attributes and state stay identical (irrespective of camera state within Surveillance Station) . The only way to make the state change in accordance with Surveillance Station is by reloading the integration. It looks like the process used to update camera states is not the same used to initialize the integration.