core: Problems retrieving videos/clips from Google Nest Camera

The problem

I’m using the Nest Integration with my Google Nest Doorbell. What I’m trying to achieve is passing the clip previews to my notifications, just like the Google Home app does today. However I’m experiencing some strange and unpredictable behaviour as some of the image previews are missing.

This is what it looks like most of the time:

And then once in a while, I actually get the preview.

One thing I discovered was that if I set a delay between the time of the event and the action (sending the notification and retrieving the image from the Nest API) most of the images appear.

So I have an assumption that there’s some kind of timing issue going on.

However as you can also see from the image below, there are still a few occurrences with some images/clip previews not being shown.

So even after a minute, they are not available.

Apart from adding a one minute delay, I’ve done exactly the same as documented here.

One of the most interesting things I’ve discovered in the log for the media that was missing is the following:

2023-01-08 12:55:00.694 DEBUG (MainThread) [google_nest_sdm.event] EventMessage raw_data={'eventId': '77c0fa90-23b3-4c99-9383-7a306cbd6bdd', 'timestamp': '2023-01-08T11:54:56.601Z', 'resourceUpdate': {'name': 'enterprises/20acf40f-33eb-4019-92be-686e464ca919/devices/AVPHwEttFask-hEGzUkEqFXAXRywmTXJC8OzVvlPp6JNKb-npc5vmk9y4EvuTPed1cjGE_oNRT4IVHafgrf9D21Ln_fMaA', 'events': {'sdm.devices.events.CameraPerson.Person': {'eventSessionId': '1633206197', 'eventId': '689036678', 'zones': ['']}}}, 'userId': 'AVPHwEt1g5AoYr8Ti49Qygg1LXO6us7Rw83iNkEgnwcQ', 'eventThreadId': '8c70fa62-3e1d-4223-8711-ac2dec050d34', 'resourceGroup': ['enterprises/20acf40f-33eb-4019-92be-686e464ca919/devices/AVPHwEttFask-hEGzUkEqFXAXRywmTXJC8OzVvlPp6JNKb-npc5vmk9y4EvuTPed1cjGE_oNRT4IVHafgrf9D21Ln_fMaA'], 'eventThreadState': 'STARTED'}
2023-01-08 12:55:00.694 DEBUG (MainThread) [google_nest_sdm.device] Processing update 77c0fa90-23b3-4c99-9383-7a306cbd6bdd @ 2023-01-08 11:54:56.601000+00:00
2023-01-08 12:55:00.694 DEBUG (MainThread) [google_nest_sdm.event_media] Event Update dict_keys(['1633206197'])
2023-01-08 12:55:00.723 DEBUG (MainThread) [google_nest_sdm.camera_traits] Ignoring event, not the right type: <ImageEventBase {'event_type': 'sdm.devices.events.CameraPerson.Person', 'event_data': {'eventSessionId': '1633206197', 'eventId': '689036678', 'zones': ['']}, 'timestamp': '2023-01-08T11:54:56.601000+00:00', 'event_image_type': 'image/jpeg'} sessions=1>

Summary

• Most of the media is saved, but it is not saved immediately, so the images are usually not available on time when attaching resulting in a 404.
• Some media never gets saved at all (like in the log above)

What version of Home Assistant Core has the issue?

2023.1.1

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Google Nest

Link to integration documentation on our website

https://www.home-assistant.io/integrations/nest/

Diagnostics information

config_entry-nest-5daa3be9b5d98b45ca1a5e46ff31630f.json.txt

Example YAML snippet

alias: Person set for dørklokke (test)
description: Skal bruges til at teste events fra dørklokke
trigger:
  - platform: device
    device_id: 34bf6f2934862c9538f6f3ace8125419
    domain: nest
    type: camera_person
condition: []
action:
  - service: notify.mobile_app_iphone_57
    data:
      message: >
        Device ID: {{ trigger.event.data.device_id }} Event ID: {{
        trigger.event.data.nest_event_id }}
      title: Der er set en person fra dørklokke kamera! (TEST)
      data:
        image: >
          /api/nest/event_media/{{ trigger.event.data.device_id }}/{{
          trigger.event.data.nest_event_id }}/thumbnail
        video: >
          /api/nest/event_media/{{ trigger.event.data.device_id }}/{{
          trigger.event.data.nest_event_id  }}
mode: single

Anything in the logs that might be useful for us?

2023-01-03 07:46:42.233 DEBUG (MainThread) [homeassistant.components.http.auth] Authenticated 195.249.79.150 for /api/nest/event_media/34bf6f2934862c9538f6f3ace8125419/WyIxNjMzMjA1OTQ4IiwgIm46MSJd/thumbnail using signed request
2023-01-03 07:46:42.234 DEBUG (MainThread) [homeassistant.components.http.view] Serving /api/nest/event_media/34bf6f2934862c9538f6f3ace8125419/WyIxNjMzMjA1OTQ4IiwgIm46MSJd/thumbnail to 195.249.79.150 (auth: True)
2023-01-03 07:46:42.246 DEBUG (SyncWorker_15) [homeassistant.components.nest.media_source] Reading event media from disk store: /config/nest/event_media/34bf6f2934862c9538f6f3ace8125419/1671638900-doorbell_chime_thumb.gif
2023-01-03 07:46:42.307 DEBUG (MainThread) [homeassistant.components.http.auth] Authenticated 195.249.79.150 for /api/nest/event_media/34bf6f2934862c9538f6f3ace8125419/WyIxNjMzMjA1OTQ4IiwgIm46MSJd/thumbnail using signed request
2023-01-03 07:46:42.307 DEBUG (MainThread) [homeassistant.components.http.view] Serving /api/nest/event_media/34bf6f2934862c9538f6f3ace8125419/WyIxNjMzMjA1OTQ4IiwgIm46MSJd/thumbnail to 195.249.79.150 (auth: True)
2023-01-03 07:46:42.310 DEBUG (SyncWorker_4) [homeassistant.components.nest.media_source] Reading event media from disk store: /config/nest/event_media/34bf6f2934862c9538f6f3ace8125419/1671638900-doorbell_chime_thumb.gif
2023-01-03 07:46:42.714 DEBUG (Thread-14 (_run)) [google.cloud.pubsub_v1.subscriber._protocol.streaming_pull_manager] Observed non-terminating stream error 503 The StreamingPull stream closed for an expected reason and should be recreated, which is done automatically if using Cloud Pub/Sub client libraries. Refer to https://cloud.google.com/pubsub/docs/pull#streamingpull for more information.
2023-01-03 07:46:42.714 DEBUG (Thread-14 (_run)) [google.cloud.pubsub_v1.subscriber._protocol.streaming_pull_manager] Observed recoverable stream error 503 The StreamingPull stream closed for an expected reason and should be recreated, which is done automatically if using Cloud Pub/Sub client libraries. Refer to https://cloud.google.com/pubsub/docs/pull#streamingpull for more information.
2023-01-03 07:46:42.717 INFO (Thread-14 (_run)) [google.api_core.bidi] Re-established stream

...

2023-01-08 12:56:00.742 INFO (MainThread) [homeassistant.components.automation.person_set_for_dorklokke_test] Person set for dørklokke (test): Executing step call service
2023-01-08 12:56:01.170 INFO (MainThread) [homeassistant.components.mobile_app.notify] mobile_app push notification rate limits for iPhone (57): 2 sent, 500 allowed, 0 errors, resets in 12:03:58
2023-01-08 12:56:02.101 DEBUG (Thread-LeaseMaintainer) [google.cloud.pubsub_v1.subscriber._protocol.leaser] The current deadline value is 10 seconds.
2023-01-08 12:56:02.102 DEBUG (Thread-LeaseMaintainer) [google.cloud.pubsub_v1.subscriber._protocol.leaser] Snoozing lease management for 4.392832 seconds.
2023-01-08 12:56:04.591 DEBUG (Thread-Heartbeater) [google.cloud.pubsub_v1.subscriber._protocol.heartbeater] Sent heartbeat.
2023-01-08 12:56:05.653 DEBUG (MainThread) [homeassistant.components.http.auth] Authenticated 87.49.147.137 for /api/nest/event_media/34bf6f2934862c9538f6f3ace8125419/WyIxNjMzMjA2MTk3IiwgIjY4OTAzNjY3OCJd using bearer token
2023-01-08 12:56:05.653 DEBUG (MainThread) [homeassistant.components.http.view] Serving /api/nest/event_media/34bf6f2934862c9538f6f3ace8125419/WyIxNjMzMjA2MTk3IiwgIjY4OTAzNjY3OCJd to 87.49.147.137 (auth: True)
2023-01-08 12:56:05.662 DEBUG (MainThread) [google_nest_sdm.event_media] No persisted media for event id <EventToken event_session_id1633206197 event_id=689036678>
2023-01-08 12:56:05.662 DEBUG (MainThread) [homeassistant.components.nest] No event found for event_id 'WyIxNjMzMjA2MTk3IiwgIjY4OTAzNjY3OCJd'

Additional information

We started the discussion here, which resulted in this issue: https://community.home-assistant.io/t/nest-doorbell-battery-does-not-store-all-videos-into-media-folder/506514/16