core: SSDP memory leak

The problem

I’m running the latest version of HA. Somewhere after the start of the new year the system started crashing about every 3 days. Just recently I connected a console port and was able to do a little looking around. All memory on the system was in use. I have a small swap partition and so the system was thrashing. Anyway for the last couple of weeks I’ve been trying to isolate the issue. I did the disable custom features, then integration and then individual devices and entities. Nothing worked. I set up three copies of my system to aid in the testing. I run supervised version of HA. One system was the raspberry pi debian 11 release, one was armbian debian 11 release and one was armbian debian 10.

Anyway I recently found the profile integration and did some samples of the memory on these systems for multiple days. This thing: async_upnp_client.utils.CaseInsensitiveDict along with dict (no owner) was pretty consistently at the top of the list.

I figured upnp runs on port 1900 so I’d just add an iptables rule to block all outgoing port 1900 request on one of my systems. Much to my delight the system stopped leaking memory. On a second system I remove default_config and then added in just the items I wanted included that were previously covered by default_config. So I did not include ssdp.

Once again this stopped the memory leak. I’ve only run the test removing ssdp for 24 hours, but considering memory usage has been stable around 23% for python3 -m homeassistant --config /config. Memory was never stable, consumption was always climbing prior to this change. If someone has an update they’d like me to test out on one of my setup I’m more than willing to help out. If there are clear instruction somewhere on turning one of these systems into a development enviroment

What version of Home Assistant Core has the issue?

core-2022.2.9

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Supervised

Integration causing the issue

ssdp

Link to integration documentation on our website

https://www.home-assistant.io/integrations/default_config/

Diagnostics information

.loader: _load_stat
.format: SetFormat
.timemade: 1645577282.874777
.count: 3952
.size: 557711
.kindname: 
.kindheader: Referrers by Kind (class / dict of class)
.numrows: 184
.r: 1321 99490 dict (no owner)
.r: 114 72960 dict of async_upnp_client.utils.CaseInsensitiveDict
.r: 130 62528 types.CoroutineType
.r: 261 45600 function
.r: 60 45082 collections.deque
.r: 227 23239 types.FrameType
.r: 21 21927 functools._lru_cache_wrapper
.r: 26 16640 sqlalchemy.orm.state.InstanceState
.r: 11 12936 dict (no owner), dict of kasa.smartplug.SmartPlug
.r: 255 11570 tuple
.r: 91 9984 dict of sqlalchemy.orm.state.InstanceState
.r: 65 9588 homeassistant.core.State
.r: 194 8924 asyncio.events.TimerHandle
.r: 13 8320 homeassistant.components.recorder.models.States
.r: 79 6936 types.CellType
.r: 44 5984 asyncio.events.TimerHandle, types.CellType
.r: 44 4928 list, types.CellType
.r: 13 4680 homeassistant.components.recorder.models.Events
.r: 55 4644 _asyncio.Task
.r: 19 4408 homeassistant.core.Event
.r: 19 4408 types.MappingProxyType
.r: 54 3374 homeassistant.components.mqtt.models.ReceiveMessage
.r: 30 2554 dict of homeassistant.components.recorder.models.States
.r: 11 2552 dict of kasa.smartplug.SmartPlug
.r: 41 2490 zeroconf._dns.DNSText
.r: 14 2482 <Nothing>
.r: 11 2200 builtins.TaskWakeupMethWrapper
.r: 10 2000 builtins.TaskWakeupMethWrapper, types.FrameType
.r: 3 1872 dict of asyncio.locks.Event
.r: 4 1856 types.GeneratorType
.r: 11 1760 _asyncio.FutureIter, _asyncio.Task, tuple, types.FrameType

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 58 (18 by maintainers)

Most upvoted comments

So the Discovered records show finding the mythtv DVR and the orbi router. grep Discovered home-assistant.log 2022-02-27 08:43:41 DEBUG (MainThread) [homeassistant.components.ssdp] Discovered upnp at http://192.168.10.252:56688/rootDesc.xml 2022-02-27 08:43:42 DEBUG (MainThread) [homeassistant.components.ssdp] Discovered dlna_dmr at http://192.168.10.3:6547/getDeviceDesc 2022-02-27 08:43:45 DEBUG (MainThread) [homeassistant.components.ssdp] Discovered dlna_dmr at http://192.168.10.3:6547/getDeviceDesc 2022-02-27 08:45:43 DEBUG (MainThread) [homeassistant.components.ssdp] Discovered dlna_dmr at http://192.168.10.3:6547/getDeviceDesc

Using your tool I see 12 unique UDNs. I did 4 runs of the command and they stayed the same. The highest value for max-age is 3600. I’m guessing that’s 5 minutes. Right now memory seems to be stable. I’ll give you an update on Tuesday.

It just hit me I should run a test while I’m away to see if I see the memory growth while I’m away. I still had the test HA set up and running were I stopped the memory growth simply by placing an iptables rule in place for port 1900. I removed the rule. Right now the device that are responding are my orbi router and the HDHomerun tuner. I’m using tcpdump to look for the responses. The orbi router responds with 22 UDP packets while the HDHomerun responds with 204 UDP packets. I’ll let you know if I see the memory growth.