mqtt-io: High CPU load in polling mode (40-60%)

Describe the bug

I’m running mqtt-io on Amlogic A113X in polling mode (CPU does not support interrupts) and it take around 40-60% CPU load

Expected behavior

Expect CPU load around 1-2%

Config

mqtt:
  host: x.x.x.x
  user: xxxx
  password: xxxx
  topic_prefix: mqtt-io
  ha_discovery:
    enabled: yes

# GPIO
gpio_modules:
    - name: gpio
      module: gpiod

digital_inputs:
    - name: button
      module: gpio
      pin: 10
      poll_interval: 0.1
      ha_discovery:
           component: binary_sensor
           name: GPIO Button

    - name: input1
      module: gpio
      pin: 46
      poll_interval: 0.1
      ha_discovery:
           component: binary_sensor
           name: GPIO Input1
    - name: input2
      module: gpio
      pin: 45
      poll_interval: 0.1
      ha_discovery:
           component: binary_sensor
           name: GPIO Input2
    - name: input3
      module: gpio
      pin: 44
      poll_interval: 0.1
      ha_discovery:
           component: binary_sensor
           name: GPIO Input3
    - name: input4
      module: gpio
      pin: 43
      poll_interval: 0.1
      ha_discovery:
           component: binary_sensor
           name: GPIO Input4

digital_outputs:
    - name: output1
      module: gpio
      pin: 30
      initial: low
      publish_initial: true
      ha_discovery:
          component: switch
          name: GPIO Output1
    - name: output2
      module: gpio
      pin: 29
      initial: low
      publish_initial: true
      ha_discovery:
          component: switch
          name: GPIO Output2
    - name: output3
      module: gpio
      pin: 28
      initial: low
      publish_initial: true
      ha_discovery:
          component: switch
          name: GPIO Output3

Hardware

Platform: Amlogic A113X (ARM Cortex-A53) quad-core up to 1.5GHz
Connected hardware: none

System:

OS: Armbian aarch64
Python version: Python 3.8.10
User you’re running as: root
Using a virtualenv?: no

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 31 (4 by maintainers)

Most upvoted comments

While not ideal, restarting (the service) is an effective workaround. (Use cron or systemd’s RuntimeMaxSec)

markferry on Dec 24, 2022

Looks good. Nice work!

markferry on Jan 16, 2023

I have been doing a bit of profiling and debugging, it seems that each time a value is read, a ‘transient_task’ is created in the mqtt_io/server.py.

As far as I can see, those tasks should be removed when done but it’s not happening.

https://github.com/flyte/mqtt-io/blob/f48e5b1207671cda92def18b2df5d776d3bd68f5/mqtt_io/server.py#L1075

In the above function, the for loop is taking more and more time as the transient_task list grows. I don’t know why self.transient_tasks.remove(task) isn’t called. More debugging is needed.

gmsoft-tuxicoman on Jan 13, 2023

I can cut out part of the code for sensor, for few days to check it.

piotruscilko on Jan 10, 2023

@markferry : I gave the suggestion to reboot a thumbs down, just because it does not address the main issue. I appreciate your comments though.

I am still hoping someone might have an idea what is eating the RAM. Perhaps it is indeed the garbage collector which is confused, but I am not experienced with Python to track this down or follow up this idea.

Torxgewinde on Dec 24, 2022