core: Voice Assistant inconsistently working

The problem

When my voice assistant works, it works great but then for no reason it stops working and I cant figure out why. I am really looking for a way to debug my issue as I have exhausted all my own resources to get the M5Stack Atom to consistently work as a voice assistant.

What version of Home Assistant Core has the issue?

2023.10.3

What was the last working version of Home Assistant Core?

2023.10.2

What type of installation are you running?

Home Assistant Container

Integration causing the issue

Voice Assistants / Assist

Link to integration documentation on our website

https://www.home-assistant.io/voice_control/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Whisper Logs:
INFO:__main__:Ready
INFO:wyoming_faster_whisper.handler: .  .  .  .
INFO:__main__:Ready
INFO:wyoming_faster_whisper.handler: .  .  .  .
INFO:wyoming_faster_whisper.handler:
INFO:wyoming_faster_whisper.handler:
INFO:wyoming_faster_whisper.handler:
INFO:wyoming_faster_whisper.handler:

Piper Logs:
INFO:__main__:Ready
INFO:__main__:Ready
INFO:__main__:Ready
INFO:wyoming_piper.download:Downloaded /data/en_GB-alan-low.onnx (https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_GB/alan/low/en_GB-alan-low.onnx)
INFO:wyoming_piper.download:Downloaded /data/en_GB-alan-low.onnx.json (https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/en/en_GB/alan/low/en_GB-alan-low.onnx.json)
INFO:__main__:Ready
INFO:__main__:Ready
INFO:__main__:Ready
INFO:__main__:Ready

OpenWakeWordLogs:
INFO:root:Ready
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO:root:Ready

ESPHome M5Stack Atom Logs:
[07:50:34][D][binary_sensor:036]: 'Button': Sending state ON
[07:50:34][D][binary_sensor:036]: 'Button': Sending state OFF
[07:50:34][D][voice_assistant:313]: State changed from 0 to 5
[07:50:34][D][voice_assistant:319]: Desired state set to 1
[07:50:34][D][voice_assistant:124]: microphone not running
[07:50:34][D][voice_assistant:206]: Requesting start...
[07:50:34][D][voice_assistant:313]: State changed from 5 to 6
[07:50:34][D][voice_assistant:334]: Client started, streaming microphone
[07:50:34][D][voice_assistant:313]: State changed from 6 to 1
[07:50:34][D][voice_assistant:319]: Desired state set to 7
[07:50:34][D][voice_assistant:159]: Starting Microphone
[07:50:34][D][voice_assistant:313]: State changed from 1 to 2
[07:50:34][D][esp-idf:000]: I (82418859) I2S: DMA Malloc info, datalen=blocksize=1024, dma_buf_count=4

[07:50:34][D][voice_assistant:414]: Event Type: 1
[07:50:34][D][voice_assistant:417]: Assist Pipeline running
[07:50:34][D][voice_assistant:313]: State changed from 2 to 7
[07:50:34][D][voice_assistant:414]: Event Type: 3
[07:50:34][D][voice_assistant:428]: STT Started
[07:50:34][D][light:036]: 'M5Stack Atom Echo 8a10c8' Setting:
[07:50:34][D][light:047]:   State: ON
[07:50:34][D][light:051]:   Brightness: 100%
[07:50:34][D][light:059]:   Red: 0%, Green: 0%, Blue: 100%
[07:50:34][D][light:109]:   Effect: 'Pulse'
[07:50:53][D][voice_assistant:414]: Event Type: 0
[07:50:53][E][voice_assistant:526]: Error: stt-no-text-recognized - No text recognized
[07:50:53][D][voice_assistant:408]: Signaling stop...
[07:50:53][D][voice_assistant:313]: State changed from 7 to 8
[07:50:53][D][voice_assistant:319]: Desired state set to 0
[07:50:53][D][light:036]: 'M5Stack Atom Echo 8a10c8' Setting:
[07:50:53][D][light:051]:   Brightness: 100%
[07:50:53][D][light:059]:   Red: 100%, Green: 0%, Blue: 0%
[07:50:53][D][light:109]:   Effect: 'None'
[07:50:53][D][voice_assistant:313]: State changed from 8 to 9
[07:50:53][D][esp-idf:000]: I (82437319) I2S: DMA queue destroyed

[07:50:53][D][voice_assistant:414]: Event Type: 2
[07:50:53][D][voice_assistant:496]: Assist Pipeline ended
[07:50:53][D][voice_assistant:313]: State changed from 9 to 0
[07:50:53][D][light:036]: 'M5Stack Atom Echo 8a10c8' Setting:
[07:50:53][D][light:047]:   State: OFF
[07:50:54][D][light:036]: 'M5Stack Atom Echo 8a10c8' Setting:
[08:01:11][D][binary_sensor:036]: 'Button': Sending state ON
[08:01:11][D][binary_sensor:036]: 'Button': Sending state OFF
[08:01:11][D][voice_assistant:313]: State changed from 0 to 5
[08:01:11][D][voice_assistant:319]: Desired state set to 1
[08:01:11][D][voice_assistant:124]: microphone not running
[08:01:11][D][voice_assistant:206]: Requesting start...
[08:01:11][D][voice_assistant:313]: State changed from 5 to 6
[08:01:11][D][voice_assistant:334]: Client started, streaming microphone
[08:01:11][D][voice_assistant:313]: State changed from 6 to 1
[08:01:11][D][voice_assistant:319]: Desired state set to 7
[08:01:11][D][voice_assistant:159]: Starting Microphone
[08:01:11][D][voice_assistant:313]: State changed from 1 to 2
[08:01:11][D][esp-idf:000]: I (83055709) I2S: DMA Malloc info, datalen=blocksize=1024, dma_buf_count=4

[08:01:11][D][voice_assistant:414]: Event Type: 1
[08:01:11][D][voice_assistant:417]: Assist Pipeline running
[08:01:11][D][voice_assistant:313]: State changed from 2 to 7
[08:01:11][D][voice_assistant:414]: Event Type: 3
[08:01:11][D][voice_assistant:428]: STT Started
[08:01:11][D][light:036]: 'M5Stack Atom Echo 8a10c8' Setting:
[08:01:11][D][light:047]:   State: ON
[08:01:11][D][light:051]:   Brightness: 100%
[08:01:11][D][light:059]:   Red: 0%, Green: 0%, Blue: 100%
[08:01:11][D][light:109]:   Effect: 'Pulse'
[08:01:29][D][voice_assistant:414]: Event Type: 0
[08:01:29][E][voice_assistant:526]: Error: stt-no-text-recognized - No text recognized
[08:01:29][D][voice_assistant:408]: Signaling stop...
[08:01:29][D][voice_assistant:313]: State changed from 7 to 8
[08:01:29][D][voice_assistant:319]: Desired state set to 0
[08:01:29][D][light:036]: 'M5Stack Atom Echo 8a10c8' Setting:
[08:01:29][D][light:051]:   Brightness: 100%
[08:01:29][D][light:059]:   Red: 100%, Green: 0%, Blue: 0%
[08:01:29][D][light:109]:   Effect: 'None'
[08:01:29][D][voice_assistant:313]: State changed from 8 to 9
[08:01:29][D][esp-idf:000]: I (83073911) I2S: DMA queue destroyed

[08:01:29][D][voice_assistant:414]: Event Type: 2
[08:01:29][D][voice_assistant:496]: Assist Pipeline ended
[08:01:29][D][voice_assistant:313]: State changed from 9 to 0
[08:01:29][D][light:036]: 'M5Stack Atom Echo 8a10c8' Setting:
[08:01:29][D][light:047]:   State: OFF
[08:01:30][D][light:036]: 'M5Stack Atom Echo 8a10c8' Setting:
[08:01:33][D][binary_sensor:036]: 'Button': Sending state ON
[08:01:33][D][binary_sensor:036]: 'Button': Sending state OFF
[08:01:33][D][voice_assistant:313]: State changed from 0 to 5
[08:01:33][D][voice_assistant:319]: Desired state set to 1
[08:01:33][D][voice_assistant:124]: microphone not running
[08:01:33][D][voice_assistant:206]: Requesting start...
[08:01:33][D][voice_assistant:313]: State changed from 5 to 6
[08:01:33][D][voice_assistant:334]: Client started, streaming microphone
[08:01:33][D][voice_assistant:313]: State changed from 6 to 1
[08:01:33][D][voice_assistant:319]: Desired state set to 7
[08:01:33][D][voice_assistant:159]: Starting Microphone
[08:01:33][D][voice_assistant:313]: State changed from 1 to 2
[08:01:33][D][esp-idf:000]: I (83077506) I2S: DMA Malloc info, datalen=blocksize=1024, dma_buf_count=4

[08:01:33][D][voice_assistant:414]: Event Type: 1
[08:01:33][D][voice_assistant:417]: Assist Pipeline running
[08:01:33][D][voice_assistant:313]: State changed from 2 to 7
[08:01:33][D][voice_assistant:414]: Event Type: 3
[08:01:33][D][voice_assistant:428]: STT Started
[08:01:33][D][light:036]: 'M5Stack Atom Echo 8a10c8' Setting:
[08:01:33][D][light:047]:   State: ON
[08:01:33][D][light:051]:   Brightness: 100%
[08:01:33][D][light:059]:   Red: 0%, Green: 0%, Blue: 100%
[08:01:33][D][light:109]:   Effect: 'Pulse'
[08:01:52][D][voice_assistant:414]: Event Type: 0
[08:01:52][E][voice_assistant:526]: Error: stt-no-text-recognized - No text recognized
[08:01:52][D][voice_assistant:408]: Signaling stop...
[08:01:52][D][voice_assistant:313]: State changed from 7 to 8
[08:01:52][D][voice_assistant:319]: Desired state set to 0
[08:01:52][D][light:036]: 'M5Stack Atom Echo 8a10c8' Setting:
[08:01:52][D][light:051]:   Brightness: 100%
[08:01:52][D][light:059]:   Red: 100%, Green: 0%, Blue: 0%
[08:01:52][D][light:109]:   Effect: 'None'
[08:01:52][D][voice_assistant:313]: State changed from 8 to 9
[08:01:52][D][esp-idf:000]: I (83096511) I2S: DMA queue destroyed

[08:01:52][D][voice_assistant:414]: Event Type: 2
[08:01:52][D][voice_assistant:496]: Assist Pipeline ended
[08:01:52][D][voice_assistant:313]: State changed from 9 to 0
[08:01:52][D][light:036]: 'M5Stack Atom Echo 8a10c8' Setting:
[08:01:52][D][light:047]:   State: OFF
[08:01:53][D][light:036]: 'M5Stack Atom Echo 8a10c8' Setting:

Additional information

Weather or not it is actually the case or not. The only time I got everything working was after changing select.m5stack_atom_echo_8a10c8_finished_speaking_detection from aggressive to default. When it was working, it worked extremely well. I am not sure where else to look to see logs to identify the issue.

About this issue

Original URL
State: closed
Created 9 months ago
Reactions: 2
Comments: 18 (9 by maintainers)

Most upvoted comments

When my Atom Echo devices are unresponsive to the wakeword, as they were again this morning, I can get them working again by flipping the “Use wake word” switch on the ESPHome device in HA. Sometimes I just need to get really, really close to them. It seems like maybe they progressively increase their noise filtering until they don’t hear anything at all.

rccoleman on Oct 23, 2023

I’m seeing similar behavior, detailed it here: https://github.com/home-assistant/core/issues/101925#issuecomment-1763176588

WebSpider on Oct 17, 2023

I’ve a similar problem - though I had to change mine to Relaxed. When looking at logs it appeared to finish listening far to quickly.

Now I’ve done that, it’s fine for the first few (under 5) minutes, and then it stops listening. Pressing the button seems to “reset” it and it will continue to work for another few minutes, until I press the button again.

Looking at the logs via ESPHome when it stops responding, the logs also stop until I press the button. And then it starts working again.

Could it be there’s a rogue process that’s blocking?

Also, sometimes it will successfully detect a wake word and perfectly understand the request, generate a response URL, but then not play anything on the speaker - it briefly flashes red as it does this. After this, it no longer listens until I press the button again.

jkad123 on Oct 17, 2023