wyoming-satellite: satellite gets stuck after only 1 wake word detection

Hello,

I built wyoming-satellite and I run it this way:

/opt/wyoming-satellite# script/run --name "my satellite" --uri tcp://0.0.0.0:10700 --mic-command "arecord -D plughw:CARD=seeed8micvoicec,DEV=0 -r 16000 -c 1 -f S16_LE -t raw" --snd-command "aplay -D plughw:CARD=seeed8micvoicec,DEV=1 -r 22050 -c 1 -f S16_LE -t raw" --awake-wav sounds/awake.wav --detection-command examples/commands/detection.sh --wake-refractory-seconds 0 --debug

I’m using a custom wakeword (Google colab) and it works fine; but it only works “once” I mean:

  1. I start the satellite process with the above command line
  2. I wait for it to start
  3. Then I say the wakeword “hé chou pee”
  4. The satellite activates, plays the sound (-awake-wav) and wait for my command
  5. The command is recognized (STT --> ASR --> Intent) and gets executed
  6. The audio feedback (TTS) is sent back to the satellite and I can hear it

And that’s it. I am unable to get the wakeword activating again without killing and restarting the satellite process. After restarting; it works (for 1 time) etc…

NB: I had the same result when using standard wakewords (hey_rhasspy, hey_jarvis, etc…)

Here are DEBUG logs of my satellite:

DEBUG:root:Namespace(mic_uri=None, mic_command='arecord -D plughw:CARD=seeed8micvoicec,DEV=0 -r 16000 -c 1 -f S16_LE -t raw', mic_command_rate=16000, mic_command_width=2, mic_command_channels=1, mic_command_samples_per_chunk=1024, mic_volume_multiplier=1.0, mic_noise_suppression=0, mic_auto_gain=0, mic_seconds_to_mute_after_awake_wav=0.5, mic_no_mute_during_awake_wav=False, mic_channel_index=None, snd_uri=None, snd_command='aplay -D plughw:CARD=seeed8micvoicec,DEV=1 -r 22050 -c 1 -f S16_LE -t raw', snd_command_rate=22050, snd_command_width=2, snd_command_channels=1, snd_volume_multiplier=1.0, wake_uri=None, wake_word_name=[], wake_command=None, wake_command_rate=16000, wake_command_width=2, wake_command_channels=1, wake_refractory_seconds=0.0, vad=False, vad_threshold=0.5, vad_trigger_level=1, vad_buffer_seconds=2, vad_wake_word_timeout=5.0, event_uri=None, startup_command=None, detect_command=None, detection_command='examples/commands/detection.sh', transcript_command=None, stt_start_command=None, stt_stop_command=None, synthesize_command=None, tts_start_command=None, tts_stop_command=None, tts_played_command=None, streaming_start_command=None, streaming_stop_command=None, error_command=None, connected_command=None, disconnected_command=None, awake_wav='sounds/awake.wav', done_wav=None, uri='tcp://0.0.0.0:10700', name='my satellite', area=None, no_zeroconf=False, zeroconf_name=None, zeroconf_host=None, debug_recording_dir=None, debug=True, log_format='%(levelname)s:%(name)s:%(message)s')
INFO:root:Ready
DEBUG:root:Detected IP: 192.168.1.244
DEBUG:root:Zeroconf discovery enabled (name=6c21a2437eae, host=None)
DEBUG:root:Connecting to mic service: ['arecord', '-D', 'plughw:CARD=seeed8micvoicec,DEV=0', '-r', '16000', '-c', '1', '-f', 'S16_LE', '-t', 'raw']
DEBUG:root:Connecting to snd service: ['aplay', '-D', 'plughw:CARD=seeed8micvoicec,DEV=1', '-r', '22050', '-c', '1', '-f', 'S16_LE', '-t', 'raw']
INFO:root:Connected to services
DEBUG:root:Connected to mic service
Recording raw data 'stdin' : Signed 16 bit Little Endian, Rate 16000 Hz, Mono
DEBUG:root:Server set: 34436877217185
INFO:root:Connected to server
INFO:root:Streaming audio
DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'wake', 'end_stage': 'tts', 'restart_on_end': True, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
DEBUG:root:Ping enabled



DEBUG:root:Wake word detected
DEBUG:root:Running ['examples/commands/detection.sh']
Wake word detected: éh_chou_pee
DEBUG:root:Muting microphone for 0.8995918367346939 second(s)
DEBUG:root:Connected to snd service
Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
DEBUG:root:Unmuted microphone
DEBUG:root:Event(type='transcript', data={'text': ' Allume la cuisine !'}, payload=None)
INFO:root:Streaming audio
DEBUG:root:Event(type='synthesize', data={'text': 'Allumé', 'voice': {'name': 'fr_FR-siwis-low'}}, payload=None)
DEBUG:root:Connected to snd service
Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono

What did I do wrong please ?

Hardware:

  • Model: Seeed studio Respeaker core 2
  • Linux speaker-salon.maison.lan 4.4.138-respeaker-r0 #2 SMP Wed Jul 25 11:29:18 CST 2018 armv7l GNU/Linux
  • 4x ARMv7 Processor rev 5 (v7l)
  • 1024 system memory
  • 16GB (sdcard)

Thank you for your help 😃

Best regards

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 2
  • Comments: 21

Most upvoted comments

Hi @synesthesiam, It seems there are several people facing the “keyword spotting only work once” issue

Can we kindly ask you for some support on this please? 🙏 Best regards

Additional context:

I’m following https://www.youtube.com/watch?v=eTKgc0YDCwE&list=WL&index=54 with the same hardware:

- Raspberry Pi Zero W 2
- SeeedStudio: 2-Mic Pi Hat 

Additional logs:

I added >>> to the examples/commands to make then easier to be viewed:

Apr 18 12:41:47 pizero systemd[1]: Started Wyoming Satellite.
Apr 18 12:41:47 pizero python3[4091]: INFO:root:Ready
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Namespace(mic_uri=None, mic_command='arecord -D plughw:CARD=seeed2micvoicec,DEV=0 -r 16000 -c 1 -f S16_LE -t raw', mic_command_rate=16000, mic_command_width=2, mic_command_channels=1, mic_command_samples_per_chunk=1024, mic_volume_multiplier=1.0, mic_noise_suppression=2, mic_auto_gain=0, mic_seconds_to_mute_after_awake_wav=0.5, mic_no_mute_during_awake_wav=False, mic_channel_index=None, snd_uri=None, snd_command='aplay -D plughw:CARD=seeed2micvoicec,DEV=0 -r 22050 -c 1 -f S16_LE -t raw', snd_command_rate=22050, snd_command_width=2, snd_command_channels=1, snd_volume_multiplier=1.0, wake_uri='tcp://127.0.0.1:10400', wake_word_name=[['hey_jarvis']], wake_command=None, wake_command_rate=16000, wake_command_width=2, wake_command_channels=1, wake_refractory_seconds=5.0, vad=False, vad_threshold=0.5, vad_trigger_level=1, vad_buffer_seconds=2, vad_wake_word_timeout=5.0, event_uri='tcp://127.0.0.1:10500', startup_command='examples/commands/startup.sh', detect_command=None, detection_command='examples/commands/detection.sh', transcript_command=None, stt_start_command='examples/commands/stt_start.sh', stt_stop_command='examples/commands/stt_stop.sh', synthesize_command='examples/commands/synthesize.sh', tts_start_command='examples/commands/tts_start.sh', tts_stop_command='examples/commands/tts_stop.sh', tts_played_command=None, streaming_start_command='examples/commands/streaming_start.sh', streaming_stop_command='examples/commands/streaming_stop.sh', error_command=None, connected_command=None, disconnected_command=None, awake_wav='sounds/awake.wav', done_wav='sounds/done.wav', uri='tcp://0.0.0.0:10700', name='Alfred Satellite', area=None, no_zeroconf=False, zeroconf_name=None, zeroconf_host=None, debug_recording_dir=None, debug=True, log_format='%(levelname)s:%(name)s:%(message)s')
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Running ['examples/commands/startup.sh']
Apr 18 12:41:48 pizero run[4099]: >>> Satellite has started.
Apr 18 12:41:48 pizero run[4095]: INFO:root:Ready
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Detected IP: 192.168.42.228
Apr 18 12:41:48 pizero run[4094]: DEBUG:root:Namespace(uri='tcp://127.0.0.1:10400', models_dir=PosixPath('/home/diego/wyoming-openwakeword/wyoming_openwakeword/models'), custom_model_dir=[], preload_model=['hey_jarvis'], threshold=0.5, trigger_level=1, output_dir=None, debug=True, log_format='%(levelname)s:%(name)s:%(message)s', debug_probability=False, version=False, model=[])
Apr 18 12:41:48 pizero run[4094]: DEBUG:root:Loading hey_jarvis_v0.1 from /home/diego/wyoming-openwakeword/wyoming_openwakeword/models/hey_jarvis_v0.1.tflite
Apr 18 12:41:48 pizero run[4094]: DEBUG:wyoming_openwakeword.handler:Started thread for hey_jarvis_v0.1
Apr 18 12:41:48 pizero run[4094]: DEBUG:root:Loading /home/diego/wyoming-openwakeword/wyoming_openwakeword/models/melspectrogram.tflite
Apr 18 12:41:48 pizero run[4094]: DEBUG:root:Loading /home/diego/wyoming-openwakeword/wyoming_openwakeword/models/embedding_model.tflite
Apr 18 12:41:48 pizero run[4094]: INFO:root:Ready
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Zeroconf discovery enabled (name=2ccf6704eefa, host=None)
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Connecting to mic service: ['arecord', '-D', 'plughw:CARD=seeed2micvoicec,DEV=0', '-r', '16000', '-c', '1', '-f', 'S16_LE', '-t', 'raw']
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Connecting to snd service: ['aplay', '-D', 'plughw:CARD=seeed2micvoicec,DEV=0', '-r', '22050', '-c', '1', '-f', 'S16_LE', '-t', 'raw']
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Connecting to wake service: tcp://127.0.0.1:10400
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Connecting to event service: tcp://127.0.0.1:10500
Apr 18 12:41:48 pizero run[4095]: INFO:root:Connected to services
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Using webrtc audio enhancements
Apr 18 12:41:48 pizero run[4094]: DEBUG:wyoming_openwakeword.handler:Client connected: 14883546340095
Apr 18 12:41:48 pizero run[4105]: Recording raw data 'stdin' : Signed 16 bit Little Endian,
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Connected to mic service
Apr 18 12:41:48 pizero run[4095]: DEBUG:root:Connected to wake service
Apr 18 12:41:48 pizero run[4105]: Rate 16000 Hz, Mono
Apr 18 12:41:48 pizero run[4094]: DEBUG:root:Loading hey_jarvis_v0.1 from /home/diego/wyoming-openwakeword/wyoming_openwakeword/models/hey_jarvis_v0.1.tflite
Apr 18 12:41:48 pizero run[4094]: DEBUG:wyoming_openwakeword.handler:Started thread for hey_jarvis_v0.1
Apr 18 12:41:49 pizero run[4094]: INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Apr 18 12:41:49 pizero run[4095]: DEBUG:root:Server set: 14884328193371
Apr 18 12:41:49 pizero run[4095]: INFO:root:Connected to server
Apr 18 12:41:49 pizero run[4095]: DEBUG:root:Running ['examples/commands/streaming_stop.sh']
Apr 18 12:41:49 pizero run[4108]: >>> Audio streaming to server has stopped
Apr 18 12:41:49 pizero run[4095]: INFO:root:Waiting for wake word
Apr 18 12:41:49 pizero run[4095]: DEBUG:root:Connected to event service
Apr 18 12:41:49 pizero run[4094]: DEBUG:root:Loading hey_jarvis_v0.1 from /home/diego/wyoming-openwakeword/wyoming_openwakeword/models/hey_jarvis_v0.1.tflite
Apr 18 12:41:49 pizero run[4094]: DEBUG:wyoming_openwakeword.handler:Started thread for hey_jarvis_v0.1
Apr 18 12:41:49 pizero run[4094]: DEBUG:wyoming_openwakeword.handler:Sent info to client: 14883546340095
Apr 18 12:41:51 pizero run[4095]: DEBUG:root:Ping enabled
Apr 18 12:42:06 pizero run[4094]: DEBUG:root:Triggered hey_jarvis_v0.1 (client=14883546340095)
Apr 18 12:42:06 pizero run[4095]: DEBUG:root:Detection(name='hey_jarvis_v0.1', timestamp=14900721294606)
Apr 18 12:42:06 pizero run[4095]: DEBUG:root:Streaming audio
Apr 18 12:42:06 pizero run[4095]: DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'asr', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)
Apr 18 12:42:06 pizero run[4095]: DEBUG:root:Running ['examples/commands/detection.sh']
Apr 18 12:42:06 pizero run[4094]: DEBUG:root:Triggered hey_jarvis_v0.1 (client=14883546340095)
Apr 18 12:42:06 pizero run[4111]: >>> Wake word detected: hey_jarvis_v0.1
Apr 18 12:42:06 pizero run[4095]: DEBUG:root:Muting microphone for 0.8995918367346939 second(s)
Apr 18 12:42:06 pizero run[4095]: DEBUG:root:Running ['examples/commands/streaming_start.sh']
Apr 18 12:42:06 pizero run[4094]: DEBUG:root:Triggered hey_jarvis_v0.1 (client=14883546340095)
Apr 18 12:42:06 pizero run[4114]: >>> Audio streaming to server has started
Apr 18 12:42:06 pizero run[4095]: DEBUG:root:Connected to snd service
Apr 18 12:42:06 pizero run[4116]: Playing raw data 'stdin' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono
Apr 18 12:42:07 pizero run[4095]: DEBUG:root:Unmuted microphone

Only the called commands logs:

Apr 18 12:41:48 pizero run[4099]: >>> Satellite has started.
Apr 18 12:41:49 pizero run[4108]: >>> Audio streaming to server has stopped
Apr 18 12:42:06 pizero run[4111]: >>> Wake word detected: hey_jarvis_v0.1
Apr 18 12:42:06 pizero run[4114]: >>> Audio streaming to server has started

And this is my service definition:

[Unit]
Description=Wyoming Satellite
Wants=network-online.target
After=network-online.target
Requires=wyoming-openwakeword.service
Requires=2mic_leds.service

[Service]
Type=simple
ExecStart=/home/diego/wyoming-satellite/script/run \
        --name 'Alfred Satellite' \
        --uri 'tcp://0.0.0.0:10700' \
        --mic-command 'arecord -D plughw:CARD=seeed2micvoicec,DEV=0 -r 16000 -c 1 -f S16_LE -t raw' \
        --snd-command 'aplay -D plughw:CARD=seeed2micvoicec,DEV=0 -r 22050 -c 1 -f S16_LE -t raw' \
        --mic-auto-gain 0 \
        --mic-noise-suppression 2 \
        --mic-volume-multiplier 1.0 \
        --wake-uri 'tcp://127.0.0.1:10400' \
        --wake-word-name 'hey_jarvis' \
        --event-uri 'tcp://127.0.0.1:10500' \
        --awake-wav sounds/awake.wav \
        --done-wav sounds/done.wav \
        --tts-stop-command 'true' \
        --detection-command examples/commands/detection.sh \
        --startup-command examples/commands/startup.sh \
        --streaming-start-command examples/commands/streaming_start.sh \
        --streaming-stop-command examples/commands/streaming_stop.sh \
        --synthesize-command examples/commands/synthesize.sh \
        --stt-start-command examples/commands/stt_start.sh \
        --stt-stop-command examples/commands/stt_stop.sh \
        --tts-start-command examples/commands/tts_start.sh \
        --tts-stop-command examples/commands/tts_stop.sh \
        --debug
WorkingDirectory=/home/diego/wyoming-satellite
Restart=always


Hopefully @synesthesiam can take a look at this 🙏

Similar issue here, in debug mode it only responds to wake word one time and gets stuck. The satellite is configured in Home Assistant. I’m using most of the defaults from the tutorial since I have the same hardware wyoming-satellite v1.2.0 wyoming-openwakeword v1.10.0 Home Assistant v2024.5.4 (docker)

richard@wysat:~ $ journalctl -u wyoming-openwakeword.service -f
-- Journal begins at Mon 2024-03-11 20:13:58 CDT. --
May 18 00:47:51 wysat run[469]: DEBUG:root:Loading /home/richard/wyoming-openwakeword/wyoming_openwakeword/models/embedding_model.tflite
May 18 00:47:51 wysat run[469]: INFO:root:Ready
May 18 00:47:56 wysat run[469]: DEBUG:wyoming_openwakeword.handler:Client connected: 20644978897
May 18 00:47:56 wysat run[469]: DEBUG:root:Loading hey_jarvis_v0.1 from /home/richard/wyoming-openwakeword/wyoming_openwakeword/models/hey_jarvis_v0.1.tflite
May 18 00:47:56 wysat run[469]: DEBUG:wyoming_openwakeword.handler:Started thread for hey_jarvis_v0.1
May 18 00:47:57 wysat run[469]: INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
May 18 00:47:58 wysat run[469]: DEBUG:wyoming_openwakeword.handler:Sent info to client: 20644978897
May 18 00:48:03 wysat run[469]: DEBUG:root:Loading hey_jarvis_v0.1 from /home/richard/wyoming-openwakeword/wyoming_openwakeword/models/hey_jarvis_v0.1.tflite
May 18 00:48:03 wysat run[469]: DEBUG:wyoming_openwakeword.handler:Started thread for hey_jarvis_v0.1
May 18 00:48:03 wysat run[469]: DEBUG:wyoming_openwakeword.handler:Sent info to client: 20644978897
May 18 00:49:31 wysat run[469]: DEBUG:root:Triggered hey_jarvis_v0.1 (client=20644978897)
May 18 00:49:31 wysat run[469]: DEBUG:root:Triggered hey_jarvis_v0.1 (client=20644978897)
May 18 00:49:31 wysat run[469]: DEBUG:root:Triggered hey_jarvis_v0.1 (client=20644978897)
^C
richard@wysat:~ $ journalctl -u wyoming-satellite.service -f
-- Journal begins at Mon 2024-03-11 20:13:58 CDT. --
May 18 00:47:56 wysat run[644]: Recording raw data 'stdin' : Signed 16 bit Little Endian, Rate 16000 Hz, Mono
May 18 00:47:56 wysat run[642]: DEBUG:root:Connected to mic service
May 18 00:47:56 wysat run[642]: DEBUG:root:Connected to wake service
May 18 00:48:03 wysat run[642]: DEBUG:root:Server set: 27291338790
May 18 00:48:03 wysat run[642]: INFO:root:Connected to server
May 18 00:48:03 wysat run[642]: INFO:root:Waiting for wake word
May 18 00:48:05 wysat run[642]: DEBUG:root:Ping enabled
May 18 00:49:31 wysat run[642]: DEBUG:root:Detection(name='hey_jarvis_v0.1', timestamp=102095540476)
May 18 00:49:31 wysat run[642]: DEBUG:root:Streaming audio
May 18 00:49:31 wysat run[642]: DEBUG:root:Event(type='run-pipeline', data={'start_stage': 'asr', 'end_stage': 'tts', 'restart_on_end': False, 'snd_format': {'rate': 22050, 'width': 2, 'channels': 1}}, payload=None)

Hello, I had a similliar issue, but just getting detected once, without ever executing any commands. I managed to locate the problem with Speech To Text in homeassistant logs.

Steps I took to fix it:

  1. installing addons Piper, Whisper and openWakeWord
  2. Adding them in integrations.
  3. Changing Speech To Text option to faster-whisper (inside settings -> voice assistant)

Not sure if all of the above is needed, but the core of the issue for me was not having a Speech to text agent configured, which made the service stuck in listening mode.

#87 I suppose this could be the same issue as I had.

Hi all, I did some more tests with ESP32-S3-BOX-3 (tutorial link)

So the device is using the same Home Assistant setup than the one I used with wyoming-satellite: wyoming openwakeword (still with my custom wakeword)

Only difference is that I replaced wyoming-satellite by an SP32-S3-BOX-3

And with this setup; I’m able to trigger the wakeword several times without having to restart anything. So I think Home Assistant setup + wyoming-openwakeword are working correctly and I think there is a problem with the configuration of my wyoming-satellite. But what can be the problem ? Really need some feedback on this please 😃

NB: In case it can help: I’m using docker version of Home Assistant and wyoming-openwakeword

Thank you for your help Best regards