jarvis: Microsoft Azure API is not working

Description

Hi, I can not speak french but i managed to translate some of the french issues about Azure and tried to implement the Azure Speech API.

• I am able to call jarvis via Snowboy
• I can say my command (in german)
• Azure understands it and replys a json

But Jarvis can not work with its reply

What am I doing wrong?

What jarvis tells me

“Hilfe” means Help and I tried to open a help dialog

~/jarvis/stt_engines/bing $ jarvis

------------ Config ------------
jv_branch            master
jv_version
jv_arch              armv7l
jv_os_name           raspbian
jv_os_version        9
language             de_DE
play_hw              hw:0,1
rec_hw               hw:1,0
speaker              ALSA
microphone           C-Media Electronics, Inc.
recorder             sox
trigger_stt          snowboy
command_stt          bing
tts_engine           svox_pico
--------------------------------

DEBUG: program_startup hook
DEBUG: start_speaking hook "Hallo! Guten Tag!"
neo: Hallo! Guten Tag!
DEBUG: stop_speaking hook
Use "?" to display possible commands (in keyboard mode)
neo: Waiting to hear 'neo'
Stroke: (listening...)
DEBUG: models=neo
INFO:snowboy:Ticks: [2, 20, 5, -1]
INFO:snowboy:Keyword 1 detected at time: 2018-12-07 16:29:02
INFO:snowboy:Ticks status: 2 1 0 1
DEBUG: modelid=0
neo
DEBUG: entering_cmd hook
DEBUG: start_speaking hook "Ja?"
neo: Ja?
DEBUG: stop_speaking hook
Stroke: (listening...)
DEBUG: goin in bin_STT()
DEBUG: start_listening hook
utils/timeout.sh 10 rec -V1 -q -r 16000 -c 1 -b 16 -e signed-integer --endian little /dev/shm/jarvis-record.wav gain 15 silence 1 0.1 2.1% 1 0.5 2.1% pad 0.5 0.5 trim 0 5
DEBUG: speech duration was 25 (10 = 1 sec)
DEBUG: stop_listening hook
DEBUG: curl https://westeurope.stt.speech.microsoft.com/speech/recognition/conversation/cognitiveservices/v1?language=de-DE&format=detailed
DEBUG: json={"RecognitionStatus":"Success","Offset":8500000,"Duration":8100000,"NBest":[{"Confidence":0.90541613101959229,"Lexical":"hilfe","ITN":"Hilfe","MaskedITN":"Hilfe","Display":"Hilfe."}]}
DEBUG: status=Success
DEBUG: forder=/dev/shm/jarvis-order
DEBUG: path=Hilfe
stt_engines/bing/main.sh: line 69: Hilfe: command not found
(listening...)
DEBUG: goin in bin_STT()
DEBUG: start_listening hook
utils/timeout.sh 10 rec -V1 -q -r 16000 -c 1 -b 16 -e signed-integer --endian little /dev/shm/jarvis-record.wav gain 15 silence 1 0.1 2.1% 1 0.5 2.1% pad 0.5 0.5 trim 0 5
DEBUG: speech duration was 17 (10 = 1 sec)
DEBUG: stop_listening hook
DEBUG: curl https://westeurope.stt.speech.microsoft.com/speech/recognition/conversation/cognitiveservices/v1?language=de-DE&format=detailed
DEBUG: json={"RecognitionStatus":"InitialSilenceTimeout","Offset":50000000,"Duration":0}
DEBUG: status=InitialSilenceTimeout
?(listening...)
DEBUG: goin in bin_STT()
DEBUG: start_listening hook
utils/timeout.sh 10 rec -V1 -q -r 16000 -c 1 -b 16 -e signed-integer --endian little /dev/shm/jarvis-record.wav gain 15 silence 1 0.1 2.1% 1 0.5 2.1% pad 0.5 0.5 trim 0 5
DEBUG: speech duration was 12 (10 = 1 sec)
DEBUG: stop_listening hook
DEBUG: curl https://westeurope.stt.speech.microsoft.com/speech/recognition/conversation/cognitiveservices/v1?language=de-DE&format=detailed
DEBUG: json={"RecognitionStatus":"InitialSilenceTimeout","Offset":50000000,"Duration":0}
DEBUG: status=InitialSilenceTimeout
?DEBUG: 3 attempts failed, end of conversation

DEBUG: exiting_cmd hook
neo: Waiting to hear 'neo'
Stroke: (listening...)
DEBUG: models=neo
INFO:snowboy:Ticks: [2, 20, 5, -1]
INFO:snowboy:Keyword 1 detected at time: 2018-12-07 16:29:21
INFO:snowboy:Keyword 1 detected at time: 2018-12-07 16:29:28
^C
DEBUG: program_exit hook

What my jarvis/stt_engines/bin/main.sh file looks like

#!/bin/bash
touch /tmp/jarvis_bing_token # initiate if don't exist
touch /tmp/jarvis_bing_expires # initiate if don't exist

_bing_transcribe () {
    if [ -z "$bing_speech_api_key" ]; then
        echo "" # new line
        jv_error "ERROR: missing bing speech api key"
        jv_warning "HELP: define bing key in Settings > Voice recognition"
        exit 1 # TODO doesn't really exit because launched with & forjv_spinner
    fi
    
    stt_bing_token="`cat /tmp/jarvis_bing_token`"
    stt_bing_expires="`cat /tmp/jarvis_bing_expires`"
    
    if [ -z "$stt_bing_expires" ] || [ "$stt_bing_expires" -lt "`date +%s`" ]; then
        $verbose && jv_debug "DEBUG: token missing or expired"
        # https://github.com/alexylem/jarvis/issues/145
        local json=`curl -v -X POST "https://westeurope.api.cognitive.microsoft.com/sts/v1.0/issueToken" \
            -H "Content-Type: application/x-www-form-urlencoded" \
            -H "Content-Length: 0" \
            -H "Ocp-Apim-Subscription-Key: $bing_speech_api_key" \
            --silent`
        $verbose && jv_debug "DEBUG: json=$json"
        
        local error=`echo $json | perl -lne 'print $1 if m{"message": "([^"]*)"}'`
        if [ -n "$error" ]; then
            jv_error "ERROR: $error"
            exit 1
        fi
        stt_bing_token="$json"
        echo $stt_bing_token > /tmp/jarvis_bing_token
        
        # TODO expiration date not provided anymore by bing, what to use?
        #local expires_in=`echo $json | perl -lne 'print $1 if m{"expires_in":"([^"]*)"}'`
        local expires_in=$(( 10 * 60 )) # 10 mins
        stt_bing_expires=`echo $(( $(date +%s) + $expires_in - 10 ))` # -10 to compensate webservice call duration
        echo $stt_bing_expires > /tmp/jarvis_bing_expires
        $verbose && jv_debug "DEBUG: token will expire in $(( $stt_bing_expires - `date +%s` )) seconds"
    fi

    [[ $OSTYPE = darwin* ]] && uuid=$(uuidgen) || uuid=$(cat /proc/sys/kernel/random/uuid)
    
    local request="https://westeurope.stt.speech.microsoft.com/speech/recognition/conversation/cognitiveservices/v1?"
    request+="language=de-DE"
    request+="&format=detailed"
    
    $verbose && jv_debug "DEBUG: curl $request"
    # don't use local or else $? will not work
    json=`curl "$request" \
	-H "Accept: application/json" \
	-H "Content-Type: audio/wav; codec=audio/pcm; samplerate=16000" \
	-H "Ocp-Apim-Subscription-Key: $bing_speech_api_key" \
        -H "Host: westeurope.stt.speech.microsoft.com" \
	-H "Transfer-Encoding: chunked" \
        --data-binary "@$audiofile" \
        --silent --fail`
    if (( $? )); then
        jv_error "ERROR: bing recognition curl failed"
        exit 1
    fi
    $verbose && jv_debug "DEBUG: json=$json"
    local status=`echo $json | perl -lne 'print $1 if m{"RecognitionStatus":"([^"]*)"}'`
    $verbose && jv_debug "DEBUG: status=$status"
    if [ "$status" = "Success" ]; then
	$verbose && jv_debug "DEBUG: forder=$forder"
	local path=`echo $json | perl -lne 'print $1 if m{"ITN":"([^"]*)"}'`
        $verbose && jv_debug "DEBUG: path=$path"
	$path > $forder
    fi
}

# Internal: Speech To Text function for Bing
# Return value: none
# Return code: 0 if success
bing_STT () {
    $verbose && jv_debug "DEBUG: goin in bin_STT()"
    LISTEN $audiofile || return $?
    _bing_transcribe &
   jv_spinner $!
   return $? # return code of _bing_transcribe
}

Thanks for helping me!!