gradio: [BUG Regression] Audio streaming in gradio 4.18.0+ does not work

Describe the bug

Audio streaming does not work in gradio 4.18.0 or 4.19.0 but works fine in gradio 3.50.2 and up to and including 4.17.0.

Have you searched existing issues? 🔎

  • I have searched and found no existing issues

Reproduction

Literal example with only autoplay=True added so streaming occurs without clicking anything: https://www.gradio.app/guides/reactive-interfaces#streaming-components

Note that this has a typo format="bytes" but that has no effect.

Add some wave file, I added example.

audio.zip

  1. unzip zip or edit with your own audio
  2. run demo
  3. click on first example
  4. click stream as bytes

You’ll see that streaming via bytes fails to work. The audio component is changed, but no audio appears and is not played.

import gradio as gr
from pydub import AudioSegment
from time import sleep

with gr.Blocks() as demo:
    input_audio = gr.Audio(label="Input Audio", type="filepath", format="mp3")
    with gr.Row():
        with gr.Column():
            stream_as_file_btn = gr.Button("Stream as File")
            format = gr.Radio(["wav", "mp3"], value="wav", label="Format")
            stream_as_file_output = gr.Audio(streaming=True)

            def stream_file(audio_file, format):
                audio = AudioSegment.from_file(audio_file)
                i = 0
                chunk_size = 1000
                while chunk_size * i < len(audio):
                    chunk = audio[chunk_size * i : chunk_size * (i + 1)]
                    i += 1
                    if chunk:
                        file = f"/tmp/{i}.{format}"
                        chunk.export(file, format=format)
                        yield file
                        sleep(0.5)

            stream_as_file_btn.click(
                stream_file, [input_audio, format], stream_as_file_output
            )

            gr.Examples(
                [["audio/speech.wav", "wav"], ["audio/speech.wav", "mp3"]],
                [input_audio, format],
                fn=stream_file,
                outputs=stream_as_file_output,
                cache_examples=True,
            )

        with gr.Column():
            stream_as_bytes_btn = gr.Button("Stream as Bytes")
            stream_as_bytes_output = gr.Audio(streaming=True, autoplay=True)

            def stream_bytes(audio_file):
                chunk_size = 20_000
                with open(audio_file, "rb") as f:
                    while True:
                        chunk = f.read(chunk_size)
                        if chunk:
                            yield chunk
                            sleep(1)
                        else:
                            break
            stream_as_bytes_btn.click(stream_bytes, input_audio, stream_as_bytes_output)

if __name__ == "__main__":
    demo.queue().launch()

Screenshot

gradio 4.16.0 or 4.17.0 that work:

image

gradio 4.18.0 where things stop working:

image

Logs

No response

System Info

many gradios tried.  gradio 4.18.0 and 4.19.0 and 4.19.1 all fail to work.

Severity

Blocking usage of gradio

About this issue

  • Original URL
  • State: open
  • Created 4 months ago
  • Reactions: 1
  • Comments: 22 (15 by maintainers)

Commits related to this issue

Most upvoted comments

Thanks @pseudotensor we’ll take a look. @hannahblair let’s add an e2e test for this as we’ve seen regressions around audio streaming before

Hi, the repro is at the top of this issue.

Not yet but we’ll look into it this week cc @aliabid94

FYI @hannahblair and @abidlabs you made some changes to Audio for 4.18.0