F5-TTS: Error in inference. Audio output with no content, all silence

warnings.warn( You have passed task=transcribe, but also have set forced_decoder_ids to [[1, None], [2, 50360]] which creates a conflict. forced_decoder_ids will be ignored in favor of

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 51 (17 by maintainers)

Commits related to this issue

Most upvoted comments

Good news 😃 it finally worked for me …

@hongyu2024 so you have already tried dtype=torch.float32 ?

Thank you so much! this is effective, the sound is output(cli & gradio)

A81792EF-8BDC-4122-B969-DE5E7620F45B

i used this ā˜ā€¦and other edit suggestions mentioned here i will provide screenshot to all what i did on utils_infer.py file:

ioho kguik kughoku

really not sure which one exactly that made it works , but it worth knowing that

@hongyu2024 so you have already tried dtype=torch.float32 ?

Thank you so much! this is effective, the sound is output(cli & gradio)

A81792EF-8BDC-4122-B969-DE5E7620F45B

Good news 😃 it finally worked for me …

@hongyu2024 so you have already tried dtype=torch.float32 ?

Thank you so much! this is effective, the sound is output(cli & gradio) A81792EF-8BDC-4122-B969-DE5E7620F45B

i used this ā˜ā€¦and other edit suggestions mentioned here i will provide screenshot to all what i did on utils_infer.py file:

ioho kguik kughoku

really not sure which one exactly that made it works , but it worth knowing that

Thanks! It works for me too,need to use three:

modify F5-TTS/src/f5_tts/infer/utils_infer.py
1态
asr_pipe = pipeline(
        "automatic-speech-recognition",
        model="openai/whisper-large-v3-turbo",
        # torch_dtype=dtype,
        device=device,
    )

2态
dtype = torch.float32 # if mel_spec_type == "bigvgan" else None

3态
audio, sr = torchaudio.load(ref_audio, backend="soundfile")

@hongyu2024 so you have already tried dtype=torch.float32 ?

Thank you a lot 😃 it works and generate an audio but it show me some errors as I mentioned below in command prompt, is anything else I need to change.

if working is fine, just ignore the warning

@281807424 @hongyu2024 @Bubarinokk @omnific9

has any one of you installed ComfyUI Locally ? kindly let me know im doing some investigation

no install

TL;DR: If you’re running it on Windows in the UI, always provide a reference text in the Advanced Settings, to make it run.

OK. I found the issue. It lies in the transcription.

In this line: https://github.com/SWivid/F5-TTS/blob/b0f482421b03e187ee7ca1893458f383e2c289d3/src/f5_tts/infer/utils_infer.py#L126

I changed whisper-large-v3-turbo to whisper-base, and I was able to get the result after a while. I suspect that when it was whisper-large-v3-turbo, it just took way longer because it’s a larger model. So when we see it ā€œstuckā€, it’s actually running the transcription. It just was taking too much time.

Somehow when running in cli this isn’t an issue, but when running in gradio it’s extremely slow.

Even when running the whisper-base, it’s still very slow, compared to what it should be (on a 4090 GPU)

Potentially some conflicts exist between gradio and the huggingface pipeline libraries on Windows? Windows is a weird system.

I have the same problem. I’m not sure if the warning itself is blocking inference, but inference is indeed blocked. Running on Windows 11

D:\github\F5-TTS\venv\lib\site-packages\transformers\models\whisper\generation_whisper.py:509: FutureWarning: The input name `inputs` is deprecated. Please make sure to use `input_features` instead.
  warnings.warn(
You have passed task=transcribe, but also have set `forced_decoder_ids` to [[1, None], [2, 50360]] which creates a conflict. `forced_decoder_ids` will be ignored in favor of task=transcribe.

image EDIT: I tried the cli. It works. Looks like problem lies in the Gradio interface.

any solution?