TTS: multiband_melgan Vocoder Fails on Step 10000 With KeyError: 'avg_loss_0'
Describe the bug
multiband_melgan vocoder trained on en, specifically, "by_book/male/elliot_miller/pirates_of_ersatz/"
(tried another language, failed with the same error), fails on step 10000 with KeyError: ‘avg_loss_0’.
To Reproduce
Run train_multiband_melgan.py on en_US dataset.
Expected behavior
No Errors.
Logs
No response
Environment
{
"CUDA": {
"GPU": [
"Tesla T4",
"Tesla T4",
"Tesla T4",
"Tesla T4"
],
"available": true,
"version": "10.2"
},
"Packages": {
"PyTorch_debug": false,
"PyTorch_version": "1.11.0+cu102",
"TTS": "0.6.2",
"numpy": "1.21.6"
},
"System": {
"OS": "Linux",
"architecture": [
"64bit",
"ELF"
],
"processor": "x86_64",
"python": "3.7.13",
"version": "#1 SMP Tue Apr 26 20:14:22 UTC 2022"
}
}
Additional context
No response
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 16 (8 by maintainers)
haven’t checked yet and I won’t be able to at least for the next two weeks, unfortunately.
If there is a fix shoot a PR
there are two issues one in trainer target_loss is not present in keep_avg_target.avg_values
another in gan.py _log output[0] is coming as None
i have put a workaround for my usecase