F5-TTS: [Finetune Exploration] KeyError: 'model_state_dict' when using test_train.py
Hey guys, appreciate the project, the model has been great so far!
So I’ve set up a dataset to be compatible with dataset.py, set the parameters inside of the test_train.py, and on trying to load F5TTS_Base, it seems like it’s missing some keys:
Exception has occurred: KeyError
'model_state_dict'
File "C:\Users\jarod\OneDrive\Desktop\code\F5-TTS\model\trainer.py", line 154, in load_checkpoint
self.accelerator.unwrap_model(self.model).load_state_dict(checkpoint['model_state_dict'])
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "C:\Users\jarod\OneDrive\Desktop\code\F5-TTS\model\trainer.py", line 240, in train
start_step = self.load_checkpoint()
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\jarod\OneDrive\Desktop\code\F5-TTS\test_train.py", line 85, in main
trainer.train(train_dataset,
File "C:\Users\jarod\OneDrive\Desktop\code\F5-TTS\test_train.py", line 92, in <module>
main()
KeyError: 'model_state_dict'
Were the pretrained models distilled down to remove these keys? If so, got any tips for loading it up into further training?
I played around with just trying to load ema_model_state_dict, but it seems like I ended up just starting from scratch with this.
Related to https://github.com/SWivid/F5-TTS/issues/16
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 15 (7 by maintainers)
@JarodMica error with keys mismatch solved. Previously we replaced the prefix of keys,
ema_model.withmodel., though it should just be removed. We fix this in 46d391a8766d8eabb967b326ccb473bc2d8b9a6c, and we did an actual run to make sure it is correct now.Thought it was updated, but still dominated by previous updates, say pretraining, as the finetune process just goes few updates.
@JarodMica we currently deal it with 0d7b47bc3b7064ce1bdb4688392c9b907e4fbf3f Give it a try~
We just missed that. Thanks for mentioning this ~
I will test on it, but place the loading in init as we’re using
accelerator, cuzemais only recorded at main_process of ddpThanks for the response!
So I made the modifications:
And training seems to be running.
I did set the model to save just to try it out and make sure it’s still working, so after 100 steps, the model still outputs comprehensible speech so I think we’re good here