diffusers: Stable Cascade Diffusers not working with FP16
You know that there is a demo released here : https://huggingface.co/spaces/multimodalart/stable-cascade
I am using it locally. Works perfectly fine
But I have to change float type to float16 from bfloat16 on Kaggle
That is where error starts
\venv\lib\site-packages\diffusers\utils\pil_utils.py:43: RuntimeWarning: invalid value encountered in cast
images = (images * 255).round().astype("uint8")
I tested on my local computer and fp16 fails - bf16 works
I tested float32 on Kaggle and works but ultra slow
the error causing file : https://github.com/kashif/diffusers/commit/d67eba0f3188a23d12c3e64cae19689b87881295#commitcomment-138598496
About this issue
- Original URL
- State: open
- Created 5 months ago
- Reactions: 1
- Comments: 15 (6 by maintainers)
The model is trained using BF16 and saved as BF16. Training on BF16 can cause the model to go outside of FP16’s range and cause invalid numbers / NaNs / black images when used with FP16. And this is the case with Stable Cascade Prior and SDXL Base VAE.
yes @FurkanGozukara i will double-check with the authors on the best course of action here
Prior model itself is borked again just like the SDXL VAE. StabilityAI releases doesn’t surprise me with borked FP16 anymore.