ml-stable-diffusion: Control Net image generation command fails
I converted the control net mlsd model and then ran the generation command as
python -m python_coreml_stable_diffusion.pipeline --prompt "a cartoon rendition of a book" -i /Users/_ImageSynthesis/coreml_stable_diffusion_out_ml_packages_sd-controlnet-mlsd -o /Users/verapurv/Pictures/coreml_stable_diffusion_out_ml_packages --compute-unit ALL --seed 1729 --model-version runwayml/stable-diffusion-v1-5 --controlnet lllyasviel/sd-controlnet-mlsd --controlnet-inputs ~/Pictures/CompressJPEG.online_512x512_image.png
but I see the following error
eml_stable_diffusion_out_ml_packages_sd-controlnet-mlsd/Stable_Diffusion_version_runwayml_stable-diffusion-v1-5_safety_checker.mlpackage
INFO:python_coreml_stable_diffusion.coreml_model:Done. Took 1.1 seconds.
INFO:__main__:Done.
INFO:__main__:Initializing Core ML pipe for image generation
INFO:__main__:Stable Diffusion configured to generate 512x512 images
INFO:__main__:Done.
INFO:__main__:Beginning image generation.
0%| | 0/51 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/Users/verapurv/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/Users/verapurv/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/Users/verapurv/ProgramFiles/ml-stable-diffusion/python_coreml_stable_diffusion/pipeline.py", line 657, in <module>
main(args)
File "/Users/verapurv/ProgramFiles/ml-stable-diffusion/python_coreml_stable_diffusion/pipeline.py", line 577, in main
image = coreml_pipe(
File "/Users/verapurv/ProgramFiles/ml-stable-diffusion/python_coreml_stable_diffusion/pipeline.py", line 395, in __call__
noise_pred = self.unet(
File "/Users/verapurv/ProgramFiles/ml-stable-diffusion/python_coreml_stable_diffusion/coreml_model.py", line 79, in __call__
return self.model.predict(kwargs)
File "/Users/verapurv/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/coremltools/models/model.py", line 569, in predict
return self.__proxy__.predict(data)
RuntimeError: {
NSLocalizedDescription = "Error computing NN outputs.";
}
Any idea what might be going wrong?
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 20 (4 by maintainers)
Thank you for your help. I will make use of the models that you have pre-converted.
The space savings with a 6-bit ControlNet model is huge, 272 MB vs 723 MB for an uncompressed model. The 6-bit ControlNets seem to work equally well with a 6-bit base model or a FP16 base model. And the results look essentially the same to my eye when compared to using an uncompressed ControlNet model.
Thanks for the note @jrittvo! The runtime memory savings are dependent on target device software version. macOS13 will still save disk space and download size for quantized models but macOS 14 is required for runtime RAM savings and the reduce latency benefit.