ml-stable-diffusion: Unsupported type in MLMultiArray when running with Swift CLI
Error:
CoreML/MLShapedArray.swift:557: Fatal error: MLMultiArray of data type Float16 is not supported.
After generation of the model in using
python -m python_coreml_stable_diffusion.torch2coreml --convert-unet --convert-text-encoder --convert-vae-decoder --model-version runwayml/stable-diffusion-v1-5 --bundle-resources-for-swift-cli --attention-implementation SPLIT_EINSUM -o swift-original-coreml-sd-v1-5
and running with
swift run StableDiffusionSample "a photo of an astronaut riding a horse on mars" --resource-path swift-original-coreml-sd-v1-5/Resources/ --seed 93 --output-path swift_images
the above error occurs.
Seems to be occuring during execution of line line 89 in TextEncoder.swift StableDiffusion/pipeline/TextEncoder.swift
return MLShapedArray<Float32>(converting: embeddingFeature!.multiArrayValue!)
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 20 (9 by maintainers)
I think I’ve narrowed this down to a change in default behavior with
compute_precision=Nonein coremltools 7.0b2, which started defaulting to float16 instead of float32 if unspecified. If you are looking for a quick solution, you could go with forcing float32 as the compute precision by manually editing the convert function. Here’s the place to do it:https://github.com/apple/ml-stable-diffusion/blob/7b59c2b663b42349bb76f0a4386b011795bd727d/python_coreml_stable_diffusion/torch2coreml.py#L129 Just change
precisiontoct.precision.FLOAT32The reason the old models work is because they defaulted to float32 at the time they were made.
Yes, macOS13 doesn’t support converting MLShapedArrays directly from FP16 MLMultiArrays. Particularly, the conversion of
MLMultiArray<Float16> -> MLShapedArray<Float32>. It would be resolved with macOS14.To keep the existing pipelines working, in the
coremltools.convertcall, we can explicitly provide output dtype tonp.float32. This would probably bypass the need ofMLMultiArray<Float16> -> MLShapedArray<Float32>conversion.Thanks @DawerG, incorporated your suggestion with this commit https://github.com/apple/ml-stable-diffusion/pull/227/commits/efda89332c2be9ae969d5cffb658c2507210ee6b and it appears to work with
7.0b2👍From the 7.0b2 release notes:
`The default neural network backend is now mlprogram for iOS15/macOS12. Previously calling coremltools.convert() without providing the convert_to or the minimum_deployment_target arguments, used the lowest deployment target (iOS11/macOS10.13) and the neuralnetwork backend. Now the conversion process will default to iOS15/macOS12 and the mlprogram backend. You can change this behavior by providing a minimum_deployment_target or convert_to value.
Changes default I/O type for Neural Networks to FP16 for iOS16/macOS13 or later when mlprogram backend is used.`
Can an argument be added to something in this repo, maybe torch2coreml.py, that would trigger the “previous” behavior? Call it
--legacy-compatabilityor something like that?I can confirm that in
7.0b2coremltools changed the default I/O type to FP16, for iOS16/macOS13 or later when mlprogram backend is used. This change was intentional and even called out in our Release Notes for 7.0b2.Thanks for the find @ZachNagengast! cc: @TobyRoseman @aseemw for coremltools visibility