ml-stable-diffusion: Unsupported type in MLMultiArray when running with Swift CLI

Error: CoreML/MLShapedArray.swift:557: Fatal error: MLMultiArray of data type Float16 is not supported.

After generation of the model in using

python -m python_coreml_stable_diffusion.torch2coreml --convert-unet --convert-text-encoder --convert-vae-decoder --model-version runwayml/stable-diffusion-v1-5 --bundle-resources-for-swift-cli --attention-implementation SPLIT_EINSUM -o swift-original-coreml-sd-v1-5

and running with

swift run StableDiffusionSample "a photo of an astronaut riding a horse on mars" --resource-path swift-original-coreml-sd-v1-5/Resources/ --seed 93 --output-path swift_images

the above error occurs.

Seems to be occuring during execution of line line 89 in TextEncoder.swift StableDiffusion/pipeline/TextEncoder.swift

return MLShapedArray<Float32>(converting: embeddingFeature!.multiArrayValue!)

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 20 (9 by maintainers)

Most upvoted comments

I think I’ve narrowed this down to a change in default behavior with compute_precision=None in coremltools 7.0b2, which started defaulting to float16 instead of float32 if unspecified. If you are looking for a quick solution, you could go with forcing float32 as the compute precision by manually editing the convert function. Here’s the place to do it:

https://github.com/apple/ml-stable-diffusion/blob/7b59c2b663b42349bb76f0a4386b011795bd727d/python_coreml_stable_diffusion/torch2coreml.py#L129 Just change precision to ct.precision.FLOAT32

The reason the old models work is because they defaulted to float32 at the time they were made.

It’s not clear to me why the change was made to coremltools but I would consider it a breaking change from 6.3.0 -> 7.0 for existing pipelines such as this one, and could probably use a specific call-out for what needs to be updated since macOS 13 doesn’t support converting MLShapedArrays directly from FP16 MLMultiArrays

Yes, macOS13 doesn’t support converting MLShapedArrays directly from FP16 MLMultiArrays. Particularly, the conversion of MLMultiArray<Float16> -> MLShapedArray<Float32>. It would be resolved with macOS14.

To keep the existing pipelines working, in the coremltools.convert call, we can explicitly provide output dtype to np.float32. This would probably bypass the need of MLMultiArray<Float16> -> MLShapedArray<Float32> conversion.

Thanks @DawerG, incorporated your suggestion with this commit https://github.com/apple/ml-stable-diffusion/pull/227/commits/efda89332c2be9ae969d5cffb658c2507210ee6b and it appears to work with 7.0b2 👍

From the 7.0b2 release notes:

`The default neural network backend is now mlprogram for iOS15/macOS12. Previously calling coremltools.convert() without providing the convert_to or the minimum_deployment_target arguments, used the lowest deployment target (iOS11/macOS10.13) and the neuralnetwork backend. Now the conversion process will default to iOS15/macOS12 and the mlprogram backend. You can change this behavior by providing a minimum_deployment_target or convert_to value.

Changes default I/O type for Neural Networks to FP16 for iOS16/macOS13 or later when mlprogram backend is used.`

Can an argument be added to something in this repo, maybe torch2coreml.py, that would trigger the “previous” behavior? Call it --legacy-compatability or something like that?

I can confirm that in 7.0b2 coremltools changed the default I/O type to FP16, for iOS16/macOS13 or later when mlprogram backend is used. This change was intentional and even called out in our Release Notes for 7.0b2.

Thanks for the find @ZachNagengast! cc: @TobyRoseman @aseemw for coremltools visibility