diffusers: Custom pipelines are broken

Describe the bug

It worked a couple of days ago, but today it stopped to. I wish I have never restarted none my space. Downgrading to diffusers 0.11.0, 0.12.0, 0.14.0 etc didn’t help.

Please help me!

Reproduction

from diffusers import DiffusionPipeline, EulerAncestralDiscreteScheduler
import gradio as gr
import numpy as np
import imageio

from PIL import Image, ImageFile
ImageFile.LOAD_TRUNCATED_IMAGES = True
import glob


import torch
from time import gmtime, strftime


device = "cuda" if torch.cuda.is_available() else "cpu"
pipe = DiffusionPipeline.from_pretrained("NickKolok/meryl-stryfe-20230123-2300-6k-2400-steps_1", custom_pipeline="lpw_stable_diffusion", torch_dtype=torch.float32)
pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config)
pipe.to(device)

def resize(height,img):
  baseheight = height
  img = Image.open(img)
  hpercent = (baseheight/float(img.size[1]))
  wsize = int((float(img.size[0])*float(hpercent)))
  img = img.resize((wsize,baseheight), Image.Resampling.LANCZOS)
  return img

def predict(prompt, negative_prompt):
    image = pipe(prompt=prompt, negative_prompt=negative_prompt, num_inference_steps=30, width=512, height=576).images[0]
    timestamp = strftime("%Y-%m-%d---%H-%M-%S", gmtime())
    #img = Image.open(image)
    image.save("./diffused__"+timestamp+"_.png")
    return glob.glob('./diffused_*.png')[-4:]

title="test"
description="test"

gr.Interface(
    fn=predict,
    inputs=[
        #gr.Image(source="upload", type="numpy", tool="sketch", elem_id="source_container"),
        gr.Textbox(label='What you want the AI to Generate, 77 Token limit'),
        gr.Textbox(label='What you Do Not want the AI to generate')
    ],
    outputs=[
        gr.Gallery(
            label="Generated images", show_label=False, elem_id="output_container_0"
        ).style(grid=[2], height="auto"),
        #gr.Image(elem_id="output_container_0"),
        #gr.Image(elem_id="output_container_1"),
    ],
    title=title,
    description=description,
    article = ""
).launch(max_threads=True, debug=True)

Logs

Downloading (…)_stable_diffusion.py:   0%|          | 0.00/13.9k [00:00<?, ?B/s]
Downloading (…)_stable_diffusion.py: 76.5kB [00:00, 17.6MB/s]                   
Traceback (most recent call last):
  File "app.py", line 16, in <module>
    pipe = DiffusionPipeline.from_pretrained("NickKolok/meryl-stryfe-20230123-2300-6k-2400-steps_1", custom_pipeline="lpw_stable_diffusion", torch_dtype=torch.float32)
  File "/home/user/.local/lib/python3.8/site-packages/diffusers/pipelines/pipeline_utils.py", line 655, in from_pretrained
    pipeline_class = get_class_from_dynamic_module(
  File "/home/user/.local/lib/python3.8/site-packages/diffusers/utils/dynamic_modules_utils.py", line 456, in get_class_from_dynamic_module
    return get_class_in_module(class_name, final_module.replace(".py", ""))
  File "/home/user/.local/lib/python3.8/site-packages/diffusers/utils/dynamic_modules_utils.py", line 163, in get_class_in_module
    module = importlib.import_module(module_path)
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/user/.cache/huggingface/modules/diffusers_modules/git/lpw_stable_diffusion.py", line 13, in <module>
    from diffusers.image_processor import VaeImageProcessor
ModuleNotFoundError: No module named 'diffusers.image_processor'

System Info

huggingface spaces (It runs on cpu, on a free HF instance - if that helps.) different diffusers versions (from 0.11.0 to 0.14.0 and even newest)

About this issue

Original URL
State: closed
Created a year ago
Comments: 29 (18 by maintainers)

Most upvoted comments

Compel uses different syntax, so it’s difficult to migrate existing workflows.

True, (()) is not there. Thinking of taking prompt weighting of lpw as separate lib, to get out of this dependency.

adhikjoshi on Apr 22, 2023

Hey, we’ll revert the PR. We think what’s happening here is that the custom pipeline code downloads the pipeline at diffusers main even if you’re on a older version of diffusers. We should better pin down the compatibility story for community pipelines. Just being safe right now and going to revert the PR.

williamberman on Apr 22, 2023

A simple change to defaulting to pulling in code from the installed version rather than from main I think is the best solution here, which I would imagine is the intended behavior.

Execution of dynamic remote code can also be a serious security risk, not sure if this is a standard practice in other places in the code.

Happy to help on a PR for that.

ghunkins on Apr 22, 2023

You can pin the dependency for dynamic imports. We use 0.15.1.

self.pipe = diffusers.DiffusionPipeline.from_pretrained(
                self.CONFIG["base"]["model_cache"],
                custom_pipeline="lpw_stable_diffusion",
                revision='0.15.1',
                custom_revision='0.15.1',
                torch_dtype=dtype,
            ).to("cuda")

ghunkins on Apr 22, 2023

That’s here again

/home/user/.local/lib/python3.8/site-packages/huggingface_hub/file_download.py:645: FutureWarning: `cached_download` is the legacy way to download files from the HF hub, please consider upgrading to `hf_hub_download`
  warnings.warn(

Downloading (…)_stable_diffusion.py:   0%|          | 0.00/10.5k [00:00<?, ?B/s]Could not locate the pipeline.py inside lpw_stable_diffusion.
Traceback (most recent call last):
  File "app.py", line 16, in <module>
    pipe = DiffusionPipeline.from_pretrained(
  File "/home/user/.local/lib/python3.8/site-packages/diffusers/pipelines/pipeline_utils.py", line 856, in from_pretrained
    cached_folder = cls.download(
  File "/home/user/.local/lib/python3.8/site-packages/diffusers/pipelines/pipeline_utils.py", line 1227, in download
    pipeline_class = _get_pipeline_class(
  File "/home/user/.local/lib/python3.8/site-packages/diffusers/pipelines/pipeline_utils.py", line 335, in _get_pipeline_class
    return get_class_from_dynamic_module(
  File "/home/user/.local/lib/python3.8/site-packages/diffusers/utils/dynamic_modules_utils.py", line 445, in get_class_from_dynamic_module
    final_module = get_cached_module_file(
  File "/home/user/.local/lib/python3.8/site-packages/diffusers/utils/dynamic_modules_utils.py", line 285, in get_cached_module_file
    resolved_module_file = cached_download(
  File "/home/user/.local/lib/python3.8/site-packages/huggingface_hub/utils/_validators.py", line 120, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/user/.local/lib/python3.8/site-packages/huggingface_hub/file_download.py", line 788, in cached_download
    http_get(
  File "/home/user/.local/lib/python3.8/site-packages/huggingface_hub/file_download.py", line 543, in http_get
    raise EnvironmentError(
OSError: Consistency check failed: file should be of size 10495 but has size 57300 ((…)_stable_diffusion.py).
We are sorry for the inconvenience. Please retry download and pass `force_download=True, resume_download=False` as argument.
If the issue persists, please let us know by opening an issue on https://github.com/huggingface/huggingface_hub.

Downloading (…)_stable_diffusion.py: 57.3kB [00:00, 440kB/s]

What’s happening? Could you please stop breaking everything every other day?

nickkolok on Apr 24, 2023

Agree @ghunkins we should default downloading to currently installed version

patrickvonplaten on Apr 22, 2023

I’m really happy to confirm that everything is working like a charm again! Thank you all for your help 😃

nickkolok on Apr 22, 2023

PR is reverted

williamberman on Apr 22, 2023

custom_revision='0.15.1'

At least, it should be emphasized in the docs. That’s really non-obvious.

@ghunkins, thank you very much!

nickkolok on Apr 22, 2023

This is a more minimal reproduction:

import glob
from time import gmtime, strftime

import torch
from diffusers import DiffusionPipeline, EulerAncestralDiscreteScheduler


device = "cuda" if torch.cuda.is_available() else "cpu"
pipe = DiffusionPipeline.from_pretrained("NickKolok/meryl-stryfe-20230123-2300-6k-2400-steps_1", custom_pipeline="lpw_stable_diffusion", torch_dtype=torch.float32)
pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config)
pipe.to(device)


if __name__ == "__main__":
    prompt="dummy"
    negative_prompt="negative dummy"
    image = pipe(prompt=prompt, negative_prompt=negative_prompt, num_inference_steps=30, width=512, height=576).images[0]
    timestamp = strftime("%Y-%m-%d---%H-%M-%S", gmtime())
    image.save("./diffused__"+timestamp+"_.png")
    print(glob.glob('./diffused_*.png')[-4:])

Error:

ImportError: cannot import name 'FromCkptMixin' from 'diffusers.loaders'
(/home/ubuntu/modal/venv/lib/python3.9/site-packages/diffusers/loaders.py)

This reproduction was run with:

diffusers==0.15.1
accelerate==0.18.0
torch==2.0.0+cu117

This import error is appearing in code that has not been redeployed, and is running the same container image that worked previously.

thundergolfer on Apr 22, 2023