InvokeAI: Safety checker error

I’d appreciate if someone could check if they can repro this issue. miranda kerr nude

>> Ksampler using model noise schedule (steps > 29)
>> Sampling with k_lms starting at step 0 of 50 (50 new sampling steps)
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:27<00:00,  1.84it/s]
Generating:   0%|                                        
Traceback (most recent call last):
  File "/Users/me/Downloads/stable-diffusion/ldm/generate.py", line 455, in prompt2image
    results = generator.generate(
  File "/Users/me/Downloads/stable-diffusion/ldm/invoke/generator/base.py", line 93, in generate
    image = self.safety_check(image)
  File "/Users/me/Downloads/stable-diffusion/ldm/invoke/generator/base.py", line 206, in safety_check
    checked_image, has_nsfw_concept = checker(images=x_image, clip_input=features.pixel_values)
  File "/Users/me/opt/anaconda3/envs/do_not_touch-osx-arm64-stable-diffusion/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/me/opt/anaconda3/envs/do_not_touch-osx-arm64-stable-diffusion/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/Users/me/opt/anaconda3/envs/do_not_touch-osx-arm64-stable-diffusion/lib/python3.9/site-packages/diffusers/pipelines/stable_diffusion/safety_checker.py", line 38, in forward
    pooled_output = self.vision_model(clip_input)[1]  # pooled_output
  File "/Users/me/opt/anaconda3/envs/do_not_touch-osx-arm64-stable-diffusion/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/me/opt/anaconda3/envs/do_not_touch-osx-arm64-stable-diffusion/lib/python3.9/site-packages/transformers/models/clip/modeling_clip.py", line 840, in forward
    return self.vision_model(
  File "/Users/me/opt/anaconda3/envs/do_not_touch-osx-arm64-stable-diffusion/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/me/opt/anaconda3/envs/do_not_touch-osx-arm64-stable-diffusion/lib/python3.9/site-packages/transformers/models/clip/modeling_clip.py", line 771, in forward
    hidden_states = self.embeddings(pixel_values)
  File "/Users/me/opt/anaconda3/envs/do_not_touch-osx-arm64-stable-diffusion/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/Users/me/opt/anaconda3/envs/do_not_touch-osx-arm64-stable-diffusion/lib/python3.9/site-packages/transformers/models/clip/modeling_clip.py", line 137, in forward
    embeddings = torch.cat([class_embeds, patch_embeds], dim=1)
RuntimeError: torch.cat(): all input tensors must be on the same device. Received mps:0 and cpu

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 19 (8 by maintainers)

Most upvoted comments

I also updated my dev branch. Now not all generations are flagged. puppy Screenshot 2022-10-30 at 19 15 07 nude

About the cpu/mps error fix, base.py update worked, but generate.py update did not. Let me check one more time after pulling latest changes. Update. As expected, didn’t work.

Okay, so updating base.py works. And only not safe for work images are flagged.

So from my side, this is fixed with this change.

if hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
    checker.to('mps')
    features.to('mps')

Any-Winter-4079 on Oct 30, 2022