stablediffusion: RuntimeError: Input type (c10::Half) and bias type (float) should be the same

I am using M1 Pro MacBook and I am trying to develop a stablediffusion using mps.

I changed the part about cuda to mps and changed it from ddim.py to float32 because mps did not support float64.

def register_buffer(self, name, attr): if type(attr) == torch.Tensor: if attr.device != torch.device("mps"): attr = attr.to(torch.float32).to(torch.device("mps")) setattr(self, name, attr)

def make_schedule(self, ddim_num_steps, ddim_discretize="uniform", ddim_eta=0., verbose=True): self.ddim_timesteps = make_ddim_timesteps(ddim_discr_method=ddim_discretize, num_ddim_timesteps=ddim_num_steps, num_ddpm_timesteps=self.ddpm_num_timesteps,verbose=verbose) alphas_cumprod = self.model.alphas_cumprod assert alphas_cumprod.shape[0] == self.ddpm_num_timesteps, 'alphas have to be defined for each timestep' to_torch = lambda x: x.clone().detach().to(torch.float32).to(self.model.device)

Since then, this problem has occurred in conv.py

def _conv_forward(self, input: Tensor, weight: Tensor, bias: Optional[Tensor]): if self.padding_mode != 'zeros': return F.conv2d(F.pad(input, self._reversed_padding_repeated_twice, mode=self.padding_mode), weight, bias, self.stride, _pair(0), self.dilation, self.groups) return F.conv2d(input, weight, bias, self.stride, self.padding, self.dilation, self.groups)

def forward(self, input: Tensor) -> Tensor: return self._conv_forward(input, self.weight, self.bias)

Help me please.

About this issue

Most upvoted comments

when use diffuser pipeline , i meet this,

my solution is :

pipe.to(“cuda”, torch.float16) try : variable.to(device, torch_type) and then it run no problem

return F.conv2d(input, weight, bias, self.stride,

RuntimeError: Input type (c10::Half) and bias type (float) should be the same

I also got the above problem @yyahav . I am using ubuntu not macOS

I would like to see all the logs and what you have run. Can you show me?

Can I connect with you on Discord? I already sent a request… 😃

Sure! But I might as well talk about it here in case anyone encounters a similar error in the future!

I was able to resolve this issue using:

  autocast_ctx = torch.autocast(pipe.device.type)

  with autocast_ctx:

Same error here, it’s just the opposite : RuntimeError: Input type (float) and bias type ([float](c10::Half)) should be the same].

Only when I use Hires fix, any model, any upscaler.

That’s not a VRAM leak because it does it every time I try, even after computer restart.

My command line : COMMANDLINE_ARGS=--upcast-sampling --medvram --xformers --no-half-vae

After days of troubleshooting, I was able to resolve this by upgrading tensorflow to 2.11.0 and editing the v2-inference.yaml file’s parameter of use_fp16 to False

Ok, I was fooling around, and got “RuntimeError: Input type (c10::Half) and bias type (float) should be the same” error again. It doesn’t show up when you use “cuda”. Why are you trying mps instead?

I think that error is caused by xformers, so I’d like you to try uninstalling it

I’d appreciate it if you did that.

it’s not work 🥺

this occurred error

Traceback (most recent call last): File “/Users/blackcat/study/stablediffusion/scripts/txt2img.py”, line 393, in <module> main(opt) File “/Users/blackcat/study/stablediffusion/scripts/txt2img.py”, line 352, in main samples, _ = sampler.sample(S=opt.steps, File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/autograd/grad_mode.py”, line 27, in decorate_context return func(*args, **kwargs) File “/Users/blackcat/study/stablediffusion/scripts/ldm/models/diffusion/ddim.py”, line 107, in sample samples, intermediates = self.ddim_sampling(conditioning, size, File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/autograd/grad_mode.py”, line 27, in decorate_context return func(*args, **kwargs) File “/Users/blackcat/study/stablediffusion/scripts/ldm/models/diffusion/ddim.py”, line 167, in ddim_sampling outs = self.p_sample_ddim(img, cond, ts, index=index, use_original_steps=ddim_use_original_steps, File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/autograd/grad_mode.py”, line 27, in decorate_context return func(*args, **kwargs) File “/Users/blackcat/study/stablediffusion/scripts/ldm/models/diffusion/ddim.py”, line 215, in p_sample_ddim model_uncond, model_t = self.model.apply_model(x_in, t_in, c_in).chunk(2) File “/Users/blackcat/study/stablediffusion/scripts/ldm/models/diffusion/ddpm.py”, line 858, in apply_model x_recon = self.model(x_noisy, t, **cond) File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1194, in _call_impl return forward_call(*input, **kwargs) File “/Users/blackcat/study/stablediffusion/scripts/ldm/models/diffusion/ddpm.py”, line 1335, in forward out = self.diffusion_model(x, t, context=cc) File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1194, in _call_impl return forward_call(*input, **kwargs) File “/Users/blackcat/study/stablediffusion/scripts/ldm/modules/diffusionmodules/openaimodel.py”, line 778, in forward h = module(h, emb, context) File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1194, in _call_impl return forward_call(*input, **kwargs) File “/Users/blackcat/study/stablediffusion/scripts/ldm/modules/diffusionmodules/openaimodel.py”, line 86, in forward x = layer(x) File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1194, in _call_impl return forward_call(*input, **kwargs) File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/nn/modules/conv.py”, line 463, in forward return self._conv_forward(input, self.weight, self.bias) File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/nn/modules/conv.py”, line 459, in _conv_forward return F.conv2d(input, weight, bias, self.stride, RuntimeError: Input type (c10::Half) and bias type (float) should be the same

I was getting these, and eventually deduced what was going on. These errors appeared during the render process when it’d show updates. Instead of updates, these errors were appearing… I did some research. I had been moving the default models folder and using my own. The folder I had didn’t have various files it was looking for. It turns out I needed to merge in the default files to get it working…

Here for example is what I typed in…

cd stable-diffusion-webui
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git SD-hold
cp -r SD-hold/models/* models/
rm -rf SD-hold

That restores the files to the models folder tree, and solved this for me.

I think the reason this happens is because you are using fp16

@Tps-F It’s faster because I reduced the batch size. Thank you. Are you interested in object detection like ssd(single shot multibox detector) or YOLO? I want trying ssd in m1 Mac but that model used to CUDA how to convert CUDA to MPS?