stablediffusion: RuntimeError: Input type (c10::Half) and bias type (float) should be the same
I am using M1 Pro MacBook and I am trying to develop a stablediffusion using mps.
I changed the part about cuda to mps and changed it from ddim.py to float32 because mps did not support float64.
def register_buffer(self, name, attr): if type(attr) == torch.Tensor: if attr.device != torch.device("mps"): attr = attr.to(torch.float32).to(torch.device("mps")) setattr(self, name, attr)
def make_schedule(self, ddim_num_steps, ddim_discretize="uniform", ddim_eta=0., verbose=True): self.ddim_timesteps = make_ddim_timesteps(ddim_discr_method=ddim_discretize, num_ddim_timesteps=ddim_num_steps, num_ddpm_timesteps=self.ddpm_num_timesteps,verbose=verbose) alphas_cumprod = self.model.alphas_cumprod assert alphas_cumprod.shape[0] == self.ddpm_num_timesteps, 'alphas have to be defined for each timestep' to_torch = lambda x: x.clone().detach().to(torch.float32).to(self.model.device)
Since then, this problem has occurred in conv.py
def _conv_forward(self, input: Tensor, weight: Tensor, bias: Optional[Tensor]): if self.padding_mode != 'zeros': return F.conv2d(F.pad(input, self._reversed_padding_repeated_twice, mode=self.padding_mode), weight, bias, self.stride, _pair(0), self.dilation, self.groups) return F.conv2d(input, weight, bias, self.stride, self.padding, self.dilation, self.groups)
def forward(self, input: Tensor) -> Tensor: return self._conv_forward(input, self.weight, self.bias)
Help me please.
About this issue
- Original URL
- State: open
- Created 4 years ago
- Comments: 54
when use diffuser pipeline , i meet this,
my solution is :
pipe.to(“cuda”, torch.float16) try :
variable.to(device, torch_type)and then it run no problemRuntimeError: Input type (c10::Half) and bias type (float) should be the same
I also got the above problem @yyahav . I am using ubuntu not macOS
I would like to see all the logs and what you have run. Can you show me?
Sure! But I might as well talk about it here in case anyone encounters a similar error in the future!
I was able to resolve this issue using:
Same error here, it’s just the opposite :
RuntimeError: Input type (float) and bias type ([float](c10::Half)) should be the same].Only when I use Hires fix, any model, any upscaler.
That’s not a VRAM leak because it does it every time I try, even after computer restart.
My command line :
COMMANDLINE_ARGS=--upcast-sampling --medvram --xformers --no-half-vaeAfter days of troubleshooting, I was able to resolve this by upgrading tensorflow to 2.11.0 and editing the
v2-inference.yamlfile’s parameter ofuse_fp16to FalseOk, I was fooling around, and got “RuntimeError: Input type (c10::Half) and bias type (float) should be the same” error again. It doesn’t show up when you use “cuda”. Why are you trying mps instead?
I think that error is caused by xformers, so I’d like you to try uninstalling it
I’d appreciate it if you did that.
it’s not work 🥺
this occurred error
Traceback (most recent call last): File “/Users/blackcat/study/stablediffusion/scripts/txt2img.py”, line 393, in <module> main(opt) File “/Users/blackcat/study/stablediffusion/scripts/txt2img.py”, line 352, in main samples, _ = sampler.sample(S=opt.steps, File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/autograd/grad_mode.py”, line 27, in decorate_context return func(*args, **kwargs) File “/Users/blackcat/study/stablediffusion/scripts/ldm/models/diffusion/ddim.py”, line 107, in sample samples, intermediates = self.ddim_sampling(conditioning, size, File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/autograd/grad_mode.py”, line 27, in decorate_context return func(*args, **kwargs) File “/Users/blackcat/study/stablediffusion/scripts/ldm/models/diffusion/ddim.py”, line 167, in ddim_sampling outs = self.p_sample_ddim(img, cond, ts, index=index, use_original_steps=ddim_use_original_steps, File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/autograd/grad_mode.py”, line 27, in decorate_context return func(*args, **kwargs) File “/Users/blackcat/study/stablediffusion/scripts/ldm/models/diffusion/ddim.py”, line 215, in p_sample_ddim model_uncond, model_t = self.model.apply_model(x_in, t_in, c_in).chunk(2) File “/Users/blackcat/study/stablediffusion/scripts/ldm/models/diffusion/ddpm.py”, line 858, in apply_model x_recon = self.model(x_noisy, t, **cond) File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1194, in _call_impl return forward_call(*input, **kwargs) File “/Users/blackcat/study/stablediffusion/scripts/ldm/models/diffusion/ddpm.py”, line 1335, in forward out = self.diffusion_model(x, t, context=cc) File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1194, in _call_impl return forward_call(*input, **kwargs) File “/Users/blackcat/study/stablediffusion/scripts/ldm/modules/diffusionmodules/openaimodel.py”, line 778, in forward h = module(h, emb, context) File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1194, in _call_impl return forward_call(*input, **kwargs) File “/Users/blackcat/study/stablediffusion/scripts/ldm/modules/diffusionmodules/openaimodel.py”, line 86, in forward x = layer(x) File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 1194, in _call_impl return forward_call(*input, **kwargs) File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/nn/modules/conv.py”, line 463, in forward return self._conv_forward(input, self.weight, self.bias) File “/opt/anaconda3/envs/cat/lib/python3.9/site-packages/torch/nn/modules/conv.py”, line 459, in _conv_forward return F.conv2d(input, weight, bias, self.stride, RuntimeError: Input type (c10::Half) and bias type (float) should be the same
I was getting these, and eventually deduced what was going on. These errors appeared during the render process when it’d show updates. Instead of updates, these errors were appearing… I did some research. I had been moving the default models folder and using my own. The folder I had didn’t have various files it was looking for. It turns out I needed to merge in the default files to get it working…
Here for example is what I typed in…
That restores the files to the models folder tree, and solved this for me.
I think the reason this happens is because you are using fp16
@Tps-F It’s faster because I reduced the batch size. Thank you. Are you interested in object detection like ssd(single shot multibox detector) or YOLO? I want trying ssd in m1 Mac but that model used to CUDA how to convert CUDA to MPS?
Here is!
https://github.com/Stability-AI/stablediffusion/pull/163#issuecomment-1422351441