stable-diffusion-webui-depthmap-script: zoedepth_nk throws error during compute of depth map

When trying to get depthmap on the depth tab, it throws error when using zoedepth_nk. res101 works fine. it does this both on the normal tab and on batch tab `DepthMap v0.3.12 (54193c4f) device: cuda Loading model weights from zoedepth_nk

img_size [512, 512] Using cache found in C:\Users\user/.cache\torch\hub\intel-isl_MiDaS_master Params passed to Resize transform: width: 512 height: 512 resize_target: True keep_aspect_ratio: True ensure_multiple_of: 32 resize_method: minimal Using pretrained resource url::https://github.com/isl-org/ZoeDepth/releases/download/v1.0/ZoeD_M12_NK.pt Loaded successfully initialize network with normal loading the model from ./models/pix2pix\latest_net_G.pth Computing depthmap(s) … 0%| | 0/1 [00:00<?, ?it/s]

wholeImage being processed in : 1536 0%| | 0/1 [00:03<?, ?it/s] *** Error completing request *** Arguments: (0, <PIL.Image.Image image mode=RGB size=2048x1152 at 0x17D8212CEB0>, None, ‘’, ‘’, 0, 9, 512, 512, False, True, False, False, 0, 1, False, 1, True, False, False, True, [‘left-right’], 2.8, 0, ‘polylines_sharp’, 0, False, False, False, False, False, ‘u2net’, False, 0, 2, False, None, True, False, True, False) {} Traceback (most recent call last): File “D:\Oculus\stable-diffusion-webui\modules\call_queue.py”, line 55, in f res = list(func(*args, **kwargs)) File “D:\Oculus\stable-diffusion-webui\modules\call_queue.py”, line 35, in f res = func(*args, **kwargs) File “D:\Oculus\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py”, line 1236, in run_generate outputs, mesh_fi, meshsimple_fi = run_depthmap( File “D:\Oculus\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py”, line 627, in run_depthmap prediction = estimateboost(img, model, model_type, pix2pixmodel) File “D:\Oculus\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py”, line 1978, in estimateboost whole_estimate = doubleestimate(img, net_receptive_field_size, whole_image_optimal_size, pix2pixsize, model, model_type, pix2pixmodel) File “D:\Oculus\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py”, line 1691, in doubleestimate estimate1 = singleestimate(img, size1, model, net_type) File “D:\Oculus\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py”, line 1718, in singleestimate return estimatezoedepth(Image.fromarray(np.uint8(img * 255)).convert(‘RGB’), model, msize, msize) File “D:\Oculus\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py”, line 1485, in estimatezoedepth prediction = model.infer_pil(img) File “D:\Oculus\stable-diffusion-webui\venv\lib\site-packages\torch\utils_contextlib.py”, line 115, in decorate_context return func(*args, **kwargs) File “D:\Oculus\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\dzoedepth\models\depth_model.py”, line 141, in infer_pil out_tensor = self.infer(x, pad_input=pad_input, with_flip_aug=with_flip_aug, **kwargs) File “D:\Oculus\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\dzoedepth\models\depth_model.py”, line 126, in infer return self.infer_with_flip_aug(x, pad_input=pad_input, **kwargs) File “D:\Oculus\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\dzoedepth\models\depth_model.py”, line 110, in infer_with_flip_aug out = self._infer_with_pad_aug(x, pad_input=pad_input, **kwargs) File “D:\Oculus\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\dzoedepth\models\depth_model.py”, line 88, in _infer_with_pad_aug out = self._infer(x) File “D:\Oculus\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\dzoedepth\models\depth_model.py”, line 55, in _infer return self(x)[‘metric_depth’] File “D:\Oculus\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py”, line 1522, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File “D:\Oculus\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py”, line 1531, in _call_impl return forward_call(*args, **kwargs) File “D:\Oculus\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\dzoedepth\models\zoedepth_nk\zoedepth_nk_v1.py”, line 178, in forward rel_depth, out = self.core(x, denorm=denorm, return_rel_depth=True) File “D:\Oculus\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py”, line 1522, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File “D:\Oculus\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py”, line 1531, in _call_impl return forward_call(*args, **kwargs) File “D:\Oculus\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\dzoedepth\models\base_models\midas.py”, line 262, in forward x = self.prep(x) File “D:\Oculus\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\dzoedepth\models\base_models\midas.py”, line 186, in call return self.normalization(self.resizer(x)) File “D:\Oculus\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\dzoedepth\models\base_models\midas.py”, line 173, in call return nn.functional.interpolate(x, (height, width), mode=‘bilinear’, align_corners=True) File “D:\Oculus\stable-diffusion-webui\venv\lib\site-packages\torch\nn\functional.py”, line 3934, in interpolate raise TypeError( TypeError: expected size to be one of int or Tuple[int] or Tuple[int, int] or Tuple[int, int, int], but got size with types [<class ‘numpy.int32’>, <class ‘numpy.int32’>]


Traceback (most recent call last): File “D:\Oculus\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py”, line 422, in run_predict output = await app.get_blocks().process_api( File “D:\Oculus\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py”, line 1326, in process_api data = self.postprocess_data(fn_index, result[“prediction”], state) File “D:\Oculus\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py”, line 1229, in postprocess_data self.validate_outputs(fn_index, predictions) # type: ignore File “D:\Oculus\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py”, line 1204, in validate_outputs raise ValueError( ValueError: An event handler (f) didn’t receive enough output values (needed: 5, received: 3). Wanted outputs: [gallery, textbox, model3d, html, html] Received outputs: [None, “”, “<div class='error'>TypeError: expected size to be one of int or Tuple[int] or Tuple[int, int] or Tuple[int, int, int], but got size with types [<class 'numpy.int32'>, <class 'numpy.int32'>]</div><div class='performance'>

Time taken: <wbr>21.60s

Torch active/reserved: 3832/3856 MiB, <wbr>Sys VRAM: 5989/12288 MiB (48.74%)

</div>”]`

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 17 (4 by maintainers)

Most upvoted comments

@thygate - can I buy you coffee somewhere?

Weird. It seemed to have worked! I deleted all extensions and the venv folder, and now it’s processing the image. Thanks so much @thygate - love this extension!