Stable-Diffusion-WebUI-TensorRT: Failed to parse ONNX model. Does the model file exist and contain a valid ONNX model?

the error is as follows

Disabling attention optimization
Exporting mix18.fp16 to TensorRT using - Batch Size: 1-1-4
Height: 512-512-768
Width: 512-512-768
Token Count: 75-75-150
{'sample': [(1, 4, 64, 64), (2, 4, 64, 64), (8, 4, 96, 96)], 'timesteps': [(1,), (2,), (8,)], 'encoder_hidden_states': [(1, 77, 768), (2, 77, 768), (8, 154, 768)]}
============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

ERROR:root:Exporting to ONNX failed. Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

ERROR:root:Exporting to ONNX failed. Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument weight in method wrapper_CUDA__native_group_norm)
Building TensorRT engine... This can take a while, please check the progress in the terminal.
Building TensorRT engine for A:\AI\Packages\A1111\models\Unet-onnx\mix18.fp16.onnx\encoder.onnx: A:\AI\Packages\A1111\models\Unet-trt\cc86\unet\mix18.fp16\id11a1472e3471eb1aa75d8acec6ea290\encoder.trt
Could not open file A:\AI\Packages\A1111\models\Unet-onnx\mix18.fp16.onnx\encoder.onnx
[E] ModelImporter.cpp:773: Failed to parse ONNX model from file: A:\AI\Packages\A1111\models\Unet-onnx\mix18.fp16.onnx\encoder.onnx
Could not open file A:\AI\Packages\A1111\models\Unet-onnx\mix18.fp16.onnx\encoder.onnx
[!] Failed to parse ONNX model. Does the model file exist and contain a valid ONNX model?
Traceback (most recent call last):
  File "A:\AI\Packages\A1111\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "A:\AI\Packages\A1111\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "A:\AI\Packages\A1111\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "A:\AI\Packages\A1111\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "A:\AI\Packages\A1111\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "A:\AI\Packages\A1111\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "A:\AI\Packages\A1111\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "A:\AI\Packages\A1111\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 249, in export_unet_to_trt
    ret = export_trt(
  File "A:\AI\Packages\A1111\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 234, in export_trt
    ret = engine.build(
  File "A:\AI\Packages\A1111\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 251, in build
    network = network_from_onnx_path(
  File "<string>", line 3, in network_from_onnx_path
  File "A:\AI\Packages\A1111\venv\lib\site-packages\polygraphy\backend\base\loader.py", line 40, in __call__
    return self.call_impl(*args, **kwargs)
  File "A:\AI\Packages\A1111\venv\lib\site-packages\polygraphy\util\util.py", line 710, in wrapped
    return func(*args, **kwargs)
  File "A:\AI\Packages\A1111\venv\lib\site-packages\polygraphy\backend\trt\loader.py", line 227, in call_impl
    trt_util.check_onnx_parser_errors(parser, success)
  File "A:\AI\Packages\A1111\venv\lib\site-packages\polygraphy\backend\trt\util.py", line 86, in check_onnx_parser_errors
    G_LOGGER.critical("Failed to parse ONNX model. Does the model file exist and contain a valid ONNX model?")
  File "A:\AI\Packages\A1111\venv\lib\site-packages\polygraphy\logger\logger.py", line 605, in critical
    raise ExceptionType(message) from None
polygraphy.exception.exception.PolygraphyException: Failed to parse ONNX model. Does the model file exist and contain a valid ONNX model?

which i’ve gotten around it by force rebuilding a before 0.2.0 version trt model of the same checkpoint, another odd thing is that after you make one, you can make any other one just fine. also pytorch fallback is weird, you need double the batch size to make it function properly. Idk whats wrong so any help as to making a model (for the first time) in V 0.2.0 would be appreciated.

About this issue

  • Original URL
  • State: open
  • Created 5 months ago
  • Reactions: 2
  • Comments: 19

Most upvoted comments

I was getting the same error today at all sizes and parameters, but I got it to work by removing --xformers in the startup config, so for me it is blank. Also, in the webui settings under optimizations I have: Cross attention optimization: Automatic 0 for all the sliders (Neg. Guidance and all 3 Token Merging)

✅Pad prompt/negative prompt ⬛Pad prompt/negative prompt (v0) ✅Persistent cond cache ✅Batch cond/uncond

FP8 weight Disable ⬛Cache FP16 weight for LoRA

Hope this helps

I make it working usint the controlnet_v2 branch!

I made some benchmark on my computer (RTX4090) and here what I got :

  • SDXL 1024x1024, 20 samples, w/o tensorRT : 5.9sec
  • SDXL 1024x1024, 20 samples, w/ tensorRT : 3.9sec
  • SDXL Lightining 1024x1024, 6 samples, w/o tensorRT : 2.0sec
  • SDXL Lightining 1024x1024, 6 samples, w/ tensorRT : 1.4sec

I cannot imagine that this is repo from 2000B dollars company 😄 No one support, hello some body here…

If you get this error after installing the TensorRT Extension, just remove any parameters in COMMANDLINE_ARGS, it worked for me.

What size are you guys trying to export, when the ONXX model error appears?

I fixed this issue by exporting the ABSOLUTE MINIMUM resolution that I can of a model first, 1 batch size. This can use almost no VRAM, leaving all of it for the ONNX export at first.

After that, the ONNX is present, so you can export any other resolution that you would like. (IF, and ONLY IF you managed to export it with the previous step.)

It’s a dumb workaround, but it worked for me.

I was just using defaults or normal SDXL resolutions. I see your posts about VRAM usage being high, but I don’t see any spikes in my VRAM at all, it just errors right away. I’ll try your workaround, thanks!

I made :

venv\Scripts\activate.bat
python -m pip uninstall torch torchvision torchaudio  
python -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 
webui-user.bat

But it does not fix the issue : ERROR:root:Exporting to ONNX failed. Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm) polygraphy.exception.exception.PolygraphyException: Failed to parse ONNX model. Does the model file exist and contain a valid ONNX model?

I tried the same exact thing with no luck.

Edit: Still no luck with either method. I have 24gb of VRAM, it’s simply unused before it errors. I’ve tried different models and settings. Maybe I’m just not understanding something. I’m using the latest commit in the main branch.

I made :

venv\Scripts\activate.bat
python -m pip uninstall torch torchvision torchaudio  
python -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 
webui-user.bat

But it does not fix the issue : ERROR:root:Exporting to ONNX failed. Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm) polygraphy.exception.exception.PolygraphyException: Failed to parse ONNX model. Does the model file exist and contain a valid ONNX model?

我尝试了安装PyTorch以后,重启电脑就解决了。 因为查资料翻阅到无法建立的原因和PyTorch有关,安装重启以后就可以运行了,我也不知道是不是和这个有关,毕竟不太懂变成。 如果你是中国用户并且用的是秋叶包,那你可以在启动页面–高级选项–环境维护里面有这个选项。 希望能帮到你。