Stable-Diffusion-WebUI-TensorRT: Error installing in Automatic1111

Watch below video to learn how to compile SDXL TensorRT and use it - it includes both manual and auto way

RTX Acceleration Quick Tutorial With Auto Installer V2 SDXL - Tensor RT

我修復了卸載 -> 重新安裝錯誤。從終端卸載了tensorrt，執行了pip快取清除（這是解決方法），重新安裝了tensorrt，現在選項卡又回來了。

您可以給出您輸入的命令來執行此操作嗎？

在webui根目錄中：

venv\scripts\activate.bat

pip uninstall tensorrt

pip cache purge

pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4

pip uninstall -y nvidia-cudnn-cu11

The is the correct answer for me, thank you.

same issue here

Watch below video to learn how to compile SDXL TensorRT and use it - it includes both manual and auto way

RTX Acceleration Quick Tutorial With Auto Installer V2 SDXL - Tensor RT

would you please stop polluting threads with clickbait images?

here a quick tutorial for how to install big tutorial still editing

RTX Acceleration Quick Tutorial With Auto Installer

here a quick tutorial for how to install big tutorial still editing

RTX Acceleration Quick Tutorial With Auto Installer

I just finished watching this as you posted it…great video and great job covering all the basics and not so basics 😃

First run: venv\scripts\activate.bat

This will run that activate batch file in the venv\scripts folder that will activate the venv virtual python environment that automatic1111 runs in.

You’ll be able to tell if the virtual environment is active, if the beginning of your command prompt line, shows (venv).

Then you’ll need to run python -m pip uninstall -y nvidia-cudnn-cu11

This will remove cudnn which isn’t needed to run TensorRT, but is currently needed to install TensorRT, however having it installed results in the error you are seeing. The normal install process should do this automatically so that this doesn’t occur, however it’s something that is currently being addressed to resolve.

So… what’s the solution? watching that video above?

install and then switch to dev version of auto1111

sdxl working

For my part, the python -m pip uninstall -y nvidia-cudnn-cu11 didn’t seem to work as the extension was “not installed”.

So instead I went to venv\Lib\site-packages and I removed the cudnn.dist-info & the cudnn folder in the nvidia folder. It seems to be working fine. At least, I can start without errors, and I can start generating the engine. It seems to be generating without issues now.

Edit: I can confirm that after doing this fix, everything works for me.

the problem is medvram. Solved!

ok, figured out the pip freeze.

absl-py==1.4.0
accelerate==0.21.0
addict==2.4.0
aenum==3.1.12
aiofiles==23.2.1
aiohttp==3.8.4
aiosignal==1.3.1
altair==5.0.0
antlr4-python3-runtime==4.9.3
anyio==3.6.2
async-timeout==4.0.2
attrs==23.1.0
basicsr==1.4.2
beautifulsoup4==4.12.2
blendmodes==2022
blis==0.7.9
boltons==23.0.0
Brotli==1.1.0
cachetools==5.3.0
catalogue==2.0.8
certifi==2023.5.7
cffi==1.15.1
chardet==4.0.0
charset-normalizer==3.1.0
clean-fid==0.1.35
click==8.1.7
clip @ git+https://github.com/openai/CLIP.git@d50d76daa670286dd6cacf3bcd80b5e4823fc8e1
colorama==0.4.6
confection==0.0.4
contourpy==1.0.7
cssselect2==0.7.0
cycler==0.11.0
cymem==2.0.7
deprecation==2.1.0
duckduckgo-search==3.9.3
dynamicprompts==0.29.0
einops==0.4.1
facexlib==0.3.0
fastapi==0.94.0
ffmpy==0.3.0
filelock==3.12.0
filterpy==1.4.5
flatbuffers==23.5.9
font-roboto==0.0.1
fonts==0.0.3
fonttools==4.39.4
freetype-py==2.3.0
frozenlist==1.3.3
fsspec==2023.5.0
ftfy==6.1.1
future==0.18.3
fvcore==0.1.5.post20221221
gdown==4.7.1
gfpgan==1.3.8
gitdb==4.0.10
GitPython==3.1.32
google-auth==2.18.1
google-auth-oauthlib==1.0.0
gradio==3.41.2
gradio_client==0.5.0
grpcio==1.54.2
h11==0.12.0
h2==4.1.0
hpack==4.0.0
httpcore==0.15.0
httpx==0.24.1
huggingface-hub==0.14.1
hyperframe==6.0.1
idna==2.10
imageio==2.28.1
importlib-resources==6.0.1
inflection==0.5.1
iopath==0.1.9
Jinja2==3.1.2
jsonmerge==1.8.0
jsonschema==4.17.3
kiwisolver==1.4.4
kornia==0.6.7
langcodes==3.3.0
lark==1.1.2
lazy_loader==0.2
lightning-utilities==0.8.0
linkify-it-py==2.0.2
llvmlite==0.40.0
lmdb==1.4.1
lpips==0.1.4
lxml==4.9.3
Markdown==3.4.3
markdown-it-py==2.2.0
MarkupSafe==2.1.2
matplotlib==3.7.1
mdit-py-plugins==0.3.3
mdurl==0.1.2
mediapipe==0.10.5
mpmath==1.3.0
multidict==6.0.4
murmurhash==1.0.9
networkx==3.1
numba==0.57.0
numpy==1.23.5
nvidia-cublas-cu11==11.11.3.6
nvidia-cuda-nvrtc-cu11==11.8.89
nvidia-cuda-runtime-cu11==11.8.89
nvidia-cudnn-cu11==8.9.4.25
oauthlib==3.2.2
omegaconf==2.2.3
onnx==1.14.1
onnx-graphsurgeon==0.3.27
open-clip-torch==2.20.0
opencv-contrib-python==4.7.0.72
opencv-python==4.8.0.76
orjson==3.8.12
packaging==23.1
pandas==2.0.1
pathy==0.10.1
piexif==1.1.3
Pillow==9.5.0
polygraphy==0.49.0
portalocker==2.7.0
preshed==3.0.8
protobuf==3.20.2
psutil==5.9.5
py-cpuinfo==9.0.0
pyasn1==0.5.0
pyasn1-modules==0.3.0
pycairo==1.23.0
pycparser==2.21
pydantic==1.10.7
pydub==0.25.1
Pygments==2.15.1
pyparsing==3.0.9
pyrsistent==0.19.3
PySocks==1.7.1
python-dateutil==2.8.2
python-multipart==0.0.6
pytorch-lightning==1.9.4
pytz==2023.3
PyWavelets==1.4.1
pywin32==306
PyYAML==6.0
realesrgan==0.3.0
regex==2023.5.5
reportlab==4.0.0
requests==2.31.0
requests-oauthlib==1.3.1
resize-right==0.0.2
rich==13.6.0
rlPyCairo==0.2.0
rsa==4.9
safetensors==0.3.1
scikit-image==0.21.0
scipy==1.10.1
seaborn==0.13.0
semantic-version==2.10.0
Send2Trash==1.8.0
sentencepiece==0.1.99
six==1.16.0
smart-open==6.3.0
smmap==5.0.0
sniffio==1.3.0
socksio==1.0.0
sounddevice==0.4.6
soupsieve==2.4.1
spacy==3.5.3
spacy-legacy==3.0.12
spacy-loggers==1.0.4
srsly==2.4.6
starlette==0.26.1
svglib==1.5.1
sympy==1.12
tabulate==0.9.0
tb-nightly==2.14.0a20230520
tensorboard-data-server==0.7.0
tensorrt==9.0.1.post11.dev4
tensorrt-bindings==9.0.1.post11.dev4
tensorrt-libs==9.0.1.post11.dev4
termcolor==2.3.0
thinc==8.1.10
thop==0.1.1.post2209072238
tifffile==2023.4.12
timm==0.9.2
tinycss2==1.2.1
tokenizers==0.13.3
tomesd==0.1.3
tomli==2.0.1
toolz==0.12.0
torch==2.0.1+cu118
torchdiffeq==0.2.3
torchmetrics==0.11.4
torchsde==0.2.5
torchvision==0.15.2+cu118
tqdm==4.65.0
trampoline==0.1.2
transformers==4.30.2
typer==0.7.0
typing_extensions==4.5.0
tzdata==2023.3
uc-micro-py==1.0.2
ultralytics==8.0.195
urllib3==1.26.15
uvicorn==0.22.0
wasabi==1.1.1
wcwidth==0.2.6
webencodings==0.5.1
websockets==11.0.3
Werkzeug==2.3.4
xformers==0.0.20
yacs==0.1.8
yapf==0.33.0
yarl==1.9.2

I went to console, invoked the batch file from that console, but the python virtual environment was not successfully started doing this method

make sure youre in cmd.exe terminal not powershell

–medvram-sdxl argument was the issue for the following error: ERROR:root:Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

Removed it and the Engine Export is now working.

i also foudn that if i remove the models i converted then the extention disappear from the UI. restoring those deleted files will make the extention appear again… wtf lol

You removed the generated trt model from the Unet-trt folder? If so you need to also remove the reference to that model in the model.json folder. Or if you only have one model generated, just delete the model file and the .json file both.

I tried stopping and restarting the webui server. I made sure I am on the SD 1.5 model. If I try to export the default engine I get the same full error that I did above, ending in 'AsyncRequest' object has no attribute '_json_response_data' I don’t think it was the model I was on, as it is giving the same error with SD 1.5.

What is that about "-vidia-cudnn-cu11" being an invalid distribution?

From my understanding, when pip is installing a package, it removes the first letter of the package name. When the installation completes, this extra file is removed. When the install crashes, the extra file isn’t removed and is left as a fake package that is ignored. It’s fine to leave it, but you can also navigate to venv\Lib\site-packages to remove it.

(I had a “-rotobuf” warning myself :p)

I can see that the packages are installed that our extension needs, but cudnn is still installed. The extension won’t uninstall it because it only checks if tensorrt is installed, and if it is, it won’t uninstall cudnn… might need a code change here. You can try to run the following in the venv of automatic1111 :

python -m pip uninstall -y nvidia-cudnn-cu11

I try anything beyond AVG person and I have a lot of technical knowledge and this S**t doesn’t work it just doesn’t work period.

i have auto installer working and a video

https://www.patreon.com/posts/86438018

https://youtu.be/eKnMVXVjVoU

I try anything beyond AVG person and I have a lot of technical knowledge and this S**t doesn’t work it just doesn’t work period.

I THINK this problem is solved for me! However, I am still getting an incompatibility with OpenPose, e.g. the “found at least two devices, cpu and cuda:0!” message. That was the initial message that led me to this (because of something to do with xformers?), which still remains unsolved.

Yes, it will not work with ControlNet

Can you look in c:\users\MYNAME\appdata\local\programs\python\python310\lib\site-packages and see if you have a rotobuf folder there, and delete it if so. You can also look in sd.webui\system\python\Lib\site-packages and check the same. The local version of site-packages should be getting used, but you may be pulling things in from your main Python installation which is causing issues.

Thats actually good to know. but aparently i got it installed now. in the portable version. i havent tried this in the other version i have installed. activate.bat in venv/scripts/ running pip upgrade command to 23.3 version

pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir pip uninstall -y nvidia-cudnn-cu11

then i ran webui.bat and installed the extension from url used the restart button in the ui and it started without errors this time.

I deleted the venv folder before i ran all this and the extension, waited for it all to download again and ran the above commands

You saved me after half a day banging my head! I’m usually reading through these threads months/years after they’ve been archived, not while people are figuring shit out…so just know I really appreciate this!

Can you look in c:\users\MYNAME\appdata\local\programs\python\python310\lib\site-packages and see if you have a rotobuf folder there, and delete it if so.

You can also look in sd.webui\system\python\Lib\site-packages and check the same.

The local version of site-packages should be getting used, but you may be pulling things in from your main Python installation which is causing issues.

Thats actually good to know. but aparently i got it installed now. in the portable version. i havent tried this in the other version i have installed. activate.bat in venv/scripts/ running pip upgrade command to 23.3 version

pip install nvidia-cudnn-cu11==8.9.4.25 --no-cache-dir pip install --pre --extra-index-url https://pypi.nvidia.com/ tensorrt==9.0.1.post11.dev4 --no-cache-dir pip uninstall -y nvidia-cudnn-cu11

then i ran webui.bat and installed the extension from url used the restart button in the ui and it started without errors this time.

I deleted the venv folder before i ran all this and the extension, waited for it all to download again and ran the above commands

I got the uninstall -> reinstall error fixed. Uninstalled tensorrt from terminal, did pip cache purge (this was the cure), reinstalled tensorrt, now the tab is back.

can you give the commands you input to do it?

In webui root directory:

venv\scripts\activate.bat

pip uninstall tensorrt

pip cache purge

pip install --pre --extra-index-url https://pypi.nvidia.com tensorrt==9.0.1.post11.dev4

pip uninstall -y nvidia-cudnn-cu11

If I change the vae before generating image it seems to work. BTW it’s not faster 😦

Export finished, and properly added to SD Unet dropdown. I can now use the TensorRT model, and it is indeed much faster on my 3060.

So it seems my errors were mostly caused by --medvram (which I still have not re-enabled), possibly conflicting with another extension (Photoshop plugin) using the --api. I had to manually uninstall cudnn via python -m pip uninstall -y nvidia-cudnn-cu11.

After all that, it seems to be working. I will try to reenable --medvram and --api commandline args and see if it still works.

Ok, I’m going to delete everything in the models/Unet-trt and models/Unet-onnx folders, reboot webui server, and try exporting default engine again. Hopefully this time the model.json file is created correctly.

i also foudn that if i remove the models i converted then the extention disappear from the UI. restoring those deleted files will make the extention appear again… wtf lol

Ok, so I removed medvram, and that did seem to be causing the problem with the tensors being on the same device. But now I get a bunch of other errors when trying to export default engine:

{'sample': [(1, 4, 64, 64), (2, 4, 64, 64), (8, 4, 96, 96)], 'timesteps': [(1,), (2,), (8,)], 'encoder_hidden_states': [(1, 77, 768), (2, 77, 768), (8, 154, 768)]}
Disabling attention optimization
D:\repos\stable-diffusion-webui\venv\lib\site-packages\einops\einops.py:314: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  known = {axis for axis in composite_axis if axis_name2known_length[axis] != _unknown_axis_length}
D:\repos\stable-diffusion-webui\venv\lib\site-packages\einops\einops.py:315: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  unknown = {axis for axis in composite_axis if axis_name2known_length[axis] == _unknown_axis_length}
D:\repos\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py:158: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert x.shape[1] == self.channels
D:\repos\stable-diffusion-webui\modules\sd_hijack_unet.py:26: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if a.shape[-2:] != b.shape[-2:]:
D:\repos\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py:109: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert x.shape[1] == self.channels
ERROR:asyncio:Exception in callback H11Protocol.timeout_keep_alive_handler()
handle: <TimerHandle when=7159.015 H11Protocol.timeout_keep_alive_handler()>
Traceback (most recent call last):
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_state.py", line 249, in _fire_event_triggered_transitions
    new_state = EVENT_TRIGGERED_TRANSITIONS[role][state][event_type]
KeyError: <class 'h11._events.ConnectionClosed'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\X\AppData\Local\Programs\Python\Python310\lib\asyncio\events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 383, in timeout_keep_alive_handler
    self.conn.send(event)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 468, in send
    data_list = self.send_with_data_passthrough(event)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 493, in send_with_data_passthrough
    self._process_event(self.our_role, event)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 242, in _process_event
    self._cstate.process_event(role, type(event), server_switch_event)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_state.py", line 238, in process_event
    self._fire_event_triggered_transitions(role, event_type)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_state.py", line 251, in _fire_event_triggered_transitions
    raise LocalProtocolError(
h11._util.LocalProtocolError: can't handle event type ConnectionClosed when role=SERVER and state=SEND_RESPONSE

I thought it might be the model I had selected, so I switched to the SD 1.5 model, and then got this error:

Loading model v1-5-pruned-emaonly.safetensors [6ce0161689] (2 out of 3)
Loading weights [6ce0161689] from D:\repos\stable-diffusion-webui\models\Stable-diffusion\v1-5-pruned-emaonly.safetensors
Creating model from config: D:\repos\stable-diffusion-webui\configs\v1-inference.yaml
============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

[W] 'colored' module is not installed, will not use colors when logging. To enable colors, please install the 'colored' module: python3 -m pip install colored
[E] ONNX-Runtime is not installed, so constant folding may be suboptimal or not work at all.
    Consider installing ONNX-Runtime: D:\repos\stable-diffusion-webui\venv\Scripts\python.exe -m pip install onnxruntime
[I] Folding Constants | Pass 1
[!] Module: 'onnxruntime.tools.symbolic_shape_infer' is required but could not be imported.
    Note: Error was: No module named 'onnxruntime'
    You can set POLYGRAPHY_AUTOINSTALL_DEPS=1 in your environment variables to allow Polygraphy to automatically install missing modules.
[W] Falling back to `onnx.shape_inference` because `onnxruntime.tools.symbolic_shape_infer` either could not be loaded or did not run successfully.
    Note that using ONNX-Runtime for shape inference may be faster and require less memory.
    Consider installing ONNX-Runtime or setting POLYGRAPHY_AUTOINSTALL_DEPS=1 in your environment variables to allow Polygraphy to do so automatically.
[W] Attempting to run shape inference on a large model (1641.0 MiB). This may require a large amount of memory.
    If memory consumption becomes too high, the process may be killed. You may want to try disabling shape inference in that case.
[I]     Total Nodes | Original:  8992, After Folding:  6216 |  2776 Nodes Folded
[I] Folding Constants | Pass 2
[W] colored module is not installed, will not use colors when logging. To enable colors, please install the colored module: python3 -m pip install colored
[W] Inference failed. You may want to try enabling partitioning to see better results. Note: Error was:
No module named 'onnxruntime'
[!] Module: 'onnxruntime.tools.symbolic_shape_infer' is required but could not be imported.
    Note: Error was: No module named 'onnxruntime'
    You can set POLYGRAPHY_AUTOINSTALL_DEPS=1 in your environment variables to allow Polygraphy to automatically install missing modules.
[W] Attempting to run shape inference on a large model (1642.0 MiB). This may require a large amount of memory.
    If memory consumption becomes too high, the process may be killed. You may want to try disabling shape inference in that case.
[I]     Total Nodes | Original:  6216, After Folding:  6152 |    64 Nodes Folded
[I] Folding Constants | Pass 3
[W] colored module is not installed, will not use colors when logging. To enable colors, please install the colored module: python3 -m pip install colored
[W] Inference failed. You may want to try enabling partitioning to see better results. Note: Error was:
No module named 'onnxruntime'
[!] Module: 'onnxruntime.tools.symbolic_shape_infer' is required but could not be imported.
    Note: Error was: No module named 'onnxruntime'
    You can set POLYGRAPHY_AUTOINSTALL_DEPS=1 in your environment variables to allow Polygraphy to automatically install missing modules.
[I]     Total Nodes | Original:  6152, After Folding:  6152 |     0 Nodes Folded
*** API error: POST: http://127.0.0.1:7860/api/predict {'error': 'LocalProtocolError', 'detail': '', 'body': '', 'errors': "Can't send data when our state is ERROR"}
    Traceback (most recent call last):
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
        await self.app(scope, receive, _send)
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 109, in __call__
        await response(scope, receive, send)
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 270, in __call__
        async with anyio.create_task_group() as task_group:
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 662, in __aexit__
        raise exceptions[0]
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 273, in wrap
        await func()
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 134, in stream_response
        return await super().stream_response(send)
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 255, in stream_response
        await send(
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 159, in _send
        await send(message)
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 512, in send
        output = self.conn.send(event)
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 468, in send
        data_list = self.send_with_data_passthrough(event)
      File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 483, in send_with_data_passthrough
        raise LocalProtocolError("Can't send data when our state is ERROR")
    h11._util.LocalProtocolError: Can't send data when our state is ERROR

---
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 428, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\fastapi\applications.py", line 273, in __call__
    await super().__call__(scope, receive, send)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 109, in __call__
    await response(scope, receive, send)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 270, in __call__
    async with anyio.create_task_group() as task_group:
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 662, in __aexit__
    raise exceptions[0]
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 273, in wrap
    await func()
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\base.py", line 134, in stream_response
    return await super().stream_response(send)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\responses.py", line 255, in stream_response
    await send(
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\starlette\middleware\errors.py", line 159, in _send
    await send(message)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 512, in send
    output = self.conn.send(event)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 468, in send
    data_list = self.send_with_data_passthrough(event)
  File "D:\repos\stable-diffusion-webui\venv\lib\site-packages\h11\_connection.py", line 483, in send_with_data_passthrough
    raise LocalProtocolError("Can't send data when our state is ERROR")
h11._util.LocalProtocolError: Can't send data when our state is ERROR
'AsyncRequest' object has no attribute '_json_response_data'
Applying attention optimization: sdp-no-mem... done.
Model loaded in 112.6s (load weights from disk: 15.3s, create model: 0.5s, apply weights to model: 96.5s).

For my part, the python -m pip uninstall -y nvidia-cudnn-cu11 didn’t seem to work as the extension was “not installed”. So instead I went to venv\Lib\site-packages and I removed the cudnn.dist-info & the cudnn folder in the nvidia folder. It seems to be working fine. At least, I can start without errors, and I can start generating the engine. It seems to be generating without issues now.

Have you exported the engine and enabled it? See here: https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT#how-to-use

Yes, I was currently following the instructions actually, to see if “seems to be working” could be converted into “is working”. I can confirm it works flawlessly! I was able to export an engine and confirm that it increases my generation speed by ~40%!

In summary, for me the issue was that during the installation, somehow it didn’t uninstall cudnn, and cudnn was taking priority over the dlls in tensorrt. After removing cudnn manually, my problem was solved.

I shouldn’t have to install anything manually… Why are there separate cudnn libraries? They seem to be conflicting… Or did TensorRT fail to remove the package after installing the tensorrt wheel?