exllamav2: Windows prebuilt install: DLL load failed while importing exllamav2_ext

Hi there,

Library works great on Linux. On Windows, I have CUDA runtime 11.8 and Python 3.10 in my conda env, so I install the relevant pre-built binary:

pip install https://github.com/turboderp/exllamav2/releases/download/v0.0.6/exllamav2-0.0.6+cu118-cp310-cp310-win_amd64.whl

However, when I import the library, I get:

Python 3.10.13 | packaged by Anaconda, Inc. | (main, Sep 11 2023, 13:24:38) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import exllamav2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\mikel\anaconda3\envs\sniper\lib\site-packages\exllamav2\__init__.py", line 3, in <module>
    from exllamav2.model import ExLlamaV2
  File "C:\Users\mikel\anaconda3\envs\sniper\lib\site-packages\exllamav2\model.py", line 11, in <module>
    from exllamav2.cache import ExLlamaV2Cache
  File "C:\Users\mikel\anaconda3\envs\sniper\lib\site-packages\exllamav2\cache.py", line 2, in <module>
    from exllamav2.ext import exllamav2_ext as ext_c
  File "C:\Users\mikel\anaconda3\envs\sniper\lib\site-packages\exllamav2\ext.py", line 14, in <module>
    import exllamav2_ext
ImportError: DLL load failed while importing exllamav2_ext: The specified procedure could not be found.

Any ideas what this could be? Thanks a lot!

About this issue

Original URL
State: closed
Created 8 months ago
Comments: 19 (7 by maintainers)

Most upvoted comments

19:01:54-799261 ERROR Failed to load the model. Traceback (most recent call last): File “C:\Users\Mst Eshita Khatun\text-generation-webui\modules\ui_model_menu.py”, line 213, in load_model_wrapper shared.model, shared.tokenizer = load_model(selected_model, loader) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “C:\Users\Mst Eshita Khatun\text-generation-webui\modules\models.py”, line 87, in load_model output = load_func_maploader ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File “C:\Users\Mst Eshita Khatun\text-generation-webui\modules\models.py”, line 387, in ExLlamav2_HF_loader from modules.exllamav2_hf import Exllamav2HF File “C:\Users\Mst Eshita Khatun\text-generation-webui\modules\exllamav2_hf.py”, line 7, in <module> from exllamav2 import ( File “C:\Conda\envs\eshi\Lib\site-packages\exllamav2_init_.py”, line 3, in <module> from exllamav2.model import ExLlamaV2 File “C:\Conda\envs\eshi\Lib\site-packages\exllamav2\model.py”, line 16, in <module> from exllamav2.config import ExLlamaV2Config File “C:\Conda\envs\eshi\Lib\site-packages\exllamav2\config.py”, line 2, in <module> from exllamav2.fasttensors import STFile File “C:\Conda\envs\eshi\Lib\site-packages\exllamav2\fasttensors.py”, line 5, in <module> from exllamav2.ext import exllamav2_ext as ext_c File “C:\Conda\envs\eshi\Lib\site-packages\exllamav2\ext.py”, line 15, in <module> import exllamav2_ext ImportError: DLL load failed while importing exllamav2_ext: The specified procedure could not be found. How to solve that? I tried above solution it doesnt work for me

Eshita66 on Jan 31, 2024

It works now. Thank you. That may have been the source of some of the problems people had here. This was what I did:

pip uninstall torch
pip cache purge
pip install torch --index-url https://download.pytorch.org/whl/cu121

killfrenzy96 on Dec 29, 2023

this is back. I don’t understand why it keeps coming back! I had the same problem for new users of lollms. They complain having this same error again. It was solved.

If it helps, people who installed old versions then upgraded don’t have this problem !

ParisNeo on Jan 31, 2024

I had to upgrade torchvision also:

pip uninstall torch torchvision torchaudio
pip cache purge
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

ricklove on Jan 4, 2024