AutoGPTQ: [BUG] Installation on Win11 failed

@TheBloke Thank you for your help and contribution to the community.

Hi guys, really sorry for the missing files in v0.2.1, I just notice this problem yesterday, a new patch will be released soon!

@unk1911 OK glad you got it sorted. It’s odd that you can install the 11.8 wheel successfully when it says it won’t compile it. Maybe that failure to compile issue is a hardcoded arbitrary restriction, like they block you compiling because they think it won’t work when it actually might. Or maybe it would fail to build anyway but does work if already built. I’m not really sure.

@Iory1998 In your case it should definitely have worked to do this:

pip uninstall -y auto-gptq
set GITHUB_ACTIONS=true
pip install auto-gptq==0.2.2

or

pip uninstall -y auto-gptq
git clone https://github.com/PanQiWei/AutoGPTQ
cd AutoGPTQ
git checkout v0.2.1
set GITHUB_ACTIONS=true
pip install .

But installing the wheel is fine as well so glad it’s working for you now.

@TheBloke I am still facing the same error on pytorch 2.0.1 with cuda11.8 I managed to install it using:

wget https://github.com/PanQiWei/AutoGPTQ/releases/download/v0.2.2/auto_gptq-0.2.2+cu118-cp310-cp310-win_amd64.whl pip install auto_gptq-0.2.2+cu118-cp310-cp310-win_amd64.whl

Or you can just download the right wheel from: https://github.com/PanQiWei/AutoGPTQ/releases/ Then pip install auto_gptq-0.2.2+cu118-cp310-cp310-win_amd64.whl

Oh yeah, that’s a different problem and that’s not AutoGPTQ’s fault. It’s pytorch that reports that message.

You need to get a pytorch compiled for 12.x, or downgrade to CUDA toolkit 11.8 if you want to compile anything new. There’s nothing AutoGPTQ can do about that I think.

You can probably override that by directly installing the 11.8 pre-built wheel, maybe that does work with 12.x. That’s what oobabooga does with text-generation-webui, he just installs the wheels directly:

https://github.com/PanQiWei/AutoGPTQ/releases/download/v0.2.2/auto_gptq-0.2.2+cu117-cp310-cp310-win_amd64.whl; platform_system == "Windows"
https://github.com/PanQiWei/AutoGPTQ/releases/download/v0.2.2/auto_gptq-0.2.2+cu117-cp310-cp310-linux_x86_64.whl; platform_system == "Linux"

In his case he can do that because his installer script creates a Miniconda environment, so he knows the user is definitely running Python 3.10.

Thank you marella for this. And I didn’t see your chatdocs tool before; That’s cool 😃 I was adding a new personality that allows you to do exactly that.

You beat me to it 😃

Congratulations. This seems really cool.

Yeah there’s something really wrong with PyPi install at the moment

I am telling everyone to install from source until PanQiWei can look into this

git clone https://github.com/PanQiWei/AutoGPTQ
cd AutoGPTQ
git checkout v0.2.1
pip install .