AutoGPTQ: Problems with cQIGen on windows

Hi. I’ve tried to use AutoGPTQForCausalLM and got the following error: ModuleNotFoundError: No module named 'cQIGen'

And I couldn’t find anything about that module in the internet. What is also confusing, in the setup.py of the project there is a code:

if platform.system() != 'Windows': extensions.append( cpp_extension.CppExtension( "cQIGen"

But in the qlinear_qigen: try: import cQIGen as qinfer except ImportError: logger.error('cQIGen not installed.') raise

So, it’s not supposed to install it on windows, but without it that module can’t be used.

How to solve that issue?

About this issue

Original URL
State: closed
Created 8 months ago
Comments: 16

Most upvoted comments

For me, it worked going to Lib/site-packages/auto_gptq/utils/peft_utils.py and changing the following lines (21-26)

from ..nn_modules.qlinear.qlinear_qigen import QuantLinear as QuantLinearQigen
from ..nn_modules.qlinear.qlinear_triton import QuantLinear as QuantLinearTriton

 LinearLayer = Union[torch.nn.Linear, GeneralQuantLinear, QuantLinearCuda,
    QuantLinearCudaOld, QuantLinearExllama, QuantLinearQigen ,QuantLinearTriton]

import platform
if platform.system() != 'Windows':
  from ..nn_modules.qlinear.qlinear_qigen import QuantLinear as QuantLinearQigen
  from ..nn_modules.qlinear.qlinear_triton import QuantLinear as QuantLinearTriton
  LinearLayer = Union[torch.nn.Linear, GeneralQuantLinear, QuantLinearCuda,
    QuantLinearCudaOld, QuantLinearExllama, QuantLinearQigen ,QuantLinearTriton]
else:
  LinearLayer = Union[torch.nn.Linear, GeneralQuantLinear, QuantLinearCuda,
    QuantLinearCudaOld, QuantLinearExllama]

and then commenting everytime QuantLinearQigen and QuantLinearTriton appear in that .py

AlbertoAltozano on Nov 6, 2023

For me, it worked going to Lib/site-packages/auto_gptq/utils/peft_utils.py and changing the following lines (21-26)

from ..nn_modules.qlinear.qlinear_qigen import QuantLinear as QuantLinearQigen
from ..nn_modules.qlinear.qlinear_triton import QuantLinear as QuantLinearTriton

 LinearLayer = Union[torch.nn.Linear, GeneralQuantLinear, QuantLinearCuda,
    QuantLinearCudaOld, QuantLinearExllama, QuantLinearQigen ,QuantLinearTriton]

import platform
if platform.system() != 'Windows':
  from ..nn_modules.qlinear.qlinear_qigen import QuantLinear as QuantLinearQigen
  from ..nn_modules.qlinear.qlinear_triton import QuantLinear as QuantLinearTriton
  LinearLayer = Union[torch.nn.Linear, GeneralQuantLinear, QuantLinearCuda,
    QuantLinearCudaOld, QuantLinearExllama, QuantLinearQigen ,QuantLinearTriton]
else:
  LinearLayer = Union[torch.nn.Linear, GeneralQuantLinear, QuantLinearCuda,
    QuantLinearCudaOld, QuantLinearExllama]

and then commenting everytime QuantLinearQigen and QuantLinearTriton appear in that .py

This worked for me as well, thanks!

franchukpetro on Nov 7, 2023

So basically there will no longer be direct support for Windows? depend on using the old version. or is there another method. besides wsl.

And if you are going to use WSL, what would you have to do? @qwopqwop200 Would it run everything in WSL?

LucasCrossDimitri on Nov 4, 2023

It seems qigen is not installed on Windows: https://github.com/PanQiWei/AutoGPTQ/blob/99b277a099d90d0ed0e2a61adb8aa8e609283741/setup.py#L99

@qwopqwop200 do you have context on https://github.com/PanQiWei/AutoGPTQ/commit/71d56c76d0d06735406da04d7509e8a1a1a740c8? @

fxmarty on Nov 3, 2023