AutoGPTQ: Issue with new wheels, people can't install 0.2.0 with CUDA 11.8

Awesome work on the 0.2.0 release and the wheels, PanQiWei! Thousands of new people are trying AutoGPTQ today and that is amazing.

Got an issue that’s affecting some of them:

Describe the bug People trying to run pip install auto-gptq or pip install auto-gptq==0.2.0 are getting the follow errors:

Requested auto-gptq==0.2.0 from https://files.pythonhosted.org/packages/b1/f9/97153ae5cf926f96fd37e61424a1bb58e0c9991cc220b2e17390fb8bde97/auto_gptq-0.2.0.tar.gz has inconsistent version: expected '0.2.0', but metadata has '0.2.0+cu1180'
ERROR: Could not find a version that satisfies the requirement auto-gptq==0.2.0 (from versions: 0.0.4, 0.0.5, 0.1.0, 0.2.0)

Full log:

Found existing installation: auto-gptq 0.1.0
Uninstalling auto-gptq-0.1.0:
  Successfully uninstalled auto-gptq-0.1.0
Using pip 23.1.2 from /usr/local/lib/python3.10/dist-packages/pip (python 3.10)
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting auto-gptq==0.2.0
  Using cached auto_gptq-0.2.0.tar.gz (47 kB)
  Running command python setup.py egg_info
  running egg_info
  creating /tmp/pip-pip-egg-info-d0sklosj/auto_gptq.egg-info
  writing /tmp/pip-pip-egg-info-d0sklosj/auto_gptq.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-pip-egg-info-d0sklosj/auto_gptq.egg-info/dependency_links.txt
  writing requirements to /tmp/pip-pip-egg-info-d0sklosj/auto_gptq.egg-info/requires.txt
  writing top-level names to /tmp/pip-pip-egg-info-d0sklosj/auto_gptq.egg-info/top_level.txt
  writing manifest file '/tmp/pip-pip-egg-info-d0sklosj/auto_gptq.egg-info/SOURCES.txt'
  /usr/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py:476: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  reading manifest file '/tmp/pip-pip-egg-info-d0sklosj/auto_gptq.egg-info/SOURCES.txt'
  adding license file 'LICENSE'
  writing manifest file '/tmp/pip-pip-egg-info-d0sklosj/auto_gptq.egg-info/SOURCES.txt'
  Preparing metadata (setup.py) ... done
Discarding https://files.pythonhosted.org/packages/b1/f9/97153ae5cf926f96fd37e61424a1bb58e0c9991cc220b2e17390fb8bde97/auto_gptq-0.2.0.tar.gz (from https://pypi.org/simple/auto-gptq/) (requires-python:>=3.8.0): Requested auto-gptq==0.2.0 from https://files.pythonhosted.org/packages/b1/f9/97153ae5cf926f96fd37e61424a1bb58e0c9991cc220b2e17390fb8bde97/auto_gptq-0.2.0.tar.gz has inconsistent version: expected '0.2.0', but metadata has '0.2.0+cu1180'
ERROR: Could not find a version that satisfies the requirement auto-gptq==0.2.0 (from versions: 0.0.4, 0.0.5, 0.1.0, 0.2.0)
ERROR: No matching distribution found for auto-gptq==0.2.0

Software version Example of one user with the problem:

ubuntu 22.04
nvidia/cuda:11.8.0-devel-ubuntu22.04 container which includes a CUDA Version 11.8.0

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

To Reproduce

pip install auto-gptq

Expected behavior Installs auto-gptq 0.2.0 + cu118

About this issue

Original URL
State: closed
Created a year ago
Comments: 34 (6 by maintainers)

Most upvoted comments

You are right, I should improve the warnings and set some arguments’ default value to None and reset to proper value internally if users not manually specify them.

PanQiWei on Jun 2, 2023

And is there any parameter or option to stream the output. did you implemented any generator function?

@TheFaheem auto-gptq is compatiple with hf transformer’s TextGenerationPipeline, so it’s streamer should also be used for auto-gptq’s models, but I haven’t try it yet.

PanQiWei on Jun 2, 2023

@TheBloke Hi, I can install successfully using pip install auto-gptq on both my local computer and cloud server, but I also re-implement your problem when adding environment variable CUDA_VERSION=11.8 before pip command. So I think if you also have added the environment variable, you can just remove it.

It’s not a problem for me personally. But I have had several support requests about it this morning, from people trying to use AutoGPTQ from Google Colab and Docker containers - eg @kumpulak is using Docker and @TheFaheem is using Google Colab.

I can inform users to unset CUDA_VERSION but is possible to fix whatever problem is causing this issue so this is not necessary going forward. Otherwise I expect it’s going to generate a lot of support requests. I’ve already had four messages about it this morning.

I just fix the problem that users set CUDA_VERSION when install auto-gptq, I will release a patch fix later.

PanQiWei on Jun 2, 2023