exllamav2: CU12+ appears to be unsupported?
pip installing the JIT version fails so below I have pasted the error message for the build from source. I think a temporary workaround for me would be to grab a dockerfile that has CU11.8, because my rolling release distro updated cuda for me and I suspect that this is what made it stop working.
My system:
$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Fri_Sep__8_19:17:24_PDT_2023
Cuda compilation tools, release 12.3, V12.3.52
Build cuda_12.3.r12.3/compiler.33281558_0
$ uname -r
6.5.9-arch2-1
$ gcc --version
gcc (GCC) 13.2.1 20230801
$ gcc-12 --version
gcc-12 (GCC) 12.3.0
Error message build from source
python setup.py install --user
Version: 0.0.7
/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
TEST FAILED: /home/seb/.local/lib/python3.11/site-packages/ does NOT support .pth files
bad install directory or PYTHONPATH
You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from. The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
/home/seb/.local/lib/python3.11/site-packages/
and your PYTHONPATH environment variable currently contains:
''
Here are some of your options for correcting the problem:
* You can choose a different installation directory, i.e., one that is
on PYTHONPATH or supports .pth files
* You can add the installation directory to the PYTHONPATH environment
variable. (It must then also be on PYTHONPATH whenever you run
Python and want to use the package(s) you are installing.)
* You can set up the installation directory to support ".pth" files by
using one of the approaches described here:
https://setuptools.pypa.io/en/latest/deprecated/easy_install.html#custom-installation-locations
Please make the appropriate changes for your system and try again.
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching 'dni_*' found anywhere in distribution
/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning: Installing 'exllamav2.exllamav2_ext' as data is deprecated, please list it in `packages`.
!!
############################
# Package would be ignored #
############################
Python recognizes 'exllamav2.exllamav2_ext' as an importable package,
but it is not listed in the `packages` configuration of setuptools.
'exllamav2.exllamav2_ext' has been automatically added to the distribution only
because it may contain data files, but this behavior is likely to change
in future versions of setuptools (and therefore is considered deprecated).
Please make sure that 'exllamav2.exllamav2_ext' is included as a package by using
the `packages` configuration field or the proper discovery methods
(for example by using `find_namespace_packages(...)`/`find_namespace:`
instead of `find_packages(...)`/`find:`).
You can read more about "package discovery" and "data files" on setuptools
documentation page.
!!
check.warn(importable)
/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning: Installing 'exllamav2.exllamav2_ext.cpp' as data is deprecated, please list it in `packages`.
!!
############################
# Package would be ignored #
############################
Python recognizes 'exllamav2.exllamav2_ext.cpp' as an importable package,
but it is not listed in the `packages` configuration of setuptools.
'exllamav2.exllamav2_ext.cpp' has been automatically added to the distribution only
because it may contain data files, but this behavior is likely to change
in future versions of setuptools (and therefore is considered deprecated).
Please make sure that 'exllamav2.exllamav2_ext.cpp' is included as a package by using
the `packages` configuration field or the proper discovery methods
(for example by using `find_namespace_packages(...)`/`find_namespace:`
instead of `find_packages(...)`/`find:`).
You can read more about "package discovery" and "data files" on setuptools
documentation page.
!!
check.warn(importable)
/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning: Installing 'exllamav2.exllamav2_ext.cuda' as data is deprecated, please list it in `packages`.
!!
############################
# Package would be ignored #
############################
Python recognizes 'exllamav2.exllamav2_ext.cuda' as an importable package,
but it is not listed in the `packages` configuration of setuptools.
'exllamav2.exllamav2_ext.cuda' has been automatically added to the distribution only
because it may contain data files, but this behavior is likely to change
in future versions of setuptools (and therefore is considered deprecated).
Please make sure that 'exllamav2.exllamav2_ext.cuda' is included as a package by using
the `packages` configuration field or the proper discovery methods
(for example by using `find_namespace_packages(...)`/`find_namespace:`
instead of `find_packages(...)`/`find:`).
You can read more about "package discovery" and "data files" on setuptools
documentation page.
!!
check.warn(importable)
/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning: Installing 'exllamav2.exllamav2_ext.cuda.quant' as data is deprecated, please list it in `packages`.
!!
############################
# Package would be ignored #
############################
Python recognizes 'exllamav2.exllamav2_ext.cuda.quant' as an importable package,
but it is not listed in the `packages` configuration of setuptools.
'exllamav2.exllamav2_ext.cuda.quant' has been automatically added to the distribution only
because it may contain data files, but this behavior is likely to change
in future versions of setuptools (and therefore is considered deprecated).
Please make sure that 'exllamav2.exllamav2_ext.cuda.quant' is included as a package by using
the `packages` configuration field or the proper discovery methods
(for example by using `find_namespace_packages(...)`/`find_namespace:`
instead of `find_packages(...)`/`find:`).
You can read more about "package discovery" and "data files" on setuptools
documentation page.
!!
check.warn(importable)
/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning: Installing 'exllamav2.generator.filters' as data is deprecated, please list it in `packages`.
!!
############################
# Package would be ignored #
############################
Python recognizes 'exllamav2.generator.filters' as an importable package,
but it is not listed in the `packages` configuration of setuptools.
'exllamav2.generator.filters' has been automatically added to the distribution only
because it may contain data files, but this behavior is likely to change
in future versions of setuptools (and therefore is considered deprecated).
Please make sure that 'exllamav2.generator.filters' is included as a package by using
the `packages` configuration field or the proper discovery methods
(for example by using `find_namespace_packages(...)`/`find_namespace:`
instead of `find_packages(...)`/`find:`).
You can read more about "package discovery" and "data files" on setuptools
documentation page.
!!
check.warn(importable)
/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning: Installing 'exllamav2.server' as data is deprecated, please list it in `packages`.
!!
############################
# Package would be ignored #
############################
Python recognizes 'exllamav2.server' as an importable package,
but it is not listed in the `packages` configuration of setuptools.
'exllamav2.server' has been automatically added to the distribution only
because it may contain data files, but this behavior is likely to change
in future versions of setuptools (and therefore is considered deprecated).
Please make sure that 'exllamav2.server' is included as a package by using
the `packages` configuration field or the proper discovery methods
(for example by using `find_namespace_packages(...)`/`find_namespace:`
instead of `find_packages(...)`/`find:`).
You can read more about "package discovery" and "data files" on setuptools
documentation page.
!!
check.warn(importable)
/home/seb/.venvs/x/lib/python3.11/site-packages/torch/utils/cpp_extension.py:414: UserWarning: The detected CUDA version (12.3) has a minor version mismatch with the version that was used to compile PyTorch (12.1). Most likely this shouldn't be a problem.
warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
/home/seb/.venvs/x/lib/python3.11/site-packages/torch/utils/cpp_extension.py:424: UserWarning: There are no g++ version bounds defined for CUDA version 12.3
warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
Emitting ninja build file /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/11] /opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/rope.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/rope.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
FAILED: /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/rope.o
/opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/rope.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/rope.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/rope.cu".
[2/11] /opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/cache.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/cache.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
FAILED: /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/cache.o
/opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/cache.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/cache.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/cache.cu".
[3/11] /opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/rms_norm.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/rms_norm.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
FAILED: /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/rms_norm.o
/opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/rms_norm.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/rms_norm.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/rms_norm.cu".
[4/11] /opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/pack_tensor.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/pack_tensor.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
FAILED: /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/pack_tensor.o
/opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/pack_tensor.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/pack_tensor.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/pack_tensor.cu".
[5/11] /opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/q_matrix.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/q_matrix.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
FAILED: /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/q_matrix.o
/opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/q_matrix.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/q_matrix.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/q_matrix.cu".
[6/11] /opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/quantize.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/quantize.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
FAILED: /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/quantize.o
/opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/quantize.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/quantize.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/quantize.cu".
[7/11] /opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/q_gemm.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/q_gemm.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
FAILED: /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/q_gemm.o
/opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/q_gemm.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/q_gemm.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/q_gemm.cu".
[8/11] /opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/q_mlp.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/q_mlp.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
FAILED: /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/q_mlp.o
/opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/q_mlp.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/q_mlp.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/q_mlp.cu".
[9/11] /opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/lora.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/lora.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
FAILED: /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/lora.o
/opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/lora.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/lora.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/lora.cu".
[10/11] /opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/h_gemm.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/h_gemm.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
FAILED: /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/h_gemm.o
/opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/h_gemm.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/h_gemm.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/h_gemm.cu".
[11/11] /opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/q_attn.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/q_attn.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
FAILED: /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/q_attn.o
/opt/cuda/bin/nvcc -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -I/home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -I/opt/cuda/include -I/home/seb/.venvs/x/include -I/usr/include/python3.11 -c -c /home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/q_attn.cu -o /home/seb/Code/x/exllamav2/build/temp.linux-x86_64-cpython-311/exllamav2/exllamav2_ext/cuda/q_attn.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -lineinfo -O3 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=exllamav2_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/Code/x/exllamav2/exllamav2/exllamav2_ext/cuda/q_attn.cu".
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "/home/seb/.venvs/x/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 2100, in _run_ninja_build
subprocess.run(
File "/usr/lib/python3.11/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/seb/Code/x/exllamav2/setup.py", line 57, in <module>
setup(
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/__init__.py", line 87, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
self.run_command(cmd)
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.run()
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/command/install.py", line 74, in run
self.do_egg_install()
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/command/install.py", line 123, in do_egg_install
self.run_command('bdist_egg')
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
self.distribution.run_command(command)
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.run()
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/command/bdist_egg.py", line 165, in run
cmd = self.call_command('install_lib', warn_dir=0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command
self.run_command(cmdname)
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
self.distribution.run_command(command)
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.run()
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/command/install_lib.py", line 11, in run
self.build()
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/_distutils/command/install_lib.py", line 112, in build
self.run_command('build_ext')
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
self.distribution.run_command(command)
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
cmd_obj.run()
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 84, in run
_build_ext.run(self)
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
self.build_extensions()
File "/home/seb/.venvs/x/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 873, in build_extensions
build_ext.build_extensions(self)
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 466, in build_extensions
self._build_extensions_serial()
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 492, in _build_extensions_serial
self.build_extension(ext)
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
_build_ext.build_extension(self, ext)
File "/home/seb/.venvs/x/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 547, in build_extension
objects = self.compiler.compile(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/seb/.venvs/x/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 686, in unix_wrap_ninja_compile
_write_ninja_file_and_compile_objects(
File "/home/seb/.venvs/x/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 1774, in _write_ninja_file_and_compile_objects
_run_ninja_build(
File "/home/seb/.venvs/x/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 2116, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension
Errror message pip install JIT
$ python test.py
Traceback (most recent call last):
File "/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/ext.py", line 14, in <module>
import exllamav2_ext
ModuleNotFoundError: No module named 'exllamav2_ext'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/seb/.venvs/x/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 2100, in _run_ninja_build
subprocess.run(
File "/usr/lib/python3.11/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/seb/Code/x/test.py", line 1, in <module>
from lib.llm.prepare import load
File "/home/seb/Code/x/lib/llm/prepare.py", line 1, in <module>
from exllamav2 import ExLlamaV2Config, ExLlamaV2, ExLlamaV2Tokenizer, ExLlamaV2Cache
File "/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/__init__.py", line 3, in <module>
from exllamav2.model import ExLlamaV2
File "/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/model.py", line 11, in <module>
from exllamav2.cache import ExLlamaV2Cache
File "/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/cache.py", line 2, in <module>
from exllamav2.ext import exllamav2_ext as ext_c
File "/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/ext.py", line 124, in <module>
exllamav2_ext = load \
^^^^^^
File "/home/seb/.venvs/x/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 1308, in load
return _jit_compile(
^^^^^^^^^^^^^
File "/home/seb/.venvs/x/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 1710, in _jit_compile
_write_ninja_file_and_build_library(
File "/home/seb/.venvs/x/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 1823, in _write_ninja_file_and_build_library
_run_ninja_build(
File "/home/seb/.venvs/x/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 2116, in _run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error building extension 'exllamav2_ext': [1/15] /opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/pack_tensor.cu -o pack_tensor.cuda.o
FAILED: pack_tensor.cuda.o
/opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/pack_tensor.cu -o pack_tensor.cuda.o
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/pack_tensor.cu".
[2/15] /opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/rms_norm.cu -o rms_norm.cuda.o
FAILED: rms_norm.cuda.o
/opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/rms_norm.cu -o rms_norm.cuda.o
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/rms_norm.cu".
[3/15] /opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/q_matrix.cu -o q_matrix.cuda.o
FAILED: q_matrix.cuda.o
/opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/q_matrix.cu -o q_matrix.cuda.o
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/q_matrix.cu".
[4/15] /opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/rope.cu -o rope.cuda.o
FAILED: rope.cuda.o
/opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/rope.cu -o rope.cuda.o
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/rope.cu".
[5/15] /opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/cache.cu -o cache.cuda.o
FAILED: cache.cuda.o
/opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/cache.cu -o cache.cuda.o
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/cache.cu".
[6/15] /opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/quantize.cu -o quantize.cuda.o
FAILED: quantize.cuda.o
/opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/quantize.cu -o quantize.cuda.o
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/quantize.cu".
[7/15] c++ -MMD -MF sampling.o.d -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -O3 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cpp/sampling.cpp -o sampling.o
[8/15] /opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/lora.cu -o lora.cuda.o
FAILED: lora.cuda.o
/opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/lora.cu -o lora.cuda.o
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/lora.cu".
[9/15] /opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/q_attn.cu -o q_attn.cuda.o
FAILED: q_attn.cuda.o
/opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/q_attn.cu -o q_attn.cuda.o
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/q_attn.cu".
[10/15] /opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/h_gemm.cu -o h_gemm.cuda.o
FAILED: h_gemm.cuda.o
/opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/h_gemm.cu -o h_gemm.cuda.o
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/h_gemm.cu".
[11/15] /opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/q_gemm.cu -o q_gemm.cuda.o
FAILED: q_gemm.cuda.o
/opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/q_gemm.cu -o q_gemm.cuda.o
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/q_gemm.cu".
[12/15] /opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/q_mlp.cu -o q_mlp.cuda.o
FAILED: q_mlp.cuda.o
/opt/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 --compiler-options '-fPIC' -lineinfo -O3 -std=c++17 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/q_mlp.cu -o q_mlp.cuda.o
/usr/include/bits/floatn.h(86): error: invalid combination of type specifiers
typedef __float128 _Float128;
^
/usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers
typedef float _Float32;
^
/usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers
typedef double _Float64;
^
/usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers
typedef double _Float32x;
^
/usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers
typedef long double _Float64x;
^
5 errors detected in the compilation of "/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cuda/q_mlp.cu".
[13/15] c++ -MMD -MF quantize_func.o.d -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -O3 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/cpp/quantize_func.cpp -o quantize_func.o
[14/15] c++ -MMD -MF ext.o.d -DTORCH_EXTENSION_NAME=exllamav2_ext -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -I/home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/TH -isystem /home/seb/.venvs/x/lib/python3.11/site-packages/torch/include/THC -isystem /opt/cuda/include -isystem /usr/include/python3.11 -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++17 -O3 -c /home/seb/.venvs/x/lib/python3.11/site-packages/exllamav2/exllamav2_ext/ext.cpp -o ext.o
ninja: build stopped: subcommand failed.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 15 (5 by maintainers)
Microsoft broke compatibility at one point, and there’s really nothing I can do about it from my end. You’ll want to either update CUDA or downgrade Visual Studio.