vllm: Build failure due to CUDA version mismatch
I failed to build the system with the latest NVIDIA PyTorch docker image. The reason is PyTorch installed by pip is built with CUDA 11.7 while the container uses CUDA 12.1.
RuntimeError:
The detected CUDA version (12.1) mismatches the version that was used to compile
PyTorch (11.7). Please make sure to use the same CUDA versions.
About this issue
- Original URL
- State: open
- Created a year ago
- Reactions: 9
- Comments: 22 (1 by maintainers)
Removing pyproject.toml may be a solution. In my case, the build system was using the version of pytorch from pyproject.toml rather than the pytorch already installed.
I have the same problem. Pytorch using cuda 11.8 installed, yet I get this error when installing.
Option 1 : You have cuda 12.1 therefore you should simply uninstall the current binaries of pytorch that you have and then reinstall it using :
pip3 install torch torchvision torchaudioOption 2 : If you do not want to use the cuda 12.1 that you have installed, you can use another version of cuda (11.7, 11.8, …). First you need to uninstall your current cuda (https://stackoverflow.com/a/56827564) and then select the one you want on the cuda install website
@WoosukKwon Thanks for the quick response! But the issue still exists after using the command above. It is weird that the detected version is 12.0 since the cudatookit installed in the env is 11.8. Looks like it will access Cuda outside the env.
I solve this by running this: conda install nvidia/label/cuda-11.8.0::cuda-nvcc
looks like your conda env has no nvcc installed, and it calls your system-based nvcc, which is not 11.8 or the cuda version you installed.
For my problem, I found the the code used /usr/bin/nvcc which will print a different version The right nvcc is in /usr/local/cuda/bin, So I delete /usr/bin/nvcc, now my code works fine.
@Joejoequ Thanks for reporting it! I think in your case, the problem can be easily solved by installing CUDA 11.8 version of PyTorch: