vllm: [Bug]: undefined symbol: __nvJitLinkComplete_12_4, version libnvJitLink.so.12
Your current environment
i cannot execute collect_env.py because of this error.
in my another environment: torch is 2.4.0 and the version of vllm is 0.6.3.post1 which works fine.
Model Input Dumps
No response
🐛 Describe the bug
following installation guide: https://docs.vllm.ai/en/stable/getting_started/installation.html#install-the-latest-code
vllm version: 0.6.3.post2.dev386+g0b8bb86b
however, it forces the installation of torch to be 2.5.1
which causes the error :
Traceback (most recent call last): File “/home/ubuntu/vllm/collect_env.py”, line 15, in <module> from vllm.envs import environment_variables File “/home/ubuntu/vllm/vllm/init.py”, line 3, in <module> from vllm.engine.arg_utils import AsyncEngineArgs, EngineArgs File “/home/ubuntu/vllm/vllm/engine/arg_utils.py”, line 8, in <module> import torch File “/opt/conda/envs/vllmsource/lib/python3.11/site-packages/torch/init.py”, line 367, in <module> from torch._C import * # noqa: F403 ^^^^^^^^^^^^^^^^^^^^^^ ImportError: /opt/conda/envs/vllmsource/lib/python3.11/site-packages/torch/lib/…/…/nvidia/cusparse/lib/libcusparse.so.12: undefined symbol: __nvJitLinkComplete_12_4, version libnvJitLink.so.12
Before submitting a new issue…
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.
About this issue
- Original URL
- State: open
- Created 2 years ago
- Reactions: 7
- Comments: 23
it can solve the bug: export LD_LIBRARY_PATH=/data/home/user/anaconda3/envs/vllm/lib/python3.10 /site-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH
unset LD_LIBRARY_PATHworks for me!Same issue for vllm==v0.6.4.post1 and torch==2.5.1. Cuda version 12.1.
Same issue, vllm==0.6.4 and torch==2.5.1
The fix is in progress, meanwhile here is a quick workaround:
patchelf --add-rpath '$ORIGIN/../../nvidia/nvjitlink/lib' /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cuda.soRef - https://github.com/pytorch/pytorch/issues/140797在2025年2月8日,测试这个解决方案成功。我的情况与vllm无关,单纯使用
python -m venv test01创建虚拟环境,然后报上面的错。使用miniconda无误。related issue: https://github.com/pytorch/pytorch/issues/111469
A very useful method. For pip users, the nvidia path may at /lib/python3.10/site-packages/nvidia/nvjitlink/lib