vllm: [Bug]: undefined symbol: __nvJitLinkComplete_12_4, version libnvJitLink.so.12

Your current environment

i cannot execute collect_env.py because of this error.

in my another environment: torch is 2.4.0 and the version of vllm is 0.6.3.post1 which works fine.

Model Input Dumps

No response

🐛 Describe the bug

following installation guide: https://docs.vllm.ai/en/stable/getting_started/installation.html#install-the-latest-code

vllm version: 0.6.3.post2.dev386+g0b8bb86b

however, it forces the installation of torch to be 2.5.1

which causes the error :

Traceback (most recent call last): File “/home/ubuntu/vllm/collect_env.py”, line 15, in <module> from vllm.envs import environment_variables File “/home/ubuntu/vllm/vllm/init.py”, line 3, in <module> from vllm.engine.arg_utils import AsyncEngineArgs, EngineArgs File “/home/ubuntu/vllm/vllm/engine/arg_utils.py”, line 8, in <module> import torch File “/opt/conda/envs/vllmsource/lib/python3.11/site-packages/torch/init.py”, line 367, in <module> from torch._C import * # noqa: F403 ^^^^^^^^^^^^^^^^^^^^^^ ImportError: /opt/conda/envs/vllmsource/lib/python3.11/site-packages/torch/lib/…/…/nvidia/cusparse/lib/libcusparse.so.12: undefined symbol: __nvJitLinkComplete_12_4, version libnvJitLink.so.12

Before submitting a new issue…

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Reactions: 7
  • Comments: 23

Most upvoted comments

it can solve the bug: export LD_LIBRARY_PATH=/data/home/user/anaconda3/envs/vllm/lib/python3.10 /site-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH

unset LD_LIBRARY_PATH works for me!

Same issue for vllm==v0.6.4.post1 and torch==2.5.1. Cuda version 12.1.

Same issue, vllm==0.6.4 and torch==2.5.1

The fix is in progress, meanwhile here is a quick workaround:

patchelf --add-rpath '$ORIGIN/../../nvidia/nvjitlink/lib' /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cuda.so Ref - https://github.com/pytorch/pytorch/issues/140797

it can solve the bug: export LD_LIBRARY_PATH=/data/home/user/anaconda3/envs/vllm/lib/python3.10 /site-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH

在2025年2月8日,测试这个解决方案成功。我的情况与vllm无关,单纯使用 python -m venv test01 创建虚拟环境,然后报上面的错。使用miniconda无误。

it can solve the bug: export LD_LIBRARY_PATH=/data/home/user/anaconda3/envs/vllm/lib/python3.10 /site-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH

A very useful method. For pip users, the nvidia path may at /lib/python3.10/site-packages/nvidia/nvjitlink/lib