vllm: [Bug]: Error while importing vllm since v0.6.6
Your current environment
The output of `python collect_env.py`
>>> import vllm
Warning: Your installation of OpenCV appears to be broken: module 'cv2.dnn' has no attribute 'DictValue'.Please follow the instructions at https://github.com/opencv/opencv-python/issues/884 to correct your environment. The import of cv2 has been skipped.
WARNING 01-02 10:53:16 cuda.py:32] You are using a deprecated `pynvml` package. Please install `nvidia-ml-py` instead, and make sure to uninstall `pynvml`. When both of them are installed, `pynvml` will take precedence and cause errors. See https://pypi.org/project/pynvml for more information.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.10/dist-packages/vllm/__init__.py", line 3, in <module>
from vllm.engine.arg_utils import AsyncEngineArgs, EngineArgs
File "/usr/local/lib/python3.10/dist-packages/vllm/engine/arg_utils.py", line 11, in <module>
from vllm.config import (CacheConfig, CompilationConfig, ConfigFormat,
File "/usr/local/lib/python3.10/dist-packages/vllm/config.py", line 22, in <module>
from vllm.model_executor.layers.quantization import (QUANTIZATION_METHODS,
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/__init__.py", line 1, in <module>
from vllm.model_executor.parameter import (BasevLLMParameter,
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/parameter.py", line 7, in <module>
from vllm.distributed import get_tensor_model_parallel_rank
File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/__init__.py", line 1, in <module>
from .communication_op import *
File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/communication_op.py", line 6, in <module>
from .parallel_state import get_tp_group
File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/parallel_state.py", line 38, in <module>
import vllm.distributed.kv_transfer.kv_transfer_agent as kv_transfer
File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/kv_transfer/kv_transfer_agent.py", line 15, in <module>
from vllm.distributed.kv_transfer.kv_connector.factory import (
File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/kv_transfer/kv_connector/factory.py", line 3, in <module>
from .base import KVConnectorBase
File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/kv_transfer/kv_connector/base.py", line 14, in <module>
from vllm.sequence import IntermediateTensors
File "/usr/local/lib/python3.10/dist-packages/vllm/sequence.py", line 18, in <module>
from vllm.multimodal import MultiModalDataDict, MultiModalPlaceholderDict
File "/usr/local/lib/python3.10/dist-packages/vllm/multimodal/__init__.py", line 6, in <module>
from .registry import MultiModalRegistry
File "/usr/local/lib/python3.10/dist-packages/vllm/multimodal/registry.py", line 19, in <module>
from .video import VideoPlugin
File "/usr/local/lib/python3.10/dist-packages/vllm/multimodal/video.py", line 4, in <module>
import cv2
File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 181, in <module>
bootstrap()
File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 175, in bootstrap
if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
py_module = importlib.import_module(module_name)
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.10/dist-packages/cv2/typing/__init__.py", line 171, in <module>
LayerId = cv2.dnn.DictValue
AttributeError: partially initialized module 'cv2' has no attribute 'dnn' (most likely due to a circular import)
Model Input Dumps
No response
š Describe the bug
Since v0.6.6, simply running import vllm results in the following error due to an issue with cv2.
error message
>>> import vllm
Warning: Your installation of OpenCV appears to be broken: module 'cv2.dnn' has no attribute 'DictValue'.Please follow the instructions at https://github.com/opencv/opencv-python/issues/884 to correct your environment. The import of cv2 has been skipped.
WARNING 01-02 10:53:16 cuda.py:32] You are using a deprecated `pynvml` package. Please install `nvidia-ml-py` instead, and make sure to uninstall `pynvml`. When both of them are installed, `pynvml` will take precedence and cause errors. See https://pypi.org/project/pynvml for more information.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.10/dist-packages/vllm/__init__.py", line 3, in <module>
from vllm.engine.arg_utils import AsyncEngineArgs, EngineArgs
File "/usr/local/lib/python3.10/dist-packages/vllm/engine/arg_utils.py", line 11, in <module>
from vllm.config import (CacheConfig, CompilationConfig, ConfigFormat,
File "/usr/local/lib/python3.10/dist-packages/vllm/config.py", line 22, in <module>
from vllm.model_executor.layers.quantization import (QUANTIZATION_METHODS,
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/__init__.py", line 1, in <module>
from vllm.model_executor.parameter import (BasevLLMParameter,
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/parameter.py", line 7, in <module>
from vllm.distributed import get_tensor_model_parallel_rank
File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/__init__.py", line 1, in <module>
from .communication_op import *
File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/communication_op.py", line 6, in <module>
from .parallel_state import get_tp_group
File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/parallel_state.py", line 38, in <module>
import vllm.distributed.kv_transfer.kv_transfer_agent as kv_transfer
File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/kv_transfer/kv_transfer_agent.py", line 15, in <module>
from vllm.distributed.kv_transfer.kv_connector.factory import (
File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/kv_transfer/kv_connector/factory.py", line 3, in <module>
from .base import KVConnectorBase
File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/kv_transfer/kv_connector/base.py", line 14, in <module>
from vllm.sequence import IntermediateTensors
File "/usr/local/lib/python3.10/dist-packages/vllm/sequence.py", line 18, in <module>
from vllm.multimodal import MultiModalDataDict, MultiModalPlaceholderDict
File "/usr/local/lib/python3.10/dist-packages/vllm/multimodal/__init__.py", line 6, in <module>
from .registry import MultiModalRegistry
File "/usr/local/lib/python3.10/dist-packages/vllm/multimodal/registry.py", line 19, in <module>
from .video import VideoPlugin
File "/usr/local/lib/python3.10/dist-packages/vllm/multimodal/video.py", line 4, in <module>
import cv2
File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 181, in <module>
bootstrap()
File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 175, in bootstrap
if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
py_module = importlib.import_module(module_name)
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.10/dist-packages/cv2/typing/__init__.py", line 171, in <module>
LayerId = cv2.dnn.DictValue
AttributeError: partially initialized module 'cv2' has no attribute 'dnn' (most likely due to a circular import)
vllm was installed using pip install vllm inside a Docker container.
docker run --gpus all -it --ipc=host nvcr.io/nvidia/pytorch:23.10-py3
While there may be workarounds to make it work, it would be better to ensure that pip install vllm works without modifications(for future versions), as it did in previous versions.
Before submitting a new issueā¦
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 18 (9 by maintainers)
FYI, installing
opencv-python-headless==4.8.0.74works fine.@ywang96 Yes, it worked with
nvidia/cuda:12.6.3-cudnn-devel-ubuntu22.04. Iāll close this issue, as everything seems clear now. However, please consider resolving the dependency issue with containers that have pytorch installed(nvcr.io/nvidia/pytorch:23.10-py3), because I believe itās more common container being used and itās also written in the official installation documentation. Thank you!