vllm: [Bug]: Error while importing vllm since v0.6.6

Your current environment

The output of `python collect_env.py`
>>> import vllm
Warning: Your installation of OpenCV appears to be broken: module 'cv2.dnn' has no attribute 'DictValue'.Please follow the instructions at https://github.com/opencv/opencv-python/issues/884 to correct your environment. The import of cv2 has been skipped.
WARNING 01-02 10:53:16 cuda.py:32] You are using a deprecated `pynvml` package. Please install `nvidia-ml-py` instead, and make sure to uninstall `pynvml`. When both of them are installed, `pynvml` will take precedence and cause errors. See https://pypi.org/project/pynvml for more information.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.10/dist-packages/vllm/__init__.py", line 3, in <module>
    from vllm.engine.arg_utils import AsyncEngineArgs, EngineArgs
  File "/usr/local/lib/python3.10/dist-packages/vllm/engine/arg_utils.py", line 11, in <module>
    from vllm.config import (CacheConfig, CompilationConfig, ConfigFormat,
  File "/usr/local/lib/python3.10/dist-packages/vllm/config.py", line 22, in <module>
    from vllm.model_executor.layers.quantization import (QUANTIZATION_METHODS,
  File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/__init__.py", line 1, in <module>
    from vllm.model_executor.parameter import (BasevLLMParameter,
  File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/parameter.py", line 7, in <module>
    from vllm.distributed import get_tensor_model_parallel_rank
  File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/__init__.py", line 1, in <module>
    from .communication_op import *
  File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/communication_op.py", line 6, in <module>
    from .parallel_state import get_tp_group
  File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/parallel_state.py", line 38, in <module>
    import vllm.distributed.kv_transfer.kv_transfer_agent as kv_transfer
  File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/kv_transfer/kv_transfer_agent.py", line 15, in <module>
    from vllm.distributed.kv_transfer.kv_connector.factory import (
  File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/kv_transfer/kv_connector/factory.py", line 3, in <module>
    from .base import KVConnectorBase
  File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/kv_transfer/kv_connector/base.py", line 14, in <module>
    from vllm.sequence import IntermediateTensors
  File "/usr/local/lib/python3.10/dist-packages/vllm/sequence.py", line 18, in <module>
    from vllm.multimodal import MultiModalDataDict, MultiModalPlaceholderDict
  File "/usr/local/lib/python3.10/dist-packages/vllm/multimodal/__init__.py", line 6, in <module>
    from .registry import MultiModalRegistry
  File "/usr/local/lib/python3.10/dist-packages/vllm/multimodal/registry.py", line 19, in <module>
    from .video import VideoPlugin
  File "/usr/local/lib/python3.10/dist-packages/vllm/multimodal/video.py", line 4, in <module>
    import cv2
  File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 181, in <module>
    bootstrap()
  File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 175, in bootstrap
    if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
  File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
    py_module = importlib.import_module(module_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/local/lib/python3.10/dist-packages/cv2/typing/__init__.py", line 171, in <module>
    LayerId = cv2.dnn.DictValue
AttributeError: partially initialized module 'cv2' has no attribute 'dnn' (most likely due to a circular import)

Model Input Dumps

No response

šŸ› Describe the bug

Since v0.6.6, simply running import vllm results in the following error due to an issue with cv2.

error message
>>> import vllm
Warning: Your installation of OpenCV appears to be broken: module 'cv2.dnn' has no attribute 'DictValue'.Please follow the instructions at https://github.com/opencv/opencv-python/issues/884 to correct your environment. The import of cv2 has been skipped.
WARNING 01-02 10:53:16 cuda.py:32] You are using a deprecated `pynvml` package. Please install `nvidia-ml-py` instead, and make sure to uninstall `pynvml`. When both of them are installed, `pynvml` will take precedence and cause errors. See https://pypi.org/project/pynvml for more information.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.10/dist-packages/vllm/__init__.py", line 3, in <module>
    from vllm.engine.arg_utils import AsyncEngineArgs, EngineArgs
  File "/usr/local/lib/python3.10/dist-packages/vllm/engine/arg_utils.py", line 11, in <module>
    from vllm.config import (CacheConfig, CompilationConfig, ConfigFormat,
  File "/usr/local/lib/python3.10/dist-packages/vllm/config.py", line 22, in <module>
    from vllm.model_executor.layers.quantization import (QUANTIZATION_METHODS,
  File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/__init__.py", line 1, in <module>
    from vllm.model_executor.parameter import (BasevLLMParameter,
  File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/parameter.py", line 7, in <module>
    from vllm.distributed import get_tensor_model_parallel_rank
  File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/__init__.py", line 1, in <module>
    from .communication_op import *
  File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/communication_op.py", line 6, in <module>
    from .parallel_state import get_tp_group
  File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/parallel_state.py", line 38, in <module>
    import vllm.distributed.kv_transfer.kv_transfer_agent as kv_transfer
  File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/kv_transfer/kv_transfer_agent.py", line 15, in <module>
    from vllm.distributed.kv_transfer.kv_connector.factory import (
  File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/kv_transfer/kv_connector/factory.py", line 3, in <module>
    from .base import KVConnectorBase
  File "/usr/local/lib/python3.10/dist-packages/vllm/distributed/kv_transfer/kv_connector/base.py", line 14, in <module>
    from vllm.sequence import IntermediateTensors
  File "/usr/local/lib/python3.10/dist-packages/vllm/sequence.py", line 18, in <module>
    from vllm.multimodal import MultiModalDataDict, MultiModalPlaceholderDict
  File "/usr/local/lib/python3.10/dist-packages/vllm/multimodal/__init__.py", line 6, in <module>
    from .registry import MultiModalRegistry
  File "/usr/local/lib/python3.10/dist-packages/vllm/multimodal/registry.py", line 19, in <module>
    from .video import VideoPlugin
  File "/usr/local/lib/python3.10/dist-packages/vllm/multimodal/video.py", line 4, in <module>
    import cv2
  File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 181, in <module>
    bootstrap()
  File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 175, in bootstrap
    if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
  File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
    py_module = importlib.import_module(module_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/local/lib/python3.10/dist-packages/cv2/typing/__init__.py", line 171, in <module>
    LayerId = cv2.dnn.DictValue
AttributeError: partially initialized module 'cv2' has no attribute 'dnn' (most likely due to a circular import)

vllm was installed using pip install vllm inside a Docker container.

docker run --gpus all -it --ipc=host nvcr.io/nvidia/pytorch:23.10-py3

While there may be workarounds to make it work, it would be better to ensure that pip install vllm works without modifications(for future versions), as it did in previous versions.

Before submitting a new issue…

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 18 (9 by maintainers)

Most upvoted comments

FYI, installing opencv-python-headless==4.8.0.74 works fine.

@ywang96 Yes, it worked with nvidia/cuda:12.6.3-cudnn-devel-ubuntu22.04. I’ll close this issue, as everything seems clear now. However, please consider resolving the dependency issue with containers that have pytorch installed(nvcr.io/nvidia/pytorch:23.10-py3), because I believe it’s more common container being used and it’s also written in the official installation documentation. Thank you!