Your current environment
raise e
File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/config.py", line 225, in get_config
config = AutoConfig.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/models/auto/configuration_auto.py", line 1073, in from_pretrained
raise ValueError(
ValueError: The checkpoint you are trying to load has model type `qwen2_5_vl` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+https://github.com/huggingface/transformers.git`
How would you like to use vllm
docker run -it --name Qwen2.5-VL-7B-Instruct \
--gpus all \
-e TZ=Asia/Shanghai \
-v /data/modelsfiles/:/root/model \
-p 8444:8000 \
--ipc=host \
vllm/vllm-openai:v0.7.2 \
--model /root/model/Qwen2.5-VL-7B-Instruct \
--tensor-parallel-size 2 \
--pipeline-parallel-size 2 \
--max-num-batched-tokens 131072 \
--max-num-seqs 4 \
--served-model-name Qwen2.5-VL-7B-Instruct
Before submitting a new issue…
first, i update transformers v4.49.0.dev0
and then: RuntimeError: Failed to import transformers.models.qwen2_5_vl.modeling_qwen2_5_vl because of the following error (look up to see its traceback): module ‘cv2.dnn’ has no attribute ‘DictValue’ i update Dockerfile
but now: ValueError: Model architectures [‘Qwen2_5_VLForConditionalGeneration’] failed to be inspected. Please check the logs for more details.
VLLM seems to have solved this problem,you can try install vllm using original code from github.
I find new bug : RuntimeError: Failed to import transformers.models.qwen2_5_vl.modeling_qwen2_5_vl because of the following error (look up to see its traceback): module ‘cv2.dnn’ has no attribute ‘DictValue’
use transformers v4.49.0.dev0 OR modify models/qwen2_5_vl.py change Qwen2_5_VLProcessor to AutoImageProcessor from transformers v4.49.0