sglang: [Bug] Key conflict of `AutoImageProcessor.register`
Checklist
- 1. I have searched related issues but cannot get the expected help.
- 2. The bug has not been fixed in the latest version.
- 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
- 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
- 5. Please use English, otherwise it will be closed.
Describe the bug
The following ValueError was raised when attempting to serve any model within a recent Docker container:
Traceback (most recent call last): File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/sgl-workspace/sglang/python/sglang/launch_server.py", line 6, in <module> from sglang.srt.entrypoints.http_server import launch_server File "/sgl-workspace/sglang/python/sglang/srt/entrypoints/http_server.py", line 44, in <module> from sglang.srt.entrypoints.engine import _launch_subprocesses File "/sgl-workspace/sglang/python/sglang/srt/entrypoints/engine.py", line 36, in <module> from sglang.srt.managers.data_parallel_controller import ( File "/sgl-workspace/sglang/python/sglang/srt/managers/data_parallel_controller.py", line 27, in <module> from sglang.srt.managers.io_struct import ( File "/sgl-workspace/sglang/python/sglang/srt/managers/io_struct.py", line 25, in <module> from sglang.srt.managers.schedule_batch import BaseFinishReason File "/sgl-workspace/sglang/python/sglang/srt/managers/schedule_batch.py", line 43, in <module> from sglang.srt.configs.model_config import ModelConfig File "/sgl-workspace/sglang/python/sglang/srt/configs/__init__.py", line 4, in <module> from sglang.srt.configs.qwen2_5_vl_config import ( File "/sgl-workspace/sglang/python/sglang/srt/configs/qwen2_5_vl_config.py", line 1005, in <module> AutoImageProcessor.register(Qwen2_5_VLConfig, None, Qwen2_5_VLImageProcessor, None, exist_ok=False) File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/image_processing_auto.py", line 629, in register IMAGE_PROCESSOR_MAPPING.register( File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 833, in register raise ValueError(f"'{key}' is already used by a Transformers model.") ValueError: '<class 'sglang.srt.configs.qwen2_5_vl_config.Qwen2_5_VLConfig'>' is already used by a Transformers model.
The problem was caused by ‘Qwen2_5_VLConfig’ already existing in transformers>=0.49.0, and was resolved by enabling exist_ok when registering it.
Update the file /sgl-workspace/sglang/python/sglang/srt/configs/qwen2_5_vl_config.py at line 1005~1006:
AutoImageProcessor.register(Qwen2_5_VLConfig, None, Qwen2_5_VLImageProcessor, None, exist_ok=True) AutoProcessor.register(Qwen2_5_VLConfig, Qwen2_5_VLProcessor, exist_ok=True)
Reproduction
python3 -m sglang.launch_server --model [any-model]
Environment
Docker image tag: v0.4.3.post3-cu125 sglang version: 0.4.3.post3 sgl-kernel: 0.0.3.post6 Python version: 3.10.12
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 22 (7 by maintainers)
Please downgrade to
transformers==4.48.3. We are currently in the process of supporting the new transformers version and are aware of this bugI am having the same issue for the latest mistral model that was released a couple of days ago:
When running this:
I get this error
Using these versions:
If I upgrade transformers, then I get the other issue:
hey @groklab can you please pull sglang main from the latest source and give it a try(do not switch to v0.4.4.post1):
I am guessing there’s some issues when you use the v0.4.4.post1 branch. will look into it more soon, but this should work. Please ping if it raises an error.