sglang: [Bug] Key conflict of `AutoImageProcessor.register`

Checklist

  • 1. I have searched related issues but cannot get the expected help.
  • 2. The bug has not been fixed in the latest version.
  • 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
  • 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
  • 5. Please use English, otherwise it will be closed.

Describe the bug

The following ValueError was raised when attempting to serve any model within a recent Docker container:

Traceback (most recent call last): File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/sgl-workspace/sglang/python/sglang/launch_server.py", line 6, in <module> from sglang.srt.entrypoints.http_server import launch_server File "/sgl-workspace/sglang/python/sglang/srt/entrypoints/http_server.py", line 44, in <module> from sglang.srt.entrypoints.engine import _launch_subprocesses File "/sgl-workspace/sglang/python/sglang/srt/entrypoints/engine.py", line 36, in <module> from sglang.srt.managers.data_parallel_controller import ( File "/sgl-workspace/sglang/python/sglang/srt/managers/data_parallel_controller.py", line 27, in <module> from sglang.srt.managers.io_struct import ( File "/sgl-workspace/sglang/python/sglang/srt/managers/io_struct.py", line 25, in <module> from sglang.srt.managers.schedule_batch import BaseFinishReason File "/sgl-workspace/sglang/python/sglang/srt/managers/schedule_batch.py", line 43, in <module> from sglang.srt.configs.model_config import ModelConfig File "/sgl-workspace/sglang/python/sglang/srt/configs/__init__.py", line 4, in <module> from sglang.srt.configs.qwen2_5_vl_config import ( File "/sgl-workspace/sglang/python/sglang/srt/configs/qwen2_5_vl_config.py", line 1005, in <module> AutoImageProcessor.register(Qwen2_5_VLConfig, None, Qwen2_5_VLImageProcessor, None, exist_ok=False) File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/image_processing_auto.py", line 629, in register IMAGE_PROCESSOR_MAPPING.register( File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 833, in register raise ValueError(f"'{key}' is already used by a Transformers model.") ValueError: '<class 'sglang.srt.configs.qwen2_5_vl_config.Qwen2_5_VLConfig'>' is already used by a Transformers model.

The problem was caused by ‘Qwen2_5_VLConfig’ already existing in transformers>=0.49.0, and was resolved by enabling exist_ok when registering it. Update the file /sgl-workspace/sglang/python/sglang/srt/configs/qwen2_5_vl_config.py at line 1005~1006:

AutoImageProcessor.register(Qwen2_5_VLConfig, None, Qwen2_5_VLImageProcessor, None, exist_ok=True) AutoProcessor.register(Qwen2_5_VLConfig, Qwen2_5_VLProcessor, exist_ok=True)

Reproduction

python3 -m sglang.launch_server --model [any-model]

Environment

Docker image tag: v0.4.3.post3-cu125 sglang version: 0.4.3.post3 sgl-kernel: 0.0.3.post6 Python version: 3.10.12

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 22 (7 by maintainers)

Most upvoted comments

Please downgrade to transformers==4.48.3. We are currently in the process of supporting the new transformers version and are aware of this bug

I am having the same issue for the latest mistral model that was released a couple of days ago:

When running this:

python3 -m sglang.launch_server --model-path mistralai/Mistral-Small-3.1-24B-Instruct-2503 --host 0.0.0.0 --port 30000

I get this error

KeyError: 'mistral3'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/root/sglang/python/sglang/launch_server.py", line 14, in <module>
    launch_server(server_args)
  File "/root/sglang/python/sglang/srt/entrypoints/http_server.py", line 643, in launch_server
    tokenizer_manager, scheduler_info = _launch_subprocesses(server_args=server_args)
  File "/root/sglang/python/sglang/srt/entrypoints/engine.py", line 546, in _launch_subprocesses
    tokenizer_manager = TokenizerManager(server_args, port_args)
  File "/root/sglang/python/sglang/srt/managers/tokenizer_manager.py", line 155, in __init__
    self.model_config = ModelConfig(
  File "/root/sglang/python/sglang/srt/configs/model_config.py", line 60, in __init__
    self.hf_config = get_config(
  File "/root/sglang/python/sglang/srt/hf_transformers_utils.py", line 81, in get_config
    config = AutoConfig.from_pretrained(
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 1073, in from_pretrained
    raise ValueError(
ValueError: The checkpoint you are trying to load has model type `mistral3` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+https://github.com/huggingface/transformers.git`

Using these versions:

sglang                            0.4.4.post1         /root/sglang/python
torch                             2.5.1
transformers                      4.48.3

If I upgrade transformers, then I get the other issue:

ValueError: '<class 'sglang.srt.configs.qwen2_5_vl_config.Qwen2_5_VLConfig'>' is already used by a Transformers model.

hey @groklab can you please pull sglang main from the latest source and give it a try(do not switch to v0.4.4.post1):

git clone https://github.com/sgl-project/sglang.git
cd sglang

pip install --upgrade pip
pip install -e "python[all]" --find-links https://flashinfer.ai/whl/cu124/torch2.5/flashinfer-python

I am guessing there’s some issues when you use the v0.4.4.post1 branch. will look into it more soon, but this should work. Please ping if it raises an error.