vllm: [Bug]: msgspec.DecodeError: MessagePack data is malformed: trailing characters (byte 13)
Your current environment
vllm==0.8.0, python=3.11
🐛 Describe the bug
I keep having this decode error when using vllm=0.8.0 for qwen-2.5 inference, I have the same error on both 1.5B and 7B model once I use apply-chat-template.
msgspec.DecodeError: MessagePack data is malformed: trailing characters (byte 13)
Before submitting a new issue…
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 2
- Comments: 21 (7 by maintainers)
Thanks @j-geuter!
same.
Hi @njhill, I’m getting a similar error. Here’s my setup:
Python: 3.10.13
Env:
Model:
Qwen/Qwen2.5-Math-7B-InstructSampling Params:
where
stop_idsis just all tokens that end in\n\n, i.e.Tokenized input:
GPU: H100
Then I get the error with:
Stacktrace:
(So I’m getting a
(byte 16)instead of(byte 13).)I cannot reproduce the error given the exact same inputs however, if I re-run this in a test script (in the same venv) it goes through for some reason. Let me know if you need any other info.
@njhill I can DM you the data you can use to repro it. Just shoot me an email at kevin [at] sweep.dev. I’ve found 3 prompts where you only get an error if you run them concurrently.