unsloth: [FIXED] Exception: data did not match any variant of untagged enum ModelWrapper at line 1251003 column 3
I get this error:
Traceback (most recent call last):
File "/home/denis/Documents/ai/unsloth/llama3-chat-template.py", line 20, in <module>
model, tokenizer = FastLanguageModel.from_pretrained(
File "/home/denis/miniconda3/envs/pytorch/lib/python3.10/site-packages/unsloth/models/loader.py", line 323, in from_pretrained
model, tokenizer = dispatch_model.from_pretrained(
File "/home/denis/miniconda3/envs/pytorch/lib/python3.10/site-packages/unsloth/models/llama.py", line 1610, in from_pretrained
tokenizer = load_correct_tokenizer(
File "/home/denis/miniconda3/envs/pytorch/lib/python3.10/site-packages/unsloth/tokenizer_utils.py", line 538, in load_correct_tokenizer
tokenizer = _load_correct_tokenizer(
File "/home/denis/miniconda3/envs/pytorch/lib/python3.10/site-packages/unsloth/tokenizer_utils.py", line 496, in _load_correct_tokenizer
fast_tokenizer = AutoTokenizer.from_pretrained(
File "/home/denis/miniconda3/envs/pytorch/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 897, in from_pretrained
return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
File "/home/denis/miniconda3/envs/pytorch/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2271, in from_pretrained
return cls._from_pretrained(
File "/home/denis/miniconda3/envs/pytorch/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2505, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
File "/home/denis/miniconda3/envs/pytorch/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 115, in __init__
fast_tokenizer = TokenizerFast.from_file(fast_tokenizer_file)
Exception: data did not match any variant of untagged enum ModelWrapper at line 1251003 column 3
It works with nsloth/Llama-3.2-1B-Instruct-bnb-4bit
About this issue
- Original URL
- State: open
- Created 2 years ago
- Reactions: 5
- Comments: 31 (7 by maintainers)
Oh it’s best to update transformers via
pip install --upgrade "transformers>=4.45"I am running into this same error as well when merging and exporting the 16bit model and using it on vllm. I have tried multiple models and the error is consistent. Most definitely the tokenizer exporter is broken Edit: by using latest version of docker image form vllm it now works (v0.6.2)
If I go and download the tokenizer files from the HuggingFace repository and replace them, it works.
Notebook with a working version:
@danielhanchen @katopz - here is a notebook for “offline” installation on Kaggle: (https://www.kaggle.com/code/kolyan1/offline-unsloth-package-installation-pt-2-working)
Generally one work-around is as follows:
Updating
tokenizersto latest 0.20.0 might helpI encountered the same issue as @selectorseb while deploying a finetuned Llama-3.2 model using vLLM with Docker. Initially, I faced the same problem mentioned in the original post @djannot, but after updating the vLLM Docker image, the issue was resolved.
I’m running into this same error when trying to quantize the trained models into gguf format
Exception: data did not match any variant of untagged enum ModelWrapper at line 1251003 column 3Edit: The tokenizer unsloth exports is broken.Upgrading
accelerateto version 0.34.0 will resolve this issue.this error has been fixed by upgrading the unsloth to version 2024.9.post3 and transformers to version 4.45.0
but when inference, it occurs that ‘ValueError: Invalid
cache_implementation(dynamic). Choose one of: [‘static’, ‘offloaded_static’, ‘sliding_window’, ‘hybrid’, ‘mamba’, ‘quantized’, ‘static’]’