transformers: Transformers can not load dependency of tensorflow - No module named 'keras.engine'
System Info
osX silicon M1
- python 3.8.16 (also tested with newer versions e.g. 3.9)
- tensorflow 2.11.0 eigen_py39h384437f_0 (also tested with tensforflow 2.13 rc0)
tried conda and venv.
- transformers 4.28.1 also tested 4.29.1
#sample code which causes the error below
from transformers import pipeline
summarizer = pipeline("summarization")
No model was supplied, defaulted to t5-small and revision d769bba (https://huggingface.co/t5-small).
Using a pipeline without specifying a model name and revision in production is not recommended.
Traceback (most recent call last):
File "/opt/homebrew/Caskroom/miniconda/base/envs/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1146, in _get_module
return importlib.import_module("." + module_name, self.__name__)
File "/opt/homebrew/Caskroom/miniconda/base/envs/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/opt/homebrew/Caskroom/miniconda/base/envs/lib/python3.9/site-packages/transformers/models/t5/modeling_tf_t5.py", line 35, in <module>
from ...modeling_tf_utils import (
File "/opt/homebrew/Caskroom/miniconda/base/envs/lib/python3.9/site-packages/transformers/modeling_tf_utils.py", line 69, in <module>
from keras.engine import data_adapter
ModuleNotFoundError: No module named 'keras.engine'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "sumary_service.py", line 3, in <module>
summarizer = pipeline("summarization")
File "/opt/homebrew/Caskroom/miniconda/base/envs/lib/python3.9/site-packages/transformers/pipelines/__init__.py", line 779, in pipeline
framework, model = infer_framework_load_model(
File "/opt/homebrew/Caskroom/miniconda/base/envs/lib/python3.9/site-packages/transformers/pipelines/base.py", line 238, in infer_framework_load_model
_class = getattr(transformers_module, f"TF{architecture}", None)
File "/opt/homebrew/Caskroom/miniconda/base/envs/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1137, in __getattr__
value = getattr(module, name)
File "/opt/homebrew/Caskroom/miniconda/base/envs/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1136, in __getattr__
module = self._get_module(self._class_to_module[name])
File "/opt/homebrew/Caskroom/miniconda/base/envs/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 1148, in _get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.models.t5.modeling_tf_t5 because of the following error (look up to see its traceback):
No module named 'keras.engine'
Who can help?
@gante and @Rocketknight1 No response
Information
- The official example scripts
- My own modified scripts
Reproduction
- follow install example from official docs (https://huggingface.co/docs/transformers/installation)
- run sample code from model page https://huggingface.co/facebook/bart-large-cnn
**error does not occur when using pytorch**
Expected behavior
transformer library does not raise exception
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 22 (10 by maintainers)
This has now been merged - if anyone else is having compatibility issues with
transformersand TensorFlow 2.13 and finds this issue, please install transformers frommainwithpip install --upgrade git+https://github.com/huggingface/transformers.git. Once we release 2.30 (probably end of May / early June) you can go back to justpip install --upgrade transformers.If anyone is still encountering this problem after installing the latest version, please reply or reopen this issue and let us know!
the key point is tensorflow==2.12.0
I’ve opened a PR at #23663 that should cover this issue as well as future-proof against other changes. My limited testing with 2.13rc0 on my local machine looked good, but if you get the chance please try it out with
pip install --upgrade git+https://github.com/huggingface/transformers.git@tf_future_proofingcc @dcdieci @elfringham @sanderpick
There are upcoming breaking changes to keras. Please see https://github.com/keras-team/tf-keras/issues/196 Also see the release notes here https://github.com/tensorflow/tensorflow/releases/tag/v2.13.0-rc0 particularly the part in ‘Breaking Changes’ that talks about restricting access so that only public symbols are accessible. This will need updates to transformers to resolve I think.
transformers==4.34.1 tensorflow==2.14.0
Hi, I have upgraded my transformers and tensorflow but I’m still facing the error. 😕 any kind suggestions on how to proceed?
Hey all - if you update transformers to the latest version, it should work fine with the latest versions of TF and Keras as well - you shouldn’t need to pin those!
Hi everyone, I resolved these issues using the following package versions (including the full list for anyone also using these additional packages, but tensorflow, keras, datasets, transformers are most important):
We will also be making a proper release of version 4.30 later this week that should correctly support TF 2.13, so hopefully after that everyone can just
pip install --upgrade transformersand stop installing frommain.Hi! This regression was caused by a PR we merged yesterday and should be fixed as of about an hour ago. Please install the latest version from
mainand try again. Thanks again to @frostming for spotting that one so quickly!