tensorflow: h5py==3.0.0 causes issues with keras model loads in tensorflow 2.1.0

@bhack I’m not going to add any more messages after this, but I think you can see based on these other issues the Keras API is essentially broken because h5py new release. I’m not sure why the version was not pinned as it is in master, but I strongly advise that you pin to h5py==2.10.0 this for all TF >= 2.1.

We cannot pin the versions without doing a patch release. We only do patch releases for security issues.

This issue has a quick workaround: pip install tensorflow h5py<3.0.0.

We are in the process of releasing TF 2.4 which should not be affected by this issue.

FWIW, I got a similar error message with h5py 3.0, and removing .decode('utf-8') from tensorflow/python/keras/saving/hdf5_format.py allows me to load Keras models as before.

Just try pip install h5py==2.10.0

It completely worked in my case. Hope it will work for you.

Removing .decode('utf-8') on lines 176 and 190 in /usr/lib/python3.9/site-packages/tensorflow/python/keras/saving/hdf5_format.py worked for me.

pip install 'h5py<3.0.0'

there might be two separate but related issues here

That seems the case. I tried h5py 3.1.0, and the error AttributeError: 'str' object has no attribute 'decode' still happens.

Thought it might be pertinent to link to the h5py repo: This has been labeled as a bug in their 3.0.0 release (rather than a backwards incompatibility), and is slated to be fixed in an upcoming release: https://github.com/h5py/h5py/issues/1732

edit: Actually, there might be two separate but related issues here. The bug in the h5py thread occurs during the model-save codepath, but the bug in this TensorFlow thread is in the model-load codepath, which if I had to wildly guess might require a typecheck/cast on TensorFlow’s side

Downgrading to h5py worked some time ago. Now old h5py does not work and #40991 appears.

Posting here because Google search for this issue led me directly to this page. Presumably it will lead others here as well.

Ubuntu 18.04.5
pip3 install tensorflow-gpu==2.0.0

You get this same problem.

Solution was:

pip3 install h5py=2.10.0

Why do I install TF 2.0.0 in March 2021, you ask? Because the book I am studying uses TF 2.0.0 and hard experience has taught me there can be subtle differences between TF minor versions that are hard to understand for TF noobs like me.

also incapable for tensorflow3.2.1

update: I mean version 2.3.1

The recommended fix of downgrading the h5py version:

pip install 'h5py < 3.0.0'

does not work on TensorFlow 2.3 because:

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py in load_weights(self, filepath, by_name, skip_mismatch, options)
   2202           'first, then load the weights.')
   2203     self._assert_weights_created()
-> 2204     with h5py.File(filepath, 'r') as f:
   2205       if 'layer_names' not in f.attrs and 'model_weights' in f:
   2206         f = f['model_weights']

AttributeError: module 'h5py' has no attribute 'File'

Please keep in mind that we don’t update older branches unless for security reasons. So, because our bounds on dependencies were too large (our mistake, fixed now, but see also #44654), you will have to manually downgrade h5py in your installation.

You can do this in two ways:

• Either you install with an upper bound, a la:

  $ pip install tensorflow 'h5py < 3.0.0'
  

• Or you install TF and then downgrade h5py, a la:

  $ pip install tensorflow
  $ pip uninstall -y h5py
  $ pip install 'h5py < 3.0.0'
  

Alternatively, the issue should be fixed in nightly and in TF 2.4 RCs (in the sense that h5py is upper bounded to not get 3.0.0).

We will have people working on making TF work with h5py >= 3 in the future, but this will only land in TF 2.5 or later.

also incapable for tensorflow3.2.1

of course, we cannot use tf3 right now unless you come from another universe

This error is also present in tensorflow 1.15: https://stackoverflow.com/questions/64795784/google-ai-platform-unexpected-error-when-loading-the-model-str-object-has-no

the new h5py version just came out today: https://pypi.org/project/h5py/#history

Pinning it to a version ❤️.0.0 (or at least throwing a warning) is a very lightweight change with no risk.

I am facing the same issue. Is there any solution came out to fix this bug:

File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/network.py", line 1424, in load_weights saving.load_weights_from_hdf5_group(f, self.layers) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/saving/hdf5_format.py", line 711, in load_weights_from_hdf5_group original_keras_version = f.attrs['keras_version'].decode('utf8') AttributeError: 'str' object has no attribute 'decode' python-BaseException

Hey guys, when Iam trying to downgrade the version of h5py I get the follwing error:

ERROR: Failed building wheel for h5py Running setup.py clean for h5py Failed to build h5py WARNING: Ignoring invalid distribution -ip (d:\tensorflow object detection\tfodcourse\tfod\lib\site-packages) Installing collected packages: h5py Attempting uninstall: h5py WARNING: Ignoring invalid distribution -ip (d:\tensorflow object detection\tfodcourse\tfod\lib\site-packages) Found existing installation: h5py 3.1.0 Uninstalling h5py-3.1.0: Successfully uninstalled h5py-3.1.0 Running setup.py install for h5py … error ERROR: Command errored out with exit status 1: command: ‘d:\tensorflow object detection\tfodcourse\tfod\scripts\python.exe’ -u -c ‘import io, os, sys, setuptools, tokenize; sys.argv[0] = "C:\Users\Arne’“'”‘s PC\AppData\Local\Temp\pip-install-8sioo065\h5py_f1f64cd207fd4e3fbbc73e1ddc4b40d2\setup.py"; file="C:\Users\Arne’“'”‘s PC\AppData\Local\Temp\pip-install-8sioo065\h5py_f1f64cd207fd4e3fbbc73e1ddc4b40d2\setup.py";f = getattr(tokenize, ‘"’“‘open’”’“‘, open)(file) if os.path.exists(file) else io.StringIO(’”‘“‘from setuptools import setup; setup()’”’“‘);code = f.read().replace(’”‘"’\r\n’“'”‘, ‘"’"’\n’“'”‘);f.close();exec(compile(code, file, ‘"’“‘exec’”’“‘))’ install --record ‘C:\Users\Arne’”'"‘s PC\AppData\Local\Temp\pip-record-prai98me\install-record.txt’ --single-version-externally-managed --compile --install-headers ‘d:\tensorflow object detection\tfodcourse\tfod\include\site\python3.9\h5py’ cwd: C:\Users\Arne’s PC\AppData\Local\Temp\pip-install-8sioo065\h5py_f1f64cd207fd4e3fbbc73e1ddc4b40d2\

Rolling back uninstall of h5py Moving to d:\tensorflow object detection\tfodcourse\tfod\lib\site-packages\h5py-3.1.0.dist-info
from D:\Tensorflow Object Detection\TFODCourse\tfod\Lib\site-packages~5py-3.1.0.dist-info Moving to d:\tensorflow object detection\tfodcourse\tfod\lib\site-packages\h5py
from D:\Tensorflow Object Detection\TFODCourse\tfod\Lib\site-packages~5py ERROR: Command errored out with exit status 1: ‘d:\tensorflow object detection\tfodcourse\tfod\scripts\python.exe’ -u -c ‘import io, os, sys, setuptools, tokenize; sys.argv[0] = "C:\Users\Arne’“'”‘s PC\AppData\Local\Temp\pip-install-8sioo065\h5py_f1f64cd207fd4e3fbbc73e1ddc4b40d2\setup.py"; file="C:\Users\Arne’“'”‘s PC\AppData\Local\Temp\pip-install-8sioo065\h5py_f1f64cd207fd4e3fbbc73e1ddc4b40d2\setup.py";f = getattr(tokenize, ‘"’“‘open’”’“‘, open)(file) if os.path.exists(file) else io.StringIO(’”‘“‘from setuptools import setup; setup()’”’“‘);code = f.read().replace(’”‘"’\r\n’“'”‘, ‘"’"’\n’“'”‘);f.close();exec(compile(code, file, ‘"’“‘exec’”’“‘))’ install --record ‘C:\Users\Arne’”'"‘s PC\AppData\Local\Temp\pip-record-prai98me\install-record.txt’ --single-version-externally-managed --compile --install-headers ‘d:\tensorflow object detection\tfodcourse\tfod\include\site\python3.9\h5py’ Check the logs for full command output. WARNING: Ignoring invalid distribution -ip (d:\tensorflow object detection\tfodcourse\tfod\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\tensorflow object detection\tfodcourse\tfod\lib\site-packages)

Would be awesome if someone can help me! 😃

I think it will be hard to expect a backport on 2.1.0.