keras: Can't load_model with error “Optimizer weight shape (256, 32) not compatible with provided weight shape (4,)”

I am using Keras 1.1.1 and am having the same problem. Deleting the optimizer weights as a workaround works for me.

Just in case someone needs to do the same, here’s the code:

import h5py
f = h5py.File('model_file.h5', 'r+')
del f['optimizer_weights']
f.close()

I had the same problem and what it seemed to work for me was to set the compile flag to False in the load_model function. Afterwards one can compile the model with the previously used optimizer.

model = load_model('my_model.hdf5', compile=False)
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

After updating from Keras 1.X to Keras 2.0.1, I have the same problem:

  File "./ensemble.py", line 56, in <module>
    models = [load_model(model_path) for model_path in model_names]
  File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 272, in load_model
    model.optimizer.set_weights(optimizer_weight_values)
  File "/usr/local/lib/python2.7/dist-packages/keras/optimizers.py", line 79, in set_weights
    'provided weight shape ' + str(w.shape))
ValueError: Optimizer weight shape (32,) not compatible with provided weight shape (3, 3, 3, 32)

Deleting the optimizer_weights group (dataset? object? directory? what is the right term?) from the HDF5 file with hdf5view fixed it as @dteoh suggested.

I updated all my models with

#!/usr/bin/env python

"""Make keras 1.x models usable in keras 2.x."""

import glob
import h5py

model_files = sorted(glob.glob('*.h5'))
for model_file in model_files:
    print("Update '{}'".format(model_file))
    with h5py.File(model_file, 'a') as f:
        if 'optimizer_weights' in f.keys():
            del f['optimizer_weights']

Now it is working again.

Also: Is it possible to save the model without optimizer parameters?

The following codes really worked! Thank you!

    with h5py.File(model_file, 'a') as f:
        if 'optimizer_weights' in f.keys():
            del f['optimizer_weights']

@xisnu Thanks for your advice! I trained many models and saved into many files, so it is laborious to delete the optimiser_weight manualy which advised by issue #3964. Finally, I solved it by a using load_weights, as follows:

# serialize model to JSON
model_json = model.to_json()
with open("model.json", "w") as json_file:
    json_file.write(model_json)
# serialize weights to HDF5
model.save_weights("model.h5")
print("Saved model into h5 file")

# later...

# load json and create model
json_file = open('model.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
loaded_model = model_from_json(loaded_model_json)
# load weights into new model
loaded_model.load_weights("model.h5")
print("Loaded model from disk")

# evaluate loaded model on test data
loaded_model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
score = loaded_model.evaluate(X, Y, verbose=0)

In my case, the problem was that on one machine everything was fine, on another I got “Optimizer weight shape … not compatible with provided weight shape …” exception.

The problem turned out to be in Keras installed using different methods on different machines.

Although it had the same version 1.1.0 on both the machine that built the model and the machine that loaded the model, the files in /usr/local/lib/python2.7/dist-packages/keras/ were different (in particular, /usr/local/lib/python2.7/dist-packages/keras/models.py).

Uninstalling and re-installing Keras on the problematic machine, solved the error, since now files in /usr/local/lib/python2.7/dist-packages/keras/ on both the machine that built the model and the machine that loaded the model were identical (to what I had on the machine that built the model).

I can confirm the same issue. Deleting the optimizer_weights as suggested by @dteoh works if your model is already trained.

However, I’m working on a project where I train for N number of epochs, stop training, adjust learning rate, then re-load and continue training the model.

In this case, deleting my optimizer_weights would lead to the error:

AttributeError: 'Model' object has no attribute 'optimizer'

This error makes sense since the optimizer attribute has been deleted. But as it currently stands, I can’t figure out how to:

1. Train a (non-Sequential) model.
2. Serialize it.
3. Load it from disk.
4. Adjust learning rate.
5. Continue training.

Without the error others in this thread have mentioned.

EDIT: Spun up a new virtual environment with Keras > 1.2 installed and was able to resume training.

@fsonntag I think the problem comes from https://github.com/fchollet/keras/commit/028aae19bf5ae6efe0b32d25d1c700224eebfcf9

If you do not use weights.sort(key=lambda x: x.name), weights order will be different from when it is saved. weights.sort(key=lambda x: x.auto_name) should not be used.

All the weights should have proper names and their order should be stable.

Confirming this issue with loading models stored after train. Issue is with weights shapes; it helps to remove optimizer from the model file. Then it loads successfully. I updated Keras to recent 1.1.0, and use Python 2.7.12 from Anaconda 4.0.0 but it does not help. I also found that if you load model in Spyder 2 times, then second load if fine. Seems to be really cool if the issue is fixed. Thanks