keras: load_model - the two structures don't have the same nested structure
I trained a model and saved it using:
model.save("model_name")
I then call
model = tf.keras.models.load_model("model_name")
and get a “The two structures don’t have the same nested structure.” error. The structure that the model is expecting upon loading is not what I defined in the model.
My model:
roberta_layer = TFRobertaModel.from_pretrained('roberta-base')
ids = tf.keras.layers.Input((64,), dtype=tf.int32)
att = tf.keras.layers.Input((64,), dtype=tf.int32)
roberta_inputs = [ids, att]
sequence_output,pooled_output = roberta_layer(roberta_inputs)
# unigram
x1 = tf.keras.layers.Conv1D(32,1,activation='relu')(sequence_output)
x1 = tf.keras.layers.GlobalMaxPool1D()(x1)
# bigram
x2 = tf.keras.layers.Conv1D(32,2,activation='relu')(sequence_output)
x2 = tf.keras.layers.GlobalMaxPool1D()(x2)
# trigram
x3 = tf.keras.layers.Conv1D(32,3,activation='relu')(sequence_output)
x3 = tf.keras.layers.GlobalMaxPool1D()(x3)
concat = tf.keras.layers.Concatenate()([x1,x2,x3])
concat = tf.keras.layers.Dropout(0.5)(concat)
outputs = tf.keras.layers.Dense(11, activation='sigmoid')(concat)
model = tf.keras.Model(inputs=roberta_inputs, outputs=outputs)
I’m using tf 2.4.0 and transformers 3.5.1
The full error is: The two structures don’t have the same nested structure.
First structure: type=dict str={‘input_ids’: TensorSpec(shape=(None, 5), dtype=tf.int32, name=‘inputs/input_ids’)}
Second structure: type=list str=[TensorSpec(shape=(None, 64), dtype=tf.int32, name=‘inputs/0’), TensorSpec(shape=(None, 64), dtype=tf.int32, name=‘inputs/1’)]
More specifically: The two namedtuples don’t have the same sequence type. First structure type=dict str={‘input_ids’: TensorSpec(shape=(None, 5), dtype=tf.int32, name=‘inputs/input_ids’)} has type dict, while second structure type=list str=[TensorSpec(shape=(None, 64), dtype=tf.int32, name=‘inputs/0’), TensorSpec(shape=(None, 64), dtype=tf.int32, name=‘inputs/1’)] has type list
During handling of the above exception, another exception occurred:
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 7
- Comments: 33 (4 by maintainers)
I finally solved this issue by using
tf.saved_model.save(model, path_to_dir)instead ofmodel.save(path_to_dir)ortf.keras.models.save_model(model, path_to_dir).followed by this guide: https://www.tensorflow.org/guide/saved_model
A solution is to call the roberta model slightly different:
sequence_output,pooled_output = roberta_layer.roberta(roberta_inputs)This fixes the problem
tf.saved_model.load() instead of tf.keras.models.load_model() on Kaggle TPUs did the job for me.
I have figured out a way to overcome this problem. Instead of saving the model using
model.saveortf.keras.models.save_model(), try saving the model weights usingmodel.save_weights('model_weight.h5')during the training process.At the prediction or testing phase, you need to create the model (architecture) manually, the same way you did in the training phase, and then load the weights to it.
Just use the post’s codes and I can reproduce it (
tf.__version__:2.2.0):Then save the model
Then the following load will produce the error
But if I load by
tf.saved_model.loadinstead oftf.keras.models.load_model, it will succeedAlso tried saving with
tf.keras.models.save_modelinstead ofmodel.save, the same error still existsI also tried tf 2.6.0, still the same. Here’s the error it will produce:
Update: if I change
to
as @eyalshafran suggested, the problem is magically solved. I wonder what’s the cause
In case this is still relevant for people, here is what I think is going on:
Keras saves the input specs on the first call of the model here. When loading a pretrained model with transformers using the
from_pretrainedclass classmethod ofTFPretrainedModel, the networks is first fed dummy inputs here. So the saved models expect their input tensors to be of sequence length 5, because that is the length of the dummy inputs.In order to change that behaviour, you can reset the input specs before saving to a saved model like this:
tensorflow 2.7.0 keras 2.7.0 transformers 4.15.0 pretrained model: “bert-base-multilingual-cased”
I also had the same issue when using model.save() & keras.model.load_model(). but, I change my python code to model.save_weights() & model.load_weights(), this issue resolved. thanks to @Allen-Qiu & @shreyas-jk
Yes, as introduced by shreyas-jk, model.save_weights(‘model_weight.h5’) can deal with this problem. My model contains a huggingface bert. It cannot be saved. However, I can save the weights. Subsequently, I build the model again in other files and load the weights.