tensorflow: Keras: inconsistent behavior of "embedded" models
Not sure if this a bug (I am inclined to believe so) or not, but it looks strange/inconsistent to me… Consider the following scenario (a simple “embedded” model case):
m1 = Sequential([
Input((100,)),
Dense(20),
Dense(10),
])
m2 = Sequential([
Input((100,)),
m1,
Dense(5),
])
Now, assume that I would like to extract features from an internal layer of the models. The following example works as expected:
test1 = Model(m1.input, m1.layers[1].output)
test2 = Model(m2.input, m2.layers[1].output)
However, this one fails, even though I would expect it to be equivalent to test1
above:
test3 = Model(m2.input, m2.layers[0].layers[1].output)
The error is:
ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_3:0", shape=(None, 100), dtype=float32) at layer "dense_3". The following previous layers were accessed without issue: []
It seems that the following “trick” does the job:
test4 = Model(m2.layers[0].input, m2.layers[0].layers[1].output)
Now, there is another peculiarity, which seems rather annoying to me:
m1.summary()
Model: "sequential_2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_3 (Dense) (None, 20) 2020
_________________________________________________________________
dense_4 (Dense) (None, 10) 210
=================================================================
Total params: 2,230
Trainable params: 2,230
Non-trainable params: 0
_________________________________________________________________
The model has two layers (no input layer), whereas this one has three layers…
test4.summary()
Model: "functional_19"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_3 (InputLayer) [(None, 100)] 0
_________________________________________________________________
dense_3 (Dense) (None, 20) 2020
_________________________________________________________________
dense_4 (Dense) (None, 10) 210
=================================================================
Total params: 2,230
Trainable params: 2,230
Non-trainable params: 0
_________________________________________________________________
What is interesting tf.keras.utils.plot_model
produces identical graphs (with input layers) for both m1
and test4
.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 31 (30 by maintainers)
The documentation source is https://www.tensorflow.org/api_docs/python/tf/keras/layers/InputLayer
If you want the historical motivation about Sequential summary see: https://github.com/tensorflow/tensorflow/blob/42c2ae60913b517a5c345bf35185afe412f587ee/tensorflow/python/keras/engine/sequential.py#L148-L157