tensorflow: Keras: inconsistent behavior of "embedded" models

Not sure if this a bug (I am inclined to believe so) or not, but it looks strange/inconsistent to me… Consider the following scenario (a simple “embedded” model case):

m1 = Sequential([
    Input((100,)),
    Dense(20),
    Dense(10),
])

m2 = Sequential([
    Input((100,)),
    m1,
    Dense(5),
])

Now, assume that I would like to extract features from an internal layer of the models. The following example works as expected:

test1 = Model(m1.input, m1.layers[1].output)
test2 = Model(m2.input, m2.layers[1].output)

However, this one fails, even though I would expect it to be equivalent to test1 above:

test3 = Model(m2.input, m2.layers[0].layers[1].output)

The error is:

ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_3:0", shape=(None, 100), dtype=float32) at layer "dense_3". The following previous layers were accessed without issue: []

It seems that the following “trick” does the job:

test4 = Model(m2.layers[0].input, m2.layers[0].layers[1].output)

Now, there is another peculiarity, which seems rather annoying to me:

m1.summary()

Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_3 (Dense)              (None, 20)                2020      
_________________________________________________________________
dense_4 (Dense)              (None, 10)                210       
=================================================================
Total params: 2,230
Trainable params: 2,230
Non-trainable params: 0
_________________________________________________________________

The model has two layers (no input layer), whereas this one has three layers…

test4.summary()

Model: "functional_19"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_3 (InputLayer)         [(None, 100)]             0         
_________________________________________________________________
dense_3 (Dense)              (None, 20)                2020      
_________________________________________________________________
dense_4 (Dense)              (None, 10)                210       
=================================================================
Total params: 2,230
Trainable params: 2,230
Non-trainable params: 0
_________________________________________________________________

What is interesting tf.keras.utils.plot_model produces identical graphs (with input layers) for both m1 and test4.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 31 (30 by maintainers)

Commits related to this issue

Most upvoted comments