tensorflow: tf.keras.layers.Softmax does not support masking?

import tensorflow as tf
outputs = tf.keras.layers.Softmax().apply(
  tf.keras.layers.Masking().apply(
    tf.zeros([3,5,7])
  )
)

Since the default mask value of Masking is zero, Softmax should skip all values in the above case, and its behavior should be like sparse softmax. Therefore, I suppose the output should be all zeros, but that is not the case.

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 16 (3 by maintainers)

Most upvoted comments

I think my question is clear enough… Maybe you need to know what is masking first…

erikchwang on Mar 24, 2019

You made too many assumptions. I do not use LSTM, neither do I calculate loss, I just want to verify if Softmax support masking. Now it seems that the answer is NO.

erikchwang on Dec 9, 2019

@chwang85 masking replace the value with 0 actually for example. t = tf.fill([2,2],5.0) m = tf.keras.layers.Masking(5) print(m.apply(t)) “”" output tf.Tensor( [[0. 0.] [0. 0.]], shape=(2, 2), dtype=float32) “”"

t = tf.fill([2,2],5.0) m = tf.keras.layers.Masking() # default value 0.0 print(m.apply(t))

“”" output [[5. 5.] [5. 5.]], shape=(2, 2), dtype=float32) “”"

Gurpreetsingh9465 on Mar 25, 2019

@chwang85 can you elaborate a bit because the output is correct i guess, which is Tensor of shape [3,5,7] of value = 0.14285715 which is correct as shown here wiki

Gurpreetsingh9465 on Mar 24, 2019