tensorflow: `tf.dynamic_stitch` gradient is incorrect

If possible, provide a minimal reproducible example (We usually don’t have time to read hundreds of lines of your code)

Original reproduction code (TensorFlow 1.0)

import tensorflow as tf

x = tf.zeros((1, 3))
y = tf.dynamic_stitch([[0], [0]], [x, tf.ones((1, 3))])

with tf.Session() as sess:
    print("y")
    print(sess.run(y))

    analytic, numeric = tf.test.compute_gradient(x, (1, 3), y, (1, 3))
    print("analytic")
    print(analytic)
    print("numeric")
    print(numeric)

Updated reproduction code (TensorFlow 2.16)

import tensorflow as tf

x = tf.zeros((1, 3))

analytic, numeric = tf.test.compute_gradient(
    lambda x: tf.dynamic_stitch([[0], [0]], [x, tf.ones((1, 3))]), [x]
)
print("analytic")
print(analytic)
print("numeric")
print(numeric)

gives output

y
[[ 1.  1.  1.]]
analytic
[[ 1.  0.  0.]
 [ 0.  1.  0.]
 [ 0.  0.  1.]]
numeric
[[ 0.  0.  0.]
 [ 0.  0.  0.]
 [ 0.  0.  0.]]

The numeric gradient correctly shows that x has no impact on y (since the value of x is completely overwritten by a constant in the dynamic_stitch). The analytic gradient is incorrect; it seems like the gradient calculation in dynamic_stitch does not handle the case where there are duplicate indices being merged.

About this issue

  • Original URL
  • State: open
  • Created 10 years ago
  • Comments: 15 (15 by maintainers)

Commits related to this issue

Most upvoted comments

This bug is still present in TensorFlow 2.16.1. The code from https://github.com/tensorflow/tensorflow/issues/7397#issuecomment-840738297 is still valid, and reproduces the bug. I have edited that into the original post for clarity.