tensorflow: `tf.dynamic_stitch` gradient is incorrect
If possible, provide a minimal reproducible example (We usually don’t have time to read hundreds of lines of your code)
Original reproduction code (TensorFlow 1.0)
import tensorflow as tf
x = tf.zeros((1, 3))
y = tf.dynamic_stitch([[0], [0]], [x, tf.ones((1, 3))])
with tf.Session() as sess:
print("y")
print(sess.run(y))
analytic, numeric = tf.test.compute_gradient(x, (1, 3), y, (1, 3))
print("analytic")
print(analytic)
print("numeric")
print(numeric)
Updated reproduction code (TensorFlow 2.16)
import tensorflow as tf
x = tf.zeros((1, 3))
analytic, numeric = tf.test.compute_gradient(
lambda x: tf.dynamic_stitch([[0], [0]], [x, tf.ones((1, 3))]), [x]
)
print("analytic")
print(analytic)
print("numeric")
print(numeric)
gives output
y
[[ 1. 1. 1.]]
analytic
[[ 1. 0. 0.]
[ 0. 1. 0.]
[ 0. 0. 1.]]
numeric
[[ 0. 0. 0.]
[ 0. 0. 0.]
[ 0. 0. 0.]]
The numeric gradient correctly shows that x has no impact on y (since the value of x is completely overwritten by a constant in the dynamic_stitch). The analytic gradient is incorrect; it seems like the gradient calculation in dynamic_stitch does not handle the case where there are duplicate indices being merged.
About this issue
- Original URL
- State: open
- Created 10 years ago
- Comments: 15 (15 by maintainers)
Commits related to this issue
- Fix dynamic_stitch gradient implementation Addresses #7397 Also expanded unit tests to cover these cases. — committed to drasmuss/tensorflow by drasmuss 10 years ago
- Fix dynamic_stitch gradient implementation Addresses #7397 Also expanded unit tests to cover these cases. — committed to drasmuss/tensorflow by drasmuss 10 years ago
- Fix dynamic_stitch gradient implementation Addresses #7397 Expanded unit tests to cover these cases. — committed to drasmuss/tensorflow by drasmuss 10 years ago
- Fix dynamic_stitch gradient implementation Addresses #7397 Expanded unit tests to cover these cases. — committed to drasmuss/tensorflow by drasmuss 10 years ago
- Fix dynamic_stitch gradient implementation Addresses #7397 Expanded unit tests to cover these cases. — committed to drasmuss/tensorflow by drasmuss 10 years ago
- Fix dynamic_stitch gradient implementation Addresses #7397 Expanded unit tests to cover these cases. — committed to drasmuss/tensorflow by drasmuss 10 years ago
- Fix dynamic_stitch gradient implementation Addresses #7397 Expanded unit tests to cover these cases. — committed to drasmuss/tensorflow by drasmuss 10 years ago
This bug is still present in TensorFlow 2.16.1. The code from https://github.com/tensorflow/tensorflow/issues/7397#issuecomment-840738297 is still valid, and reproduces the bug. I have edited that into the original post for clarity.