tensorflow: tf.layers uses wrong variable scope
TF 1.4.
import tensorflow as tf
def f(x):
return tf.layers.conv2d(x, 30, 3)
x = tf.zeros([3, 20, 20, 1])
with tf.variable_scope('a'):
print(f(x))
with tf.variable_scope('a', reuse=True):
print(f(x)) # works
print(f(x))
with tf.variable_scope(tf.get_variable_scope(), reuse=True):
print(f(x)) # failed with:
"""
ValueError: Variable conv2d_1/kernel does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?
"""
From what I can see, tf.layers is trying to create the variables under a wrong variable scope name, making variable sharing impossible if used under the root scope.
I found that this works:
with tf.variable_scope(tf.get_variable_scope()):
print(f(x))
with tf.variable_scope(tf.get_variable_scope(), reuse=True):
print(f(x)) # works
But the first line seems redundant and counter-intuitive.
//UPDATE: It also works if I don’t use tf.layers:
def f(x):
W = tf.get_variable('w', shape=[1])
return x + W
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 24 (20 by maintainers)
Commits related to this issue
- Fix the tf.layers name bug See https://github.com/tensorflow/tensorflow/issues/14703 — committed to Oghma/sns-lstm by Oghma 6 years ago
- Fix the tf.layers name bug See https://github.com/tensorflow/tensorflow/issues/14703 — committed to Oghma/sns-lstm by Oghma 6 years ago
- Fix the tf.layers name bug See https://github.com/tensorflow/tensorflow/issues/14703 — committed to Oghma/sns-lstm by Oghma 6 years ago
- Fix the tf.layers name bug See https://github.com/tensorflow/tensorflow/issues/14703 — committed to Oghma/sns-lstm by Oghma 6 years ago
The definitive answer on this issue is:
tf.keras
. It will never be fixed intf.layers
(since it is necessary to maintain existing code working – which is not the case in Keras).tf.layers
will eventually be removed (in TF 2.0).Thank you for you response. For the tf.keras.layers, is there any way of sharing initialization parameters like the “arg_scope” in slim?
@ybsave If I’m not wrong,
tf.layers
will be replaced bytf.keras.layers
in the future.If tf.layers would be removed later, what should I use in the future? Is there any Tensorflow based middle level package like slim but supported officially by Google in later versions? I mean “pure tensorflow package”, but not Keras.