tensorflow: tf.layers uses wrong variable scope

TF 1.4.

import tensorflow as tf
def f(x):
    return tf.layers.conv2d(x, 30, 3)

x = tf.zeros([3, 20, 20, 1])

with tf.variable_scope('a'):
    print(f(x))
with tf.variable_scope('a', reuse=True):
    print(f(x)) # works

print(f(x))
with tf.variable_scope(tf.get_variable_scope(), reuse=True):
    print(f(x)) # failed with:
"""
ValueError: Variable conv2d_1/kernel does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?
"""

From what I can see, tf.layers is trying to create the variables under a wrong variable scope name, making variable sharing impossible if used under the root scope.

I found that this works:

with tf.variable_scope(tf.get_variable_scope()):
    print(f(x))
with tf.variable_scope(tf.get_variable_scope(), reuse=True):
    print(f(x)) # works

But the first line seems redundant and counter-intuitive.

//UPDATE: It also works if I don’t use tf.layers:

def f(x):
    W = tf.get_variable('w', shape=[1])
    return x + W

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 24 (20 by maintainers)

Commits related to this issue

Most upvoted comments

The definitive answer on this issue is:

  • Yes, this is an issue.
  • We cannot fix it without breaking backwards compatibility.
  • We will fix it soon in tf.keras. It will never be fixed in tf.layers (since it is necessary to maintain existing code working – which is not the case in Keras). tf.layers will eventually be removed (in TF 2.0).

Thank you for you response. For the tf.keras.layers, is there any way of sharing initialization parameters like the “arg_scope” in slim?

@ybsave If I’m not wrong, tf.layers will be replaced by tf.keras.layers in the future.

If tf.layers would be removed later, what should I use in the future? Is there any Tensorflow based middle level package like slim but supported officially by Google in later versions? I mean “pure tensorflow package”, but not Keras.