tensorflow: Inconsistent behavior for tf.variable_scope

Problem:

Tensorflow doesn’t place ops (e.g. mul) in pre-existing variable scopes (and automatically creates a new scope instead).

Minimal Reproducible Example

with tf.variable_scope('layer123'):
    v = tf.get_variable('v', [], initializer=tf.constant_initializer(42., tf.float32))
    w = v * 2
print(w.name)    # Prints 'layer123/mul:0'

However,

with tf.variable_scope('layer123'):
    v = tf.get_variable('v', [], initializer=tf.constant_initializer(42., tf.float32))

with tf.variable_scope('layer123'):
    w = v * 2

print(w.name)    # Prints 'layer123_1/mul:0'

Observe that for the latter, the op w is placed in a different variable scope, auto-named layer123_1.

I’ve tried the following, to the same effect:

with tf.variable_scope('layer123') as scope:
    v = tf.get_variable('v', [], initializer=tf.constant_initializer(42., tf.float32))

with tf.variable_scope(scope):
    w = v * 2

print(w.name)    # Prints 'layer123_1/mul:0'

with tf.variable_scope('layer123'):
    v = tf.get_variable('v', [], initializer=tf.constant_initializer(42., tf.float32))

with tf.variable_scope('layer123', reuse=True):
    w = v * 2

print(w.name)    # Prints 'layer123_1/mul:0'

VersionSpec

Tensorflow version: 0.11.0 (GPU) OS: Ubuntu 14.04 (w/ CUDA 8)

About this issue

Original URL
State: closed
Created 8 years ago
Reactions: 10
Comments: 17 (7 by maintainers)

Most upvoted comments

@eamartin My bad earlier. I just found you can do something like this: (from #6007)

class Foo(object):
    def __init__(self):
        with tf.variable_scope(None, 'Foo'):
            self.sc = tf.get_variable_scope()
            self.x = tf.get_variable('x', initializer=0.0)
            print(self.x)

    def meth(self, foo):
        with tf.name_scope(self.sc.original_name_scope):
            z = tf.multiply(foo, 3.14, name='z')
            print(z)

f0 = Foo()      # Foo/x
f1 = Foo()      # Foo_1/x
f0.meth(1.0)    # Foo/z
f0.meth(2.0)    # Foo/z_1
f1.meth(3.0)    # Foo_1/z

Maybe this will help?

ppwwyyxx on Feb 20, 2017

@eamartin Yes that’s a bit tricky. I hope tensorflow can change the default behavior, i.e. when you reenter a variable scope, you should expect to automatically reenter its name scope as well instead of creating a new one.

ppwwyyxx on Feb 20, 2017