tensorflow: Optimizer (Adam) does not propagate hyperparameter update on first update

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): no
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04 64b
• TensorFlow version (use command below): 2.0.0b1
• Python version: 3.6

Describe the current behavior Using tf.keras.optimizers.Adam and updating hyperparameters results in hyperparameters not updating on the first update.

Describe the expected behavior Hyperparameters DO update on the first run, ie they are initialized to tf.Variables at constructor of the optimizer, not on first call.

Code to reproduce the issue adam = tf.keras.optimizers.Adam() adam.learning_rate = 0.00001 # this does not update learning_rate, but changes python-typed hyperparameters in "_hyper" dictionary to tf.Variables adam.learning_rate = 0.00001 # this finally updates the tf.Variable

Calling adam._hyper before update yields:

{'learning_rate': 0.001, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999}

Calling adam._hyper after first update yields:

{'learning_rate': <tf.Variable 'learning_rate:0' shape=() dtype=float32, numpy=0.001>, 'decay': <tf.Variable 'decay:0' shape=() dtype=float32, numpy=0.0>, 'beta_1': <tf.Variable 'beta_1:0' shape=() dtype=float32, numpy=0.9>, 'beta_2': <tf.Variable 'beta_2:0' shape=() dtype=float32, numpy=0.999>}