tensorflow: Attempting to use uninitialized value lstm/LSTMCell/W_0

Environment info

Operating System: Rocks OS (Centos 6.5)

I installed from sources, and here is my version: https://github.com/shiyemin/tensorflow/ Nothing changed but to make it compile successfully on our server.

Steps to reproduce

add a LSTM layer between local3 and local4 in cifar10.py

# lstm
with tf.variable_scope('lstm') as scope:
  lstm_cell = rnn_cell.LSTMCell(512, input_size=384)
  feed_in = tf.split(0, 16, local3)
  outputs, states = rnn.rnn(lstm_cell, feed_in, scope=scope, dtype=tf.float32)
  lstm1 = tf.concat(0, outputs)

# local4
with tf.variable_scope('local4') as scope:
  weights = _variable_with_weight_decay('weights', shape=[512, 192],
                                        stddev=0.04, wd=0.004)
  biases = _variable_on_cpu('biases', [192], tf.constant_initializer(0.1))
  local4 = tf.nn.relu(tf.matmul(lstm1, weights) + biases, name=scope.name)
  _activation_summary(local4)

run cifar10_multi_gpu_train.py
Error: Attempting to use uninitialized value lstm/LSTMCell/W_0

What have you tried?

Run this network in Test mode, and everything is fine.
Run this network in Train mode, error “Attempting to use uninitialized value lstm/LSTMCell/W_0” occurs.

Logs or other output that would be helpful

I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:107] Allocating 10.60GiB bytes. I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:118] GPU 0 memory begins at 0x23ee00000 extends to 0x4e573199a W tensorflow/core/common_runtime/executor.cc:1221] 0xb26c6f0 Compute status: Failed precondition: Attempting to use uninitialized value lstm/LSTMCell/W_0 [[Node: lstm/LSTMCell/W_0/_9 = _SendT=DT_FLOAT, client_terminated=false, recv_device=“/job:localhost/replica:0/task:0/cpu:0”, send_device=“/job:localhost/replica:0/task:0/gpu:0”, send_device_incarnation=1, tensor_name=“edge_292_lstm/LSTMCell/W_0”, _device=“/job:localhost/replica:0/task:0/gpu:0”]] [[Node: Identity_7/_8 = _Recvclient_terminated=false, recv_device=“/job:localhost/replica:0/task:0/gpu:0”, send_device=“/job:localhost/replica:0/task:0/cpu:0”, send_device_incarnation=1, tensor_name=“edge_267_Identity_7”, tensor_type=DT_FLOAT, _device=“/job:localhost/replica:0/task:0/gpu:0”]] W tensorflow/core/common_runtime/executor.cc:1221] 0xb26c6f0 Compute status: Failed precondition: Attempting to use uninitialized value lstm/LSTMCell/W_0 [[Node: lstm/LSTMCell/W_0/_9 = _SendT=DT_FLOAT, client_terminated=false, recv_device=“/job:localhost/replica:0/task:0/cpu:0”, send_device=“/job:localhost/replica:0/task:0/gpu:0”, send_device_incarnation=1, tensor_name=“edge_292_lstm/LSTMCell/W_0”, _device=“/job:localhost/replica:0/task:0/gpu:0”]] [[Node: Identity_6/_16 = _Recvclient_terminated=false, recv_device=“/job:localhost/replica:0/task:0/gpu:0”, send_device=“/job:localhost/replica:0/task:0/cpu:0”, send_device_incarnation=1, tensor_name=“edge_295_Identity_6”, tensor_type=DT_FLOAT, _device=“/job:localhost/replica:0/task:0/gpu:0”]] W tensorflow/core/common_runtime/executor.cc:1221] 0xb255bf0 Compute status: Failed precondition: Attempting to use uninitialized value lstm/LSTMCell/W_0 [[Node: lstm/LSTMCell/W_0/_9 = _SendT=DT_FLOAT, client_terminated=false, recv_device=“/job:localhost/replica:0/task:0/cpu:0”, send_device=“/job:localhost/replica:0/task:0/gpu:0”, send_device_incarnation=1, tensor_name=“edge_292_lstm/LSTMCell/W_0”, _device=“/job:localhost/replica:0/task:0/gpu:0”]] [[Node: lstm/LSTMCell/B/Assign/_5 = _Recvclient_terminated=false, recv_device=“/job:localhost/replica:0/task:0/cpu:0”, send_device=“/job:localhost/replica:0/task:0/gpu:0”, send_device_incarnation=1, tensor_name=“edge_265_lstm/LSTMCell/B/Assign”, tensor_type=DT_FLOAT, _device=“/job:localhost/replica:0/task:0/cpu:0”]] W tensorflow/core/common_runtime/executor.cc:1221] 0xb255bf0 Compute status: Failed precondition: Attempting to use uninitialized value lstm/LSTMCell/W_0 [[Node: lstm/LSTMCell/W_0/_9 = _SendT=DT_FLOAT, client_terminated=false, recv_device=“/job:localhost/replica:0/task:0/cpu:0”, send_device=“/job:localhost/replica:0/task:0/gpu:0”, send_device_incarnation=1, tensor_name=“edge_292_lstm/LSTMCell/W_0”, _device=“/job:localhost/replica:0/task:0/gpu:0”]] [[Node: lstm/LSTMCell/W_0/Assign/_13 = _Recvclient_terminated=false, recv_device=“/job:localhost/replica:0/task:0/cpu:0”, send_device=“/job:localhost/replica:0/task:0/gpu:0”, send_device_incarnation=1, tensor_name=“edge_293_lstm/LSTMCell/W_0/Assign”, tensor_type=DT_FLOAT, _device=“/job:localhost/replica:0/task:0/cpu:0”]] W tensorflow/core/common_runtime/executor.cc:1221] 0xb255bf0 Compute status: Failed precondition: Attempting to use uninitialized value lstm/LSTMCell/W_0 [[Node: lstm/LSTMCell/W_0/_9 = _SendT=DT_FLOAT, client_terminated=false, recv_device=“/job:localhost/replica:0/task:0/cpu:0”, send_device=“/job:localhost/replica:0/task:0/gpu:0”, send_device_incarnation=1, tensor_name=“edge_292_lstm/LSTMCell/W_0”, _device=“/job:localhost/replica:0/task:0/gpu:0”]] [[Node: init/NoOp_1/_19 = _Recvclient_terminated=false, recv_device=“/job:localhost/replica:0/task:0/cpu:0”, send_device=“/job:localhost/replica:0/task:0/gpu:0”, send_device_incarnation=1, tensor_name=“edge_5848_init/NoOp_1”, tensor_type=DT_FLOAT, _device=“/job:localhost/replica:0/task:0/cpu:0”]] W tensorflow/core/common_runtime/executor.cc:1221] 0xb26c6f0 Compute status: Failed precondition: Attempting to use uninitialized value lstm/LSTMCell/W_0 [[Node: lstm/LSTMCell/W_0/_9 = _SendT=DT_FLOAT, client_terminated=false, recv_device=“/job:localhost/replica:0/task:0/cpu:0”, send_device=“/job:localhost/replica:0/task:0/gpu:0”, send_device_incarnation=1, tensor_name=“edge_292_lstm/LSTMCell/W_0”, _device=“/job:localhost/replica:0/task:0/gpu:0”]] [[Node: lstm/LSTMCell/B/Assign/_4 = _SendT=DT_FLOAT, client_terminated=false, recv_device=“/job:localhost/replica:0/task:0/cpu:0”, send_device=“/job:localhost/replica:0/task:0/gpu:0”, send_device_incarnation=1, tensor_name=“edge_265_lstm/LSTMCell/B/Assign”, _device=“/job:localhost/replica:0/task:0/gpu:0”]] W tensorflow/core/common_runtime/executor.cc:1221] 0xb26c6f0 Compute status: Failed precondition: Attempting to use uninitialized value lstm/LSTMCell/W_0 [[Node: lstm/LSTMCell/W_0/_9 = _SendT=DT_FLOAT, client_terminated=false, recv_device=“/job:localhost/replica:0/task:0/cpu:0”, send_device=“/job:localhost/replica:0/task:0/gpu:0”, send_device_incarnation=1, tensor_name=“edge_292_lstm/LSTMCell/W_0”, _device=“/job:localhost/replica:0/task:0/gpu:0”]] [[Node: lstm/LSTMCell/W_0/Assign/_12 = _SendT=DT_FLOAT, client_terminated=false, recv_device=“/job:localhost/replica:0/task:0/cpu:0”, send_device=“/job:localhost/replica:0/task:0/gpu:0”, send_device_incarnation=1, tensor_name=“edge_293_lstm/LSTMCell/W_0/Assign”, _device=“/job:localhost/replica:0/task:0/gpu:0”]] Traceback (most recent call last): File “cifar10_multi_gpu_train.py”, line 282, in <module> tf.app.run() File “/home/shiyemin/code/tensorflow/_python_build/tensorflow/python/platform/default/_app.py”, line 30, in run sys.exit(main(sys.argv)) File “cifar10_multi_gpu_train.py”, line 278, in main train() File “cifar10_multi_gpu_train.py”, line 237, in train sess.run(init) File “/home/shiyemin/code/tensorflow/_python_build/tensorflow/python/client/session.py”, line 315, in run return self._run(None, fetches, feed_dict) File “/home/shiyemin/code/tensorflow/_python_build/tensorflow/python/client/session.py”, line 511, in _run feed_dict_string) File “/home/shiyemin/code/tensorflow/_python_build/tensorflow/python/client/session.py”, line 564, in _do_run target_list) File “/home/shiyemin/code/tensorflow/_python_build/tensorflow/python/client/session.py”, line 586, in _do_call e.code) tensorflow.python.framework.errors.FailedPreconditionError: Attempting to use uninitialized value lstm/LSTMCell/W_0 [[Node: lstm/LSTMCell/W_0/_9 = _SendT=DT_FLOAT, client_terminated=false, recv_device=“/job:localhost/replica:0/task:0/cpu:0”, send_device=“/job:localhost/replica:0/task:0/gpu:0”, send_device_incarnation=1, tensor_name=“edge_292_lstm/LSTMCell/W_0”, _device=“/job:localhost/replica:0/task:0/gpu:0”]] [[Node: lstm/LSTMCell/B/Assign/_5 = _Recvclient_terminated=false, recv_device=“/job:localhost/replica:0/task:0/cpu:0”, send_device=“/job:localhost/replica:0/task:0/gpu:0”, send_device_incarnation=1, tensor_name=“edge_265_lstm/LSTMCell/B/Assign”, tensor_type=DT_FLOAT, _device=“/job:localhost/replica:0/task:0/cpu:0”]]

About this issue

Original URL
State: closed
Created 8 years ago
Comments: 20 (3 by maintainers)

Commits related to this issue

Merge pull request #1390 from ROCmSoftwarePlatform/develop-upstream-deven-misc-210623 Removing trivial differences with upstream repo — committed to fsx950223/tensorflow by deven-amd 3 years ago

Most upvoted comments

I also met a similar error, but it was cause by my own fault. for my case, I defined the LSTM within a function (within a class), say

def prediction(self): some codes… lstm_cell = tf.nn.rnn_cell.LSTMCell(self.size_inner_vector) lstm_cell = tf.nn.rnn_cell.DropoutWrapper(lstm_cell, output_keep_prob=self.dropout) lstm_stack = tf.nn.rnn_cell.MultiRNNCell([lstm_cell] * self.num_hiddden_layers) output, _ = tf.nn.dynamic_rnn(lstm_stack, batch_3D_input2lstm, dtype=tf.float32) some codes…

return result

For my case, I initialised the class without calling this function. When I fetch the value of a variable within this function, I got the above error.

Thus a possible reason is that: your expected graph has not been constructed, but you are fetching the value of a non-existing node. ‘this is my stupid case>_<’

By sharing it here, I hope it can help someone else.

laurel2935 on Dec 23, 2016

It sounds like the variables were not initialized, but that’s strange if tf.initialize_all_variables is being run. Can you check that you initialized the variables? Also - could you make a repo with just the problematic model and test it against the head TensorFlow?

lukaszkaiser on Mar 8, 2016