tensorflow: tensorflow/core/grappler/optimizers/meta_optimizer.cc:561] layout failed: Invalid argument: MutableGraphView::SortTopologically error:

Hi! I have the following message whenever I train a ConvLSTM with attention, after the samples are loaded in the buffer. The training goes on, but I haven’t found anything on the internet related to this. I am not sure if its a bug, or I’ve made a mistake.

The complete code I am using is from here. I suspect one of the last 3 lines of the function leads to this error. I tried debugging, but it didn’t helped much.

System information OS: Ubuntu 16.04 Tensorflow 2.1 installed from pip Python 3.6 CUDA 10.2 GPU Tesla K80, 12GB memory

`@tf.function

def train_step(self, images, labels):

    loss = 0

    with tf.GradientTape() as tape:

        if self.network_name == 'ALSTM':
            batch_feature_cube_sequence = self.network.get_batch_feature_cube_sequence(images)
            hidden_state, cell_state = self.network.reset_hidden_and_cell_state(batch_feature_cube_sequence)
            self.network.lstm1.initial_state = cell_state
            all_attention_weights = tf.zeros((0,
                                              self.batch_size,
                                              batch_feature_cube_sequence.shape[2],
                                              batch_feature_cube_sequence.shape[2]))

            for i in range(0, images.shape[1]):
                input_image = images[:, i, :, :, :]
                predictions, hidden_state, attention_weights = self.network(input_image, hidden_state)

                self.train_accuracy.update_state(labels, predictions)
                self.train_precision.update_state(labels, predictions)
                self.train_recall.update_state(labels, predictions)

                loss = loss + self.loss_object(labels, predictions)
                attention_weights = tf.expand_dims(attention_weights, 0)
                all_attention_weights = tf.concat([all_attention_weights, attention_weights], 0)

            classification_loss = loss / int(images.shape[1])
            attention_loss = self.attention_penalty(all_attention_weights)/ int(images.shape[1])*self.batch_size
            regularization_term = tf.add_n([tf.nn.l2_loss(v) for v in self.network.trainable_variables if 'bias' not in v.name])

            total_loss = classification_loss + self.penalty_coefficient*attention_loss + self.weight_decay*regularization_term

        elif self.network_name == 'ConvALSTM':

            hidden_state = self.network.get_initial_hidden_state(self.batch_size)
            for i in range(0, images.shape[1]):
                input_image = images[:, i, :, :, :]
                predictions, hidden_state, attention_weights = self.network(input_image, hidden_state)

                self.train_accuracy.update_state(labels, predictions)
                self.train_precision.update_state(labels, predictions)
                self.train_recall.update_state(labels, predictions)

                loss = loss + self.loss_object(labels, predictions)

            classification_loss = loss / int(images.shape[1])
            total_loss = classification_loss

    gradients = tape.gradient(total_loss, self.network.trainable_variables)
    self.optimizer.apply_gradients(zip(gradients, self.network.trainable_variables))
    self.train_loss.update_state(total_loss)
    return total_loss`

2020-02-19 14:50:31.311596: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:150] Filling up shuffle buffer (this may take a while): 464 of 1000 2020-02-19 14:50:41.318354: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:150] Filling up shuffle buffer (this may take a while): 926 of 1000 2020-02-19 14:50:42.932254: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:199] Shuffle buffer filled. 2020-02-19 14:52:10.425158: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:561] layout failed: Invalid argument: MutableGraphView::SortTopologically error: detected edge(s) creating cycle(s) {'Func/conv_alstm_7/conv_lst_m2d/while_grad/body/_15513/input/_27997' -> 'conv_alstm_7/conv_lst_m2d/while_grad/body/_15513/gradients/AddN', 'Func/conv_alstm_6/conv_lst_m2d/while_grad/body/_15704/input/_28113' -> 'conv_alstm_6/conv_lst_m2d/while_grad/body/_15704/gradients/AddN', 'Func/conv_alstm_5/conv_lst_m2d/while_grad/body/_15895/input/_28229' -> 'conv_alstm_5/conv_lst_m2d/while_grad/body/_15895/gradients/AddN', 'Func/conv_alstm_4/conv_lst_m2d/while_grad/body/_16086/input/_28345' -> 'conv_alstm_4/conv_lst_m2d/while_grad/body/_16086/gradients/AddN', 'Func/conv_alstm_3/conv_lst_m2d/while_grad/body/_16277/input/_28461' -> 'conv_alstm_3/conv_lst_m2d/while_grad/body/_16277/gradients/AddN', 'Func/conv_alstm_2/conv_lst_m2d/while_grad/body/_16468/input/_28577' -> 'conv_alstm_2/conv_lst_m2d/while_grad/body/_16468/gradients/AddN', 'Func/conv_alstm_1/conv_lst_m2d/while_grad/body/_16659/input/_28693' -> 'conv_alstm_1/conv_lst_m2d/while_grad/body/_16659/gradients/AddN', 'conv_alstm/conv_lst_m2d/while_grad/body/_16850/gradients/AddN_2' -> 'conv_alstm/conv_lst_m2d/while_grad/next_iteration/_17007', 'Func/conv_alstm_33/conv_lst_m2d/while_grad/body/_10547/input/_24981' -> 'conv_alstm_33/conv_lst_m2d/while_grad/body/_10547/gradients/AddN', 'Func/conv_alstm_32/conv_lst_m2d/while_grad/body/_10738/input/_25097' -> 'conv_alstm_32/conv_lst_m2d/while_grad/body/_10738/gradients/AddN', 'Func/conv_alstm_31/conv_lst_m2d/while_grad/body/_10929/input/_25213' -> 'conv_alstm_31/conv_lst_m2d/while_grad/body/_10929/gradients/AddN', 'Func/conv_alstm_30/conv_lst_m2d/while_grad/body/_11120/input/_25329' -> 'conv_alstm_30/conv_lst_m2d/while_grad/body/_11120/gradients/AddN', 'Func/conv_alstm_29/conv_lst_m2d/while_grad/body/_11311/input/_25445' -> 'conv_alstm_29/conv_lst_m2d/while_grad/body/_11311/gradients/AddN', 'Func/conv_alstm_28/conv_lst_m2d/while_grad/body/_11502/input/_25561' -> 'conv_alstm_28/conv_lst_m2d/while_grad/body/_11502/gradients/AddN', 'Func/conv_alstm_27/conv_lst_m2d/while_grad/body/_11693/input/_25677' -> 'conv_alstm_27/conv_lst_m2d/while_grad/body/_11693/gradients/AddN', 'Func/conv_alstm_26/conv_lst_m2d/while_grad/body/_11884/input/_25793' -> 'conv_alstm_26/conv_lst_m2d/while_grad/body/_11884/gradients/AddN', 'Func/conv_alstm_25/conv_lst_m2d/while_grad/body/_12075/input/_25909' -> 'conv_alstm_25/conv_lst_m2d/while_grad/body/_12075/gradients/AddN', 'Func/conv_alstm_24/conv_lst_m2d/while_grad/body/_12266/input/_26025' -> 'conv_alstm_24/conv_lst_m2d/while_grad/body/_12266/gradients/AddN', 'Func/conv_alstm_23/conv_lst_m2d/while_grad/body/_12457/input/_26141' -> 'conv_alstm_23/conv_lst_m2d/while_grad/body/_12457/gradients/AddN', 'Func/conv_alstm_21/conv_lst_m2d/while_grad/body/_12839/input/_26373' -> 'conv_alstm_21/conv_lst_m2d/while_grad/body/_12839/gradients/AddN', 'Func/conv_alstm_20/conv_lst_m2d/while_grad/body/_13030/input/_26489' -> 'conv_alstm_20/conv_lst_m2d/while_grad/body/_13030/gradients/AddN', 'Func/conv_alstm_19/conv_lst_m2d/while_grad/body/_13221/input/_26605' -> 'conv_alstm_19/conv_lst_m2d/while_grad/body/_13221/gradients/AddN', 'Func/conv_alstm_18/conv_lst_m2d/while_grad/body/_13412/input/_26721' -> 'conv_alstm_18/conv_lst_m2d/while_grad/body/_13412/gradients/AddN', 'Func/conv_alstm_17/conv_lst_m2d/while_grad/body/_13603/input/_26837' -> 'conv_alstm_17/conv_lst_m2d/while_grad/body/_13603/gradients/AddN', 'Func/conv_alstm_16/conv_lst_m2d/while_grad/body/_13794/input/_26953' -> 'conv_alstm_16/conv_lst_m2d/while_grad/body/_13794/gradients/AddN', 'Func/conv_alstm_15/conv_lst_m2d/while_grad/body/_13985/input/_27069' -> 'conv_alstm_15/conv_lst_m2d/while_grad/body/_13985/gradients/AddN', 'Func/conv_alstm_14/conv_lst_m2d/while_grad/body/_14176/input/_27185' -> 'conv_alstm_14/conv_lst_m2d/while_grad/body/_14176/gradients/AddN', 'Func/conv_alstm_13/conv_lst_m2d/while_grad/body/_14367/input/_27301' -> 'conv_alstm_13/conv_lst_m2d/while_grad/body/_14367/gradients/AddN', 'Func/conv_alstm_12/conv_lst_m2d/while_grad/body/_14558/input/_27417' -> 'conv_alstm_12/conv_lst_m2d/while_grad/body/_14558/gradients/AddN', 'Func/conv_alstm_11/conv_lst_m2d/while_grad/body/_14749/input/_27533' -> 'conv_alstm_11/conv_lst_m2d/while_grad/body/_14749/gradients/AddN', 'Func/conv_alstm_10/conv_lst_m2d/while_grad/body/_14940/input/_27649' -> 'conv_alstm_10/conv_lst_m2d/while_grad/body/_14940/gradients/AddN', 'Func/conv_alstm_9/conv_lst_m2d/while_grad/body/_15131/input/_27765' -> 'conv_alstm_9/conv_lst_m2d/while_grad/body/_15131/gradients/AddN', 'Func/conv_alstm_8/conv_lst_m2d/while_grad/body/_15322/input/_27881' -> 'conv_alstm_8/conv_lst_m2d/while_grad/body/_15322/gradients/AddN', 'conv_alstm_22/conv_lst_m2d/while_grad/body/_12648/gradients/convolution_4_grad/Conv2DBackpropInput' -> 'conv_alstm_22/conv_lst_m2d/while_grad/body/_12648/gradients/AddN_2', 'conv_alstm_22/conv_lst_m2d/while_grad/body/_12648/gradients/convolution_5_grad/Conv2DBackpropInput' -> 'conv_alstm_22/conv_lst_m2d/while_grad/body/_12648/gradients/AddN_2', 'conv_alstm_22/conv_lst_m2d/while_grad/body/_12648/gradients/convolution_6_grad/Conv2DBackpropInput' -> 'conv_alstm_22/conv_lst_m2d/while_grad/body/_12648/gradients/AddN_2', 'conv_alstm_22/conv_lst_m2d/while_grad/body/_12648/gradients/convolution_7_grad/Conv2DBackpropInput-2-TransposeNHWCToNCHW-LayoutOptimizer' -> 'conv_alstm_22/conv_lst_m2d/while_grad/body/_12648/gradients/convolution_7_grad/Conv2DBackpropInput', 'Func/conv_alstm_34/conv_lst_m2d/while_grad/body/_10356/input/_24865' -> 'conv_alstm_34/conv_lst_m2d/while_grad/body/_10356/gradients/AddN', 'Func/conv_alstm_35/conv_lst_m2d/while_grad/body/_10165/input/_24749' -> 'conv_alstm_35/conv_lst_m2d/while_grad/body/_10165/gradients/AddN', 'Func/conv_alstm_36/conv_lst_m2d/while_grad/body/_9974/input/_24633' -> 'conv_alstm_36/conv_lst_m2d/while_grad/body/_9974/gradients/AddN', 'conv_alstm_21/conv_lst_m2d/while/body/_5240/clip_by_value' -> 'conv_alstm_21/conv_lst_m2d/while/body/_5240/mul_3', 'conv_alstm_21/conv_lst_m2d/while/body/_5240/mul_2' -> 'conv_alstm_21/conv_lst_m2d/while/body/_5240/add_5', 'conv_alstm_21/conv_lst_m2d/while/body/_5240/clip_by_value_2' -> 'conv_alstm_21/conv_lst_m2d/while/body/_5240/mul_5', 'conv_alstm_21/conv_lst_m2d/while/body/_5240/convolution_6' -> 'conv_alstm_21/conv_lst_m2d/while/body/_5240/add_4', 'conv_alstm_22/conv_lst_m2d/while/body/_5459/clip_by_value_2' -> 'conv_alstm_22/conv_lst_m2d/while/body/_5459/mul_5', 'conv_alstm_22/conv_lst_m2d/while/body/_5459/convolution_6' -> 'conv_alstm_22/conv_lst_m2d/while/body/_5459/add_4', 'conv_alstm_22/conv_lst_m2d/while/body/_5459/clip_by_value' -> 'conv_alstm_22/conv_lst_m2d/while/body/_5459/mul_3', 'conv_alstm_22/conv_lst_m2d/while/body/_5459/mul_2' -> 'conv_alstm_22/conv_lst_m2d/while/body/_5459/add_5', 'conv_alstm_23/conv_lst_m2d/while/body/_5678/clip_by_value_2' -> 'conv_alstm_23/conv_lst_m2d/while/body/_5678/mul_5', 'conv_alstm_23/conv_lst_m2d/while/body/_5678/convolution_6' -> 'conv_alstm_23/conv_lst_m2d/while/body/_5678/add_4', 'conv_alstm_23/conv_lst_m2d/while/body/_5678/clip_by_value' -> 'conv_alstm_23/conv_lst_m2d/while/body/_5678/mul_3', 'conv_alstm_23/conv_lst_m2d/while/body/_5678/mul_2' -> 'conv_alstm_23/conv_lst_m2d/while/body/_5678/add_5', 'conv_alstm_24/conv_lst_m2d/while/body/_5897/clip_by_value_2' -> 'conv_alstm_24/conv_lst_m2d/while/body/_5897/mul_5', 'conv_alstm_24/conv_lst_m2d/while/body/_5897/convolution_6' -> 'conv_alstm_24/conv_lst_m2d/while/body/_5897/add_4', 'conv_alstm_24/conv_lst_m2d/while/body/_5897/clip_by_value' -> 'conv_alstm_24/conv_lst_m2d/while/body/_5897/mul_3', 'conv_alstm_24/conv_lst_m2d/while/body/_5897/mul_2' -> 'conv_alstm_24/conv_lst_m2d/while/body/_5897/add_5', 'conv_alstm_25/conv_lst_m2d/while/body/_6116/clip_by_value_2' -> 'conv_alstm_25/conv_lst_m2d/while/body/_6116/mul_5', 'conv_alstm_25/conv_lst_m2d/while/body/_6116/convolution_6' -> 'conv_alstm_25/conv_lst_m2d/while/body/_6116/add_4', 'conv_alstm_25/conv_lst_m2d/while/body/_6116/clip_by_value' -> 'conv_alstm_25/conv_lst_m2d/while/body/_6116/mul_3', 'conv_alstm_25/conv_lst_m2d/while/body/_6116/mul_2' -> 'conv_alstm_25/conv_lst_m2d/while/body/_6116/add_5', 'conv_alstm_26/conv_lst_m2d/while/body/_6335/clip_by_value' -> 'conv_alstm_26/conv_lst_m2d/while/body/_6335/mul_3', 'conv_alstm_26/conv_lst_m2d/while/body/_6335/mul_2' -> 'conv_alstm_26/conv_lst_m2d/while/body/_6335/add_5', 'conv_alstm_26/conv_lst_m2d/while/body/_6335/clip_by_value_2' -> 'conv_alstm_26/conv_lst_m2d/while/body/_6335/mul_5', 'conv_alstm_26/conv_lst_m2d/while/body/_6335/convolution_6' -> 'conv_alstm_26/conv_lst_m2d/while/body/_6335/add_4', 'conv_alstm_27/conv_lst_m2d/while/body/_6554/convolution_6' -> 'conv_alstm_27/conv_lst_m2d/while/body/_6554/add_4', 'conv_alstm_27/conv_lst_m2d/while/body/_6554/clip_by_value' -> 'conv_alstm_27/conv_lst_m2d/while/body/_6554/mul_3', 'conv_alstm_27/conv_lst_m2d/while/body/_6554/mul_2' -> 'conv_alstm_27/conv_lst_m2d/while/body/_6554/add_5', 'conv_alstm_27/conv_lst_m2d/while/body/_6554/clip_by_value_2' -> 'conv_alstm_27/conv_lst_m2d/while/body/_6554/mul_5', 'conv_alstm_28/conv_lst_m2d/while/body/_6773/mul_2' -> 'conv_alstm_28/conv_lst_m2d/while/body/_6773/add_5', 'conv_alstm_29/conv_lst_m2d/while/body/_6992/clip_by_value_2' -> 'conv_alstm_29/conv_lst_m2d/while/body/_6992/mul_5', 'conv_alstm_29/conv_lst_m2d/while/body/_6992/convolution_6' -> 'conv_alstm_29/conv_lst_m2d/while/body/_6992/add_4', 'conv_alstm_29/conv_lst_m2d/while/body/_6992/clip_by_value' -> 'conv_alstm_29/conv_lst_m2d/while/body/_6992/mul_3', 'conv_alstm_29/conv_lst_m2d/while/body/_6992/mul_2' -> 'conv_alstm_29/conv_lst_m2d/while/body/_6992/add_5', 'conv_alstm_30/conv_lst_m2d/while/body/_7211/clip_by_value_2' -> 'conv_alstm_30/conv_lst_m2d/while/body/_7211/mul_5', 'conv_alstm_30/conv_lst_m2d/while/body/_7211/convolution_6' -> 'conv_alstm_30/conv_lst_m2d/while/body/_7211/add_4', 'conv_alstm_30/conv_lst_m2d/while/body/_7211/clip_by_value' -> 'conv_alstm_30/conv_lst_m2d/while/body/_7211/mul_3', 'conv_alstm_30/conv_lst_m2d/while/body/_7211/mul_2' -> 'conv_alstm_30/conv_lst_m2d/while/body/_7211/add_5', 'conv_alstm_31/conv_lst_m2d/while/body/_7430/mul_2' -> 'conv_alstm_31/conv_lst_m2d/while/body/_7430/add_5', 'conv_alstm_32/conv_lst_m2d/while/body/_7649/mul_2' -> 'conv_alstm_32/conv_lst_m2d/while/body/_7649/add_5', 'conv_alstm_33/conv_lst_m2d/while/body/_7868/clip_by_value' -> 'conv_alstm_33/conv_lst_m2d/while/body/_7868/mul_3', 'conv_alstm_33/conv_lst_m2d/while/body/_7868/mul_2' -> 'conv_alstm_33/conv_lst_m2d/while/body/_7868/add_5', 'conv_alstm_33/conv_lst_m2d/while/body/_7868/clip_by_value_2' -> 'conv_alstm_33/conv_lst_m2d/while/body/_7868/mul_5', 'conv_alstm_33/conv_lst_m2d/while/body/_7868/convolution_6' -> 'conv_alstm_33/conv_lst_m2d/while/body/_7868/add_4', 'conv_alstm_34/conv_lst_m2d/while/body/_8087/mul_2' -> 'conv_alstm_34/conv_lst_m2d/while/body/_8087/add_5', 'conv_alstm_35/conv_lst_m2d/while/body/_8306/clip_by_value' -> 'conv_alstm_35/conv_lst_m2d/while/body/_8306/mul_3', 'conv_alstm_35/conv_lst_m2d/while/body/_8306/mul_2' -> 'conv_alstm_35/conv_lst_m2d/while/body/_8306/add_5', 'conv_alstm_35/conv_lst_m2d/while/body/_8306/clip_by_value_2' -> 'conv_alstm_35/conv_lst_m2d/while/body/_8306/mul_5', 'conv_alstm_35/conv_lst_m2d/while/body/_8306/convolution_6' -> 'conv_alstm_35/conv_lst_m2d/while/body/_8306/add_4', 'conv_alstm_36/conv_lst_m2d/while/body/_8525/convolution_6' -> 'conv_alstm_36/conv_lst_m2d/while/body/_8525/add_4', 'conv_alstm_36/conv_lst_m2d/while/body/_8525/clip_by_value' -> 'conv_alstm_36/conv_lst_m2d/while/body/_8525/mul_3', 'conv_alstm_36/conv_lst_m2d/while/body/_8525/mul_2' -> 'conv_alstm_36/conv_lst_m2d/while/body/_8525/add_5', 'conv_alstm_36/conv_lst_m2d/while/body/_8525/clip_by_value_2' -> 'conv_alstm_36/conv_lst_m2d/while/body/_8525/mul_5', 'conv_alstm_37/conv_lst_m2d/while/body/_8744/convolution_6' -> 'conv_alstm_37/conv_lst_m2d/while/body/_8744/add_4', 'conv_alstm_37/conv_lst_m2d/while/body/_8744/clip_by_value' -> 'conv_alstm_37/conv_lst_m2d/while/body/_8744/mul_3', 'conv_alstm_37/conv_lst_m2d/while/body/_8744/mul_2' -> 'conv_alstm_37/conv_lst_m2d/while/body/_8744/add_5', 'conv_alstm_37/conv_lst_m2d/while/body/_8744/clip_by_value_2' -> 'conv_alstm_37/conv_lst_m2d/while/body/_8744/mul_5', 'conv_alstm_38/conv_lst_m2d/while/body/_8963/clip_by_value_2' -> 'conv_alstm_38/conv_lst_m2d/while/body/_8963/mul_5', 'conv_alstm_38/conv_lst_m2d/while/body/_8963/convolution_6' -> 'conv_alstm_38/conv_lst_m2d/while/body/_8963/add_4', 'conv_alstm_38/conv_lst_m2d/while/body/_8963/clip_by_value' -> 'conv_alstm_38/conv_lst_m2d/while/body/_8963/mul_3', 'conv_alstm_38/conv_lst_m2d/while/body/_8963/mul_2' -> 'conv_alstm_38/conv_lst_m2d/while/body/_8963/add_5', 'conv_alstm_39/conv_lst_m2d/while/body/_9182/mul_2' -> 'conv_alstm_39/conv_lst_m2d/while/body/_9182/add_5', 'conv_alstm_39/conv_lst_m2d/while/body/_9182/convolution_6' -> 'conv_alstm_39/conv_lst_m2d/while/body/_9182/add_4', 'conv_alstm_39/conv_lst_m2d/while/body/_9182/clip_by_value' -> 'conv_alstm_39/conv_lst_m2d/while/body/_9182/mul_3', 'conv_alstm_39/conv_lst_m2d/while/body/_9182/clip_by_value_2' -> 'conv_alstm_39/conv_lst_m2d/while/body/_9182/mul_5', 'Func/conv_alstm_38/conv_lst_m2d/while_grad/body/_9592/input/_24401' -> 'conv_alstm_38/conv_lst_m2d/while_grad/body/_9592/gradients/AddN', 'Func/conv_alstm_37/conv_lst_m2d/while_grad/body/_9783/input/_24517' -> 'conv_alstm_37/conv_lst_m2d/while_grad/body/_9783/gradients/AddN', 'conv_alstm_39/conv_lst_m2d/while_grad/body/_9401/gradients/AddN_2' -> 'conv_alstm_39/conv_lst_m2d/while_grad/next_iteration/_9558', 'conv_alstm_34/conv_lst_m2d/while/body/_8087/convolution_6' -> 'conv_alstm_34/conv_lst_m2d/while/body/_8087/add_4', 'conv_alstm_34/conv_lst_m2d/while/body/_8087/clip_by_value' -> 'conv_alstm_34/conv_lst_m2d/while/body/_8087/mul_3', 'conv_alstm_34/conv_lst_m2d/while/body/_8087/clip_by_value_2' -> 'conv_alstm_34/conv_lst_m2d/while/body/_8087/mul_5', 'conv_alstm_32/conv_lst_m2d/while/body/_7649/clip_by_value_2' -> 'conv_alstm_32/conv_lst_m2d/while/body/_7649/mul_5', 'conv_alstm_32/conv_lst_m2d/while/body/_7649/convolution_6' -> 'conv_alstm_32/conv_lst_m2d/while/body/_7649/add_4', 'conv_alstm_32/conv_lst_m2d/while/body/_7649/clip_by_value' -> 'conv_alstm_32/conv_lst_m2d/while/body/_7649/mul_3', 'conv_alstm_31/conv_lst_m2d/while/body/_7430/clip_by_value_2' -> 'conv_alstm_31/conv_lst_m2d/while/body/_7430/mul_5', 'conv_alstm_31/conv_lst_m2d/while/body/_7430/convolution_6' -> 'conv_alstm_31/conv_lst_m2d/while/body/_7430/add_4', 'conv_alstm_31/conv_lst_m2d/while/body/_7430/clip_by_value' -> 'conv_alstm_31/conv_lst_m2d/while/body/_7430/mul_3', 'conv_alstm_28/conv_lst_m2d/while/body/_6773/clip_by_value' -> 'conv_alstm_28/conv_lst_m2d/while/body/_6773/mul_3', 'conv_alstm_28/conv_lst_m2d/while/body/_6773/clip_by_value_2' -> 'conv_alstm_28/conv_lst_m2d/while/body/_6773/mul_5', 'conv_alstm_28/conv_lst_m2d/while/body/_6773/convolution_6' -> 'conv_alstm_28/conv_lst_m2d/while/body/_6773/add_4', 'conv_alstm_3/conv_lst_m2d/while/body/_1298/clip_by_value' -> 'conv_alstm_3/conv_lst_m2d/while/body/_1298/mul_3', 'conv_alstm_3/conv_lst_m2d/while/body/_1298/mul_2' -> 'conv_alstm_3/conv_lst_m2d/while/body/_1298/add_5', 'conv_alstm_3/conv_lst_m2d/while/body/_1298/clip_by_value_2' -> 'conv_alstm_3/conv_lst_m2d/while/body/_1298/mul_5', 'conv_alstm_3/conv_lst_m2d/while/body/_1298/convolution_6' -> 'conv_alstm_3/conv_lst_m2d/while/body/_1298/add_4', 'conv_alstm_4/conv_lst_m2d/while/body/_1517/mul_2' -> 'conv_alstm_4/conv_lst_m2d/while/body/_1517/add_5', 'conv_alstm_5/conv_lst_m2d/while/body/_1736/clip_by_value' -> 'conv_alstm_5/conv_lst_m2d/while/body/_1736/mul_3', 'conv_alstm_5/conv_lst_m2d/while/body/_1736/mul_2' -> 'conv_alstm_5/conv_lst_m2d/while/body/_1736/add_5', 'conv_alstm_5/conv_lst_m2d/while/body/_1736/clip_by_value_2' -> 'conv_alstm_5/conv_lst_m2d/while/body/_1736/mul_5', 'conv_alstm_5/conv_lst_m2d/while/body/_1736/convolution_6' -> 'conv_alstm_5/conv_lst_m2d/while/body/_1736/add_4', 'conv_alstm_6/conv_lst_m2d/while/body/_1955/convolution_6' -> 'conv_alstm_6/conv_lst_m2d/while/body/_1955/add_4', 'conv_alstm_6/conv_lst_m2d/while/body/_1955/clip_by_value' -> 'conv_alstm_6/conv_lst_m2d/while/body/_1955/mul_3', 'conv_alstm_6/conv_lst_m2d/while/body/_1955/mul_2' -> 'conv_alstm_6/conv_lst_m2d/while/body/_1955/add_5', 'conv_alstm_6/conv_lst_m2d/while/body/_1955/clip_by_value_2' -> 'conv_alstm_6/conv_lst_m2d/while/body/_1955/mul_5', 'conv_alstm_7/conv_lst_m2d/while/body/_2174/mul_2' -> 'conv_alstm_7/conv_lst_m2d/while/body/_2174/add_5', 'conv_alstm_8/conv_lst_m2d/while/body/_2393/mul_2' -> 'conv_alstm_8/conv_lst_m2d/while/body/_2393/add_5', 'conv_alstm_9/conv_lst_m2d/while/body/_2612/clip_by_value' -> 'conv_alstm_9/conv_lst_m2d/while/body/_2612/mul_3', 'conv_alstm_9/conv_lst_m2d/while/body/_2612/mul_2' -> 'conv_alstm_9/conv_lst_m2d/while/body/_2612/add_5', 'conv_alstm_9/conv_lst_m2d/while/body/_2612/clip_by_value_2' -> 'conv_alstm_9/conv_lst_m2d/while/body/_2612/mul_5', 'conv_alstm_9/conv_lst_m2d/while/body/_2612/convolution_6' -> 'conv_alstm_9/conv_lst_m2d/while/body/_2612/add_4', 'conv_alstm_10/conv_lst_m2d/while/body/_2831/clip_by_value_2' -> 'conv_alstm_10/conv_lst_m2d/while/body/_2831/mul_5', 'conv_alstm_10/conv_lst_m2d/while/body/_2831/convolution_6' -> 'conv_alstm_10/conv_lst_m2d/while/body/_2831/add_4', 'conv_alstm_10/conv_lst_m2d/while/body/_2831/clip_by_value' -> 'conv_alstm_10/conv_lst_m2d/while/body/_2831/mul_3', 'conv_alstm_10/conv_lst_m2d/while/body/_2831/mul_2' -> 'conv_alstm_10/conv_lst_m2d/while/body/_2831/add_5', 'conv_alstm_11/conv_lst_m2d/while/body/_3050/mul_2' -> 'conv_alstm_11/conv_lst_m2d/while/body/_3050/add_5', 'conv_alstm_11/conv_lst_m2d/while/body/_3050/clip_by_value_2' -> 'conv_alstm_11/conv_lst_m2d/while/body/_3050/mul_5', 'conv_alstm_11/conv_lst_m2d/while/body/_3050/convolution_6' -> 'conv_alstm_11/conv_lst_m2d/while/body/_3050/add_4', 'conv_alstm_11/conv_lst_m2d/while/body/_3050/clip_by_value' -> 'conv_alstm_11/conv_lst_m2d/while/body/_3050/mul_3', 'conv_alstm_12/conv_lst_m2d/while/body/_3269/clip_by_value' -> 'conv_alstm_12/conv_lst_m2d/while/body/_3269/mul_3', 'conv_alstm_12/conv_lst_m2d/while/body/_3269/mul_2' -> 'conv_alstm_12/conv_lst_m2d/while/body/_3269/add_5', 'conv_alstm_12/conv_lst_m2d/while/body/_3269/clip_by_value_2' -> 'conv_alstm_12/conv_lst_m2d/while/body/_3269/mul_5', 'conv_alstm_12/conv_lst_m2d/while/body/_3269/convolution_6' -> 'conv_alstm_12/conv_lst_m2d/while/body/_3269/add_4', 'conv_alstm_13/conv_lst_m2d/while/body/_3488/clip_by_value' -> 'conv_alstm_13/conv_lst_m2d/while/body/_3488/mul_3', 'conv_alstm_13/conv_lst_m2d/while/body/_3488/mul_2' -> 'conv_alstm_13/conv_lst_m2d/while/body/_3488/add_5', 'conv_alstm_13/conv_lst_m2d/while/body/_3488/clip_by_value_2' -> 'conv_alstm_13/conv_lst_m2d/while/body/_3488/mul_5', 'conv_alstm_13/conv_lst_m2d/while/body/_3488/convolution_6' -> 'conv_alstm_13/conv_lst_m2d/while/body/_3488/add_4', 'conv_alstm_14/conv_lst_m2d/while/body/_3707/clip_by_value' -> 'conv_alstm_14/conv_lst_m2d/while/body/_3707/mul_3', 'conv_alstm_14/conv_lst_m2d/while/body/_3707/mul_2' -> 'conv_alstm_14/conv_lst_m2d/while/body/_3707/add_5', 'conv_alstm_14/conv_lst_m2d/while/body/_3707/clip_by_value_2' -> 'conv_alstm_14/conv_lst_m2d/while/body/_3707/mul_5', 'conv_alstm_14/conv_lst_m2d/while/body/_3707/convolution_6' -> 'conv_alstm_14/conv_lst_m2d/while/body/_3707/add_4', 'conv_alstm_15/conv_lst_m2d/while/body/_3926/convolution_6' -> 'conv_alstm_15/conv_lst_m2d/while/body/_3926/add_4', 'conv_alstm_15/conv_lst_m2d/while/body/_3926/clip_by_value' -> 'conv_alstm_15/conv_lst_m2d/while/body/_3926/mul_3', 'conv_alstm_15/conv_lst_m2d/while/body/_3926/mul_2' -> 'conv_alstm_15/conv_lst_m2d/while/body/_3926/add_5', 'conv_alstm_15/conv_lst_m2d/while/body/_3926/clip_by_value_2' -> 'conv_alstm_15/conv_lst_m2d/while/body/_3926/mul_5', 'conv_alstm_16/conv_lst_m2d/while/body/_4145/clip_by_value_2' -> 'conv_alstm_16/conv_lst_m2d/while/body/_4145/mul_5', 'conv_alstm_16/conv_lst_m2d/while/body/_4145/convolution_6' -> 'conv_alstm_16/conv_lst_m2d/while/body/_4145/add_4', 'conv_alstm_16/conv_lst_m2d/while/body/_4145/clip_by_value' -> 'conv_alstm_16/conv_lst_m2d/while/body/_4145/mul_3', 'conv_alstm_16/conv_lst_m2d/while/body/_4145/mul_2' -> 'conv_alstm_16/conv_lst_m2d/while/body/_4145/add_5', 'conv_alstm_17/conv_lst_m2d/while/body/_4364/clip_by_value_2' -> 'conv_alstm_17/conv_lst_m2d/while/body/_4364/mul_5', 'conv_alstm_17/conv_lst_m2d/while/body/_4364/convolution_6' -> 'conv_alstm_17/conv_lst_m2d/while/body/_4364/add_4', 'conv_alstm_17/conv_lst_m2d/while/body/_4364/clip_by_value' -> 'conv_alstm_17/conv_lst_m2d/while/body/_4364/mul_3', 'conv_alstm_17/conv_lst_m2d/while/body/_4364/mul_2' -> 'conv_alstm_17/conv_lst_m2d/while/body/_4364/add_5', 'conv_alstm_18/conv_lst_m2d/while/body/_4583/clip_by_value_2' -> 'conv_alstm_18/conv_lst_m2d/while/body/_4583/mul_5', 'conv_alstm_18/conv_lst_m2d/while/body/_4583/convolution_6' -> 'conv_alstm_18/conv_lst_m2d/while/body/_4583/add_4', 'conv_alstm_18/conv_lst_m2d/while/body/_4583/clip_by_value' -> 'conv_alstm_18/conv_lst_m2d/while/body/_4583/mul_3', 'conv_alstm_18/conv_lst_m2d/while/body/_4583/mul_2' -> 'conv_alstm_18/conv_lst_m2d/while/body/_4583/add_5', 'conv_alstm_19/conv_lst_m2d/while/body/_4802/mul_2' -> 'conv_alstm_19/conv_lst_m2d/while/body/_4802/add_5', 'conv_alstm_20/conv_lst_m2d/while/body/_5021/mul_2' -> 'conv_alstm_20/conv_lst_m2d/while/body/_5021/add_5', 'conv_alstm_20/conv_lst_m2d/while/body/_5021/clip_by_value_2' -> 'conv_alstm_20/conv_lst_m2d/while/body/_5021/mul_5', 'conv_alstm_20/conv_lst_m2d/while/body/_5021/convolution_6' -> 'conv_alstm_20/conv_lst_m2d/while/body/_5021/add_4', 'conv_alstm_20/conv_lst_m2d/while/body/_5021/clip_by_value' -> 'conv_alstm_20/conv_lst_m2d/while/body/_5021/mul_3', 'conv_alstm_19/conv_lst_m2d/while/body/_4802/clip_by_value_2' -> 'conv_alstm_19/conv_lst_m2d/while/body/_4802/mul_5', 'conv_alstm_19/conv_lst_m2d/while/body/_4802/convolution_6' -> 'conv_alstm_19/conv_lst_m2d/while/body/_4802/add_4', 'conv_alstm_19/conv_lst_m2d/while/body/_4802/clip_by_value' -> 'conv_alstm_19/conv_lst_m2d/while/body/_4802/mul_3', 'conv_alstm_8/conv_lst_m2d/while/body/_2393/clip_by_value_2' -> 'conv_alstm_8/conv_lst_m2d/while/body/_2393/mul_5', 'conv_alstm_8/conv_lst_m2d/while/body/_2393/convolution_6' -> 'conv_alstm_8/conv_lst_m2d/while/body/_2393/add_4', 'conv_alstm_8/conv_lst_m2d/while/body/_2393/clip_by_value' -> 'conv_alstm_8/conv_lst_m2d/while/body/_2393/mul_3', 'conv_alstm_7/conv_lst_m2d/while/body/_2174/clip_by_value_2' -> 'conv_alstm_7/conv_lst_m2d/while/body/_2174/mul_5', 'conv_alstm_7/conv_lst_m2d/while/body/_2174/convolution_6' -> 'conv_alstm_7/conv_lst_m2d/while/body/_2174/add_4', 'conv_alstm_7/conv_lst_m2d/while/body/_2174/clip_by_value' -> 'conv_alstm_7/conv_lst_m2d/while/body/_2174/mul_3', 'conv_alstm_4/conv_lst_m2d/while/body/_1517/convolution_6' -> 'conv_alstm_4/conv_lst_m2d/while/body/_1517/add_4', 'conv_alstm_4/conv_lst_m2d/while/body/_1517/clip_by_value' -> 'conv_alstm_4/conv_lst_m2d/while/body/_1517/mul_3', 'conv_alstm_4/conv_lst_m2d/while/body/_1517/clip_bdy/_3488/mul_3', 'conv_alstm_13/conv_lst_m2d/while/body/_3488/mul_2' -> 'conv_alstm_13/conv_lst_m2d/while/body/_3488/add_5', 'conv_alstm_13/conv_lst_m2d/while/body/_3488/clip_by_value_2' -> 'conv_alstm_13/conv_lst_m2d/while/body/_3488/mul_5', 'conv_alstm_13/conv_lst_m2d/while/body/_3488/convolution_6' -> 'conv_alstm_13/conv_lst_m2d/while/body/_3488/add_4', 'conv_alstm_14/conv_lst_m2d/while/body/_3707/clip_by_value' -> 'conv_alstm_14/conv_lst_m2d/while/body/_3707/mul_3', 'conv_alstm_14/conv_lst_m2d/while/body/_3707/mul_2' -> 'conv_alstm_14/conv_lst_m2d/while/body/_3707/add_5', 'conv_alstm_14/conv_lst_m2d/while/body/_3707/clip_by_value_2' -> 'conv_alstm_14/conv_lst_m2d/while/body/_3707/mul_5', 'conv_alstm_14/conv_lst_m2d/while/body/_3707/convolution_6' -> 'conv_alstm_14/conv_lst_m2d/while/body/_3707/add_4', 'conv_alstm_15/conv_lst_m2d/while/body/_3926/convolution_6' -> 'conv_alstm_15/conv_lst_m2d/while/body/_3926/add_4', 'conv_alstm_15/conv_lst_m2d/while/body/_3926/clip_by_value' -> 'conv_alstm_15/conv_lst_m2d/while/body/_3926/mul_3', 'conv_alstm_15/conv_lst_m2d/while/body/_3926/mul_2' -> 'conv_alstm_15/conv_lst_m2d/while/body/_3926/add_5', 'conv_alstm_15/conv_lst_m2d/while/body/_3926/clip_by_value_2' -> 'conv_alstm_15/conv_lst_m2d/while/body/_3926/mul_5', 'conv_alstm_16/conv_lst_m2d/while/body/_4145/clip_by_value_2' -> 'conv_alstm_16/conv_lst_m2d/while/body/_4145/mul_5', 'conv_alstm_16/conv_lst_m2d/while/body/_4145/convolution_6' -> 'conv_alstm_16/conv_lst_m2d/while/body/_4145/add_4', 'conv_alstm_16/conv_lst_m2d/while/body/_4145/clip_by_value' -> 'con v_alstm_16/conv_lst_m2d/while/body/_4145/mul_3', 'conv_alstm_16/conv_lst_m2d/while/body/_4145/mul_2' -> 'conv_alstm_16/conv_lst_m2d/while/body/_4145/add_5', 'conv_alstm_17/conv_lst_m2d/while/body/_ 4364/clip_by_value_2' -> 'conv_alstm_17/conv_lst_m2d/while/body/_4364/mul_5', 'conv_alstm_17/conv_lst_m2d/while/body/_4364/convolution_6' -> 'conv_alstm_17/conv_lst_m2d/while/body/_4364/add_4', 'co nv_alstm_17/conv_lst_m2d/while/body/_4364/clip_by_value' -> 'conv_alstm_17/conv_lst_m2d/while/body/_4364/mul_3', 'conv_alstm_17/conv_lst_m2d/while/body/_4364/mul_2' -> 'conv_alstm_17/conv_lst_m2d/w hile/body/_4364/add_5', 'conv_alstm_18/conv_lst_m2d/while/body/_4583/clip_by_value_2' -> 'conv_alstm_18/conv_lst_m2d/while/body/_4583/mul_5', 'conv_alstm_18/conv_lst_m2d/while/body/_4583/convolutio n_6' -> 'conv_alstm_18/conv_lst_m2d/while/body/_4583/add_4', 'conv_alstm_18/conv_lst_m2d/while/body/_4583/clip_by_value' -> 'conv_alstm_18/conv_lst_m2d/while/body/_4583/mul_3', 'conv_alstm_18/conv_ lst_m2d/while/body/_4583/mul_2' -> 'conv_alstm_18/conv_lst_m2d/while/body/_4583/add_5', 'conv_alstm_19/conv_lst_m2d/while/body/_4802/mul_2' -> 'conv_alstm_19/conv_lst_m2d/while/body/_4802/add_5', ' conv_alstm_20/conv_lst_m2d/while/body/_5021/mul_2' -> 'conv_alstm_20/conv_lst_m2d/while/body/_5021/add_5', 'conv_alstm_20/conv_lst_m2d/while/body/_5021/clip_by_value_2' -> 'conv_alstm_20/conv_lst_m 2d/while/body/_5021/mul_5', 'conv_alstm_20/conv_lst_m2d/while/body/_5021/convolution_6' -> 'conv_alstm_20/conv_lst_m2d/while/body/_5021/add_4', 'conv_alstm_20/conv_lst_m2d/while/body/_5021/clip_by_ value' -> 'conv_alstm_20/conv_lst_m2d/while/body/_5021/mul_3', 'conv_alstm_19/conv_lst_m2d/while/body/_4802/clip_by_value_2' -> 'conv_alstm_19/conv_lst_m2d/while/body/_4802/mul_5', 'conv_alstm_19/c onv_lst_m2d/while/body/_4802/convolution_6' -> 'conv_alstm_19/conv_lst_m2d/while/body/_4802/add_4', 'conv_alstm_19/conv_lst_m2d/while/body/_4802/clip_by_value' -> 'conv_alstm_19/conv_lst_m2d/while/ body/_4802/mul_3', 'conv_alstm_8/conv_lst_m2d/while/body/_2393/clip_by_value_2' -> 'conv_alstm_8/conv_lst_m2d/while/body/_2393/mul_5', 'conv_alstm_8/conv_lst_m2d/while/body/_2393/convolution_6' -> 'conv_alstm_8/conv_lst_m2d/while/body/_2393/add_4', 'conv_alstm_8/conv_lst_m2d/while/body/_2393/clip_by_value' -> 'conv_alstm_8/conv_lst_m2d/while/body/_2393/mul_3', 'conv_alstm_7/conv_lst_m2d/whil e/body/_2174/clip_by_value_2' -> 'conv_alstm_7/conv_lst_m2d/while/body/_2174/mul_5', 'conv_alstm_7/conv_lst_m2d/while/body/_2174/convolution_6' -> 'conv_alstm_7/conv_lst_m2d/while/body/_2174/add_4' , 'conv_alstm_7/conv_lst_m2d/while/body/_2174/clip_by_value' -> 'conv_alstm_7/conv_lst_m2d/while/body/_2174/mul_3', 'conv_alstm_4/conv_lst_m2d/while/body/_1517/convolution_6' -> 'conv_alstm_4/conv_ lst_m2d/while/body/_1517/add_4', 'conv_alstm_4/conv_lst_m2d/while/body/_1517/clip_by_value' -> 'conv_alstm_4/conv_lst_m2d/while/body/_1517/mul_3', 'conv_alstm_4/conv_lst_m2d/while/body/_1517/clip_b y_value_2' -> 'conv_alstm_4/conv_lst_m2d/while/body/_1517/mul_5', 'conv_alstm_1/conv_lst_m2d/while/body/_860/clip_by_value' -> 'conv_alstm_1/conv_lst_m2d/while/body/_860/mul_3', 'conv_alstm_1/conv_ lst_m2d/while/body/_860/mul_2' -> 'conv_alstm_1/conv_lst_m2d/while/body/_860/add_5', 'conv_alstm_1/conv_lst_m2d/while/body/_860/clip_by_value_2' -> 'conv_alstm_1/conv_lst_m2d/while/body/_860/mul_5' , 'conv_alstm_1/conv_lst_m2d/while/body/_860/convolution_6' -> 'conv_alstm_1/conv_lst_m2d/while/body/_860/add_4', 'conv_alstm_2/conv_lst_m2d/while/body/_1079/clip_by_value' -> 'conv_alstm_2/conv_ls t_m2d/while/body/_1079/mul_3', 'conv_alstm_2/conv_lst_m2d/while/body/_1079/mul_2' -> 'conv_alstm_2/conv_lst_m2d/while/body/_1079/add_5', 'conv_alstm_2/conv_lst_m2d/while/body/_1079/clip_by_value_2' -> 'conv_alstm_2/conv_lst_m2d/while/body/_1079/mul_5', 'conv_alstm_2/conv_lst_m2d/while/body/_1079/convolution_6' -> 'conv_alstm_2/conv_lst_m2d/while/body/_1079/add_4', 'conv_alstm/conv_lst_m2d/wh ile/body/_641/mul_2' -> 'conv_alstm/conv_lst_m2d/while/body/_641/add_5', 'conv_alstm/conv_lst_m2d/while/body/_641/clip_by_value' -> 'conv_alstm/conv_lst_m2d/while/body/_641/mul_3', 'conv_alstm/conv _lst_m2d/while/body/_641/clip_by_value_2' -> 'conv_alstm/conv_lst_m2d/while/body/_641/mul_5', 'conv_alstm/conv_lst_m2d/while/body/_641/convolution_6' -> 'conv_alstm/conv_lst_m2d/while/body/_641/add _4'}.