tensorboard: TensorFlow v2: Graph does not appear on TensorBoard

@cottrell Yup correct. Pure eager builds no graph. Keras always has built some graph underneath (the details escape me at the moment but IIRC, in v1, it built a graph using its own session and, in v2, it constructs graph with FuncGraph) and is able to show some graph unless you pass run_eagerly to its compile method.

I have the same problem

Device: macbook pro 2018 , macOS Mojave 10.14.5 tensorflow version: tensorflow==2.0.0a0 tensorflow-estimator-2.0-preview==1.14.0.dev2019052100

I have the same issues too, i followed the code shows in https://tensorflow.org/tensorboard/r2/graphs the most below method.

# The function to be traced.
@tf.function
def my_func(x, y):
  # A simple hand-rolled layer.
  return tf.nn.relu(tf.matmul(x, y))

# Set up logging.
stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
logdir = 'logs/func/%s' % stamp
writer = tf.summary.create_file_writer(logdir)

# Sample data for your function.
x = tf.random.uniform((3, 3))
y = tf.random.uniform((3, 3))

# Bracket the function call with
# tf.summary.trace_on() and tf.summary.trace_export().
tf.summary.trace_on(graph=True, profiler=True)
# Call only one tf.function when tracing.
z = my_func(x, y)
with writer.as_default():
  tf.summary.trace_export(
      name="my_func_trace",
      step=0,
      profiler_outdir=logdir)

in the tutorials, it shows the graphs, however when i ran this code in my environment it doesn’t shows the graph in the tensorboard

So to clarify, if you use eager only code, you can not debug with tensorboard graphs? I’m confused as to how the keras writer does this. Maybe that is a tf.function under the hood now.

Hi @stephanwlee,

Thanks for your answer and your insights, I’ve been able to change my code a little bit and make tit work. 👍🏻 Just in case other people stumble upon this, the following is a small script that highlights one of the differences between autograph.to_graph and tf.function

import os
import tensorflow as tf

file_dir = os.path.realpath(os.path.dirname(__file__))
writer = tf.summary.create_file_writer(file_dir)

print('1, executing_eagerly: {}'.format(tf.executing_eagerly()))

with tf.Graph().as_default():
    print('2, executing_eagerly: {}'.format(tf.executing_eagerly()))


# @tf.function
def foo(v, name=''):
    tf.print(name + ', executing_eagerly: {}'.format(tf.executing_eagerly()))
    v.assign(tf.math.sigmoid(v))


v = tf.Variable([1, 2], dtype=tf.float32)
foo(v, 'foo')

tf.summary.trace_on(graph=True)

foo_to_g = tf.autograph.to_graph(foo)
foo_to_g(v, tf.constant('foo_to_g', dtype=tf.string))

foo_tf_func = tf.function(foo)
foo_tf_func(v, tf.constant('foo_tf_func', dtype=tf.string))

with writer.as_default():   
    tf.summary.trace_export(
        name='tf2_graph',
        step=0
    )

Which outputs

1, executing_eagerly: True
2, executing_eagerly: False
foo, executing_eagerly: True
foo_to_g, executing_eagerly: True
foo_tf_func, executing_eagerly: False

Ok now we have a ticket at https://github.com/tensorflow/profiler/issues/503. I hope that we don’t need to do too much forth and back and that for the ownership between repos 😸

/cc @MarkDaoust This bug it is very annoying and it is confusing new users especially cause we still have the same example in the official documentation on the TF website and that this was submitted in 2019.

I am running into a graph not appearing on TensorBoard after wrapping one of the tf.keras.applications models (Namely MobileNet) with tf.function and then following the suggested code snippets.

import tensorflow as tf
from datetime import datetime

model = tf.keras.applications.MobileNet()

# The function to be traced.
@tf.function()
def optimized_model(x):
    return model(x)

# Set up logging.
stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
logdir = 'logs\\func\\%s' % stamp
writer = tf.summary.create_file_writer(logdir)

# Sample data for your function, ImageNet standard size
x = tf.random.uniform((1, 224, 224,3))

# Bracket the function call with
# tf.summary.trace_on() and tf.summary.trace_export().
tf.summary.trace_on(graph=True, profiler=True)
# Call only one tf.function when tracing.
z = optimized_model(x)
with writer.as_default():
    tf.summary.trace_export(name="my_func_trace",step=0,profiler_outdir=logdir)

Does anyone have an idea on why does this happen?

FYI, I am running tensorflow==2.1.0 and tensorboard==2.1.1 (latest stable builds installed with pip install tensorflow), and tf.executing_eagerly() is certainly False for the optimized_model function.

When using a model which inherits from tensorflow.python.keras.Model, there is a graph which can be visualized! This even works with run_eagerly=True. The trick is to save the graph at the right moment. Such a moment occurs, when keras/tensorflow builds the model, because it invokes model.call in non-eager mode (not exactly sure why). The resulting tensor holds the model graph (without losses afaik), which can be written using the “old” FileWriter. Example code that works for me with Tensorflow 1.13.1 and enabled eager execution:

import tensorflow
# Only use tensorflow's keras!
from tensorflow.python import keras as tfkeras
from tensorflow.python.training.rmsprop import RMSPropOptimizer
import numpy as np

tensorflow.enable_eager_execution()


class MyModel(tfkeras.Model):
    def __init__(self, tensorboard_folder_path):
        super(MyModel, self).__init__()
        self.dense1 = tfkeras.layers.LSTM(units=6)
        self.dense2 = tfkeras.layers.Dense(units=4)
        self.graph_has_been_written = False
        self.tensorboard_folder_path = tensorboard_folder_path

    def call(self, input, **kwargs):
        print("input shape", input.shape)
        result = self.dense1(input)
        result = self.dense2(result)
        if not tensorflow.executing_eagerly() and not self.graph_has_been_written:
            # In non eager mode and a graph is available which can be written to Tensorboard using the "old" FileWriter:
            model_graph = result.graph
            writer = tensorflow.summary.FileWriter(logdir=self.tensorboard_folder_path, graph=model_graph)
            writer.flush()
            self.graph_has_been_written = True
            print("Wrote eager graph to", self.tensorboard_folder_path)
        return result


if __name__ == "__main__":
    print("Eager execution:", tensorflow.executing_eagerly())
    # Create model and specify tensorboard folder:
    model = MyModel("/home/your_username/tensorboardtest/")
    optimizer = RMSPropOptimizer(learning_rate=0.001)
    model.compile(optimizer, tensorflow.losses.softmax_cross_entropy, run_eagerly=True)
    # Build the model (this will invoke model.call in non-eager mode). If model.build is not called explicitly here, it
    # will be called by model.fit_generator implicitly when the first batch is about to be feed to the network.
    model.build((None, None, 5))
    # Can only be called after the model has been built:
    model.summary()

    # Two arbitrary batches with different batch size and different sequence length:
    x1 = np.array([[[1, 2, 3, 4, 5], [1, 2, 3, 4, 5], [1, 2, 3, 4, 5], [1, 2, 3, 4, 5]]],
                  dtype=np.float32)
    y1 = np.array([[1, 0, 0, 0]], dtype=np.float32)
    print("x1 shape", x1.shape)
    print("y1 shape", y1.shape)

    x2 = np.array([[[1, 2, 3, 4, 5], [1, 2, 3, 4, 5], [1, 2, 3, 4, 5]],
                   [[1, 2, 3, 4, 5], [1, 2, 3, 4, 5], [1, 2, 3, 4, 5]]], dtype=np.float32)
    y2 = np.array([[1, 0, 0, 0], [1, 0, 0, 0]], dtype=np.float32)
    print("x2 shape", x2.shape)
    print("y2 shape", y2.shape)

    # Simply yield the two batches alternately
    def iterator():
        switcher = False
        while 1:
            if switcher:
                yield x1, y1
            else:
                yield x2, y2
            switcher = not switcher

    model.fit_generator(iterator(), steps_per_epoch=10, epochs=1)

Latter is a bug of TensorFlow’s and former is working as intended; if your code is purely eager, there is absolutely no computational graphs and thus the graph plugin cannot do anything about it.