tensorflow: Keras.gradients() returns None in loss function

Good day and thank you for TensorFlow 😃

I’m trying to implement a Wasserstein GAN with gradient penalty in TF2 (https://arxiv.org/pdf/1704.00028.pdf) When I compute the gradient of the discriminator output with respect to the interpolated input, I always get None. I have tried to compute the gradient in the layers of my model and in my loss function, in eager mode or not, using a GradientTape and I have checked that my discriminator has gradients activated. To reproduce the issue, I have adapted the implementation here to work with TensorFlow as backend and I get exactly the same problem. (Link to adapted version below)

What must I do to get this to work?

Thank you very much 😃

System information

== check python ===================================================
python version: 3.7.4
python branch: v3.7.4
python build version: ('v3.7.4:e09359112e', 'Jul  8 2019 14:54:52')
python compiler version: Clang 6.0 (clang-600.0.57)
python implementation: CPython


== check os platform ===============================================
os: Darwin
os kernel version: Darwin Kernel Version 18.7.0: Sun Dec  1 18:59:03 PST 2019; root:xnu-4903.278.19~1/RELEASE_X86_64
os release version: 18.7.0
os platform: Darwin-18.7.0-x86_64-i386-64bit
linux distribution: ('', '', '')
linux os distribution: ('', '', '')
mac version: ('10.14.6', ('', '', ''), 'x86_64')
uname: uname_result(system='Darwin', node='MBP-van-Michel', release='18.7.0', version='Darwin Kernel Version 18.7.0: Sun Dec  1 18:59:03 PST 2019; root:xnu-4903.278.19~1/RELEASE_X86_64', machine='x86_64', processor='i386')
architecture: ('64bit', '')
machine: x86_64


== are we in docker =============================================
No

== compiler =====================================================
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

== check pips ===================================================
numpy                    1.18.0    
protobuf                 3.11.2    
tensorflow               2.1.0     
tensorflow-datasets      1.3.2     
tensorflow-estimator     2.1.0     
tensorflow-metadata      0.15.2    

== check for virtualenv =========================================
False

== tensorflow import ============================================
tf.version.VERSION = 2.1.0
tf.version.GIT_VERSION = v2.1.0-rc2-17-ge5bf8de410
tf.version.COMPILER_VERSION = 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.46.4)

== env ==========================================================
LD_LIBRARY_PATH is unset
DYLD_LIBRARY_PATH is unset

== nvidia-smi ===================================================
./env.sh: line 147: nvidia-smi: command not found

== cuda libs  ===================================================

== tensorflow installed from info ==================
Name: tensorflow
Version: 2.1.0
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author-email: packages@tensorflow.org
License: Apache 2.0
Location: /Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages
Required-by: 

== python version  ==============================================
(major, minor, micro, releaselevel, serial)
(3, 7, 4, 'final', 0)

== bazel version  ===============================================

Describe the current behavior

python src/cwan_gp.py
2020-02-03 14:37:17.446341: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fef2a728470 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-02-03 14:37:17.446368: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
Tensor("loss/model_1_2_loss/gradients/model_1_2/flatten_3/Reshape_grad/Reshape:0", shape=(32, 28, 28, 1), dtype=float32)
None
Traceback (most recent call last):
  File "src/cwan_gp.py", line 302, in <module>
    wgan.train()
  File "src/cwan_gp.py", line 232, in train
    d_loss = self.critic_model.train_on_batch([imgs, labels, noise], [valid, fake, dummy])
  File "/Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py", line 1078, in train_on_batch
    standalone=True)
  File "/Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2_utils.py", line 433, in train_on_batch
    output_loss_metrics=model._output_loss_metrics)
  File "/Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py", line 568, in __call__
    result = self._call(*args, **kwds)
  File "/Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py", line 615, in _call
    self._initialize(args, kwds, add_initializers_to=initializers)
  File "/Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py", line 497, in _initialize
    *args, **kwds))
  File "/Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py", line 2389, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "/Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py", line 2703, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/eager/function.py", line 2593, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py", line 978, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/eager/def_function.py", line 439, in wrapped_fn
    return weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "/Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/framework/func_graph.py", line 968, in wrapper
    raise e.ag_error_metadata.to_exception(e)
ValueError: in converted code:

    /Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_eager.py:305 train_on_batch  *
        outs, total_loss, output_losses, masks = (
    /Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_eager.py:253 _process_single_batch
        training=training))
    /Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_eager.py:167 _model_loss
        per_sample_losses = loss_fn.call(targets[i], outs[i])
    /Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/keras/losses.py:221 call
        return self.fn(y_true, y_pred, **self._fn_kwargs)
    src/cwan_gp.py:123 gradient_penalty_loss
        gradients_sqr = K.square(gradients)
    /Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/keras/backend.py:2162 square
        return math_ops.square(x)
    /Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_math_ops.py:9965 square
        "Square", x=x, name=name)
    /Users/michel/workspace/tf-text2image/venv/lib/python3.7/site-packages/tensorflow_core/python/framework/op_def_library.py:486 _apply_op_helper
        (input_name, err))

    ValueError: Tried to convert 'x' to a tensor and failed. Error: None values not supported.

Describe the expected behavior The computed gradient is not None

Code to reproduce the issue This is a minimal example to reproduce the error https://github.com/MichelHalmes/tf-text2image/blob/72df135786c7273fb2e0bea27a30ab60024e92b2/src/cwan_gp.py

Other info / logs This person belivees that doing higher order gradients is only possible with theano: https://github.com/LuEE-C/WGAN-GP-with-keras-for-text/blob/master/README.md

This person seems to have had the same issue about a year ago : https://stackoverflow.com/questions/54076901/k-gradients-returning-none

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 15 (3 by maintainers)

Most upvoted comments

Hi! I suggest you to create a custom model class the way it can be done via tensorflow 2. Here I talked about the resources I followed up. I did not get good results for 1D data btw.

Best regards!

pabloi09 on Apr 28, 2020

Just an update: I managed to get around my issue by converting the interpolated_image into a tf.Variable(). I compute the gradient outside the Model using a GradientTape

In the example above however, since the gradient is computed as part of the model, get the following issue:

ValueError: Tensor-typed variable initializers must either be wrapped in an init_scope or callable (e.g., `tf.Variable(lambda : tf.truncated_normal([10, 40]))`) when building functions. Please file a feature request if this restriction inconveniences you.

I tried both suggested solutions without success.

This unblocks me, but I’m still surprised that this doesn’t work.

Thanks for your help and for TF-2 👍

MichelHalmes on Feb 7, 2020

Was able to reproduce the issue with tf 2.1. Please see the gist here. Thanks!

gadagashwini-zz on Feb 4, 2020