tensorflow: dataset windows don't work in batches

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template

System information

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow):yes (see below)
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04):Windows 10 64 bit
  • TensorFlow installed from (source or binary): pip install tensorflow, pip install tensorflow-nightly
  • TensorFlow version (use command below):Currently on nightly - v1.12.1-43485-g0af213f96c 2.4.0-dev20201012 , but also happens on 2.3 release.
  • Python version: Python 3.8.5 (default, Sep 3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc.
  • Bazel version (if compiling from source):
  • GCC/Compiler version (if compiling from source):
  • CUDA/cuDNN version: Tried 10.1 and 11.0
  • GPU model and memory: GeForce RTX 2080 with Max-Q Design computeCapability: 7.5 coreClock: 1.23GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 357.69GiB/s

Describe the current behavior

If I create a dataset of windowed data, then batch it up using tf.data.Dataset.window(5,1,1,True).batch(10,True) the output of iterating the resulting dataset doesn’t evaluate as a tensor and I can’t run model.fit() on it. It throws an exception:

TypeError: Inputs to a layer should be tensors. Got: <tensorflow.python.data.ops.dataset_ops._NestedVariant object at 0x000001BC82925250>

Describe the expected behavior

This not working basically breaks the tf.data loading mechanism for windowed data for me. I want to be able to create a preprocessing pipeline that does the following:

  1. load some data
  2. apply a sliding window to the data
  3. batch the data
  4. prefetch the data
  5. run model.fit

Standalone code to reproduce the issue

import tensorflow as tf

# make a source dataset
source_data=tf.data.Dataset.range(100)
# take two windows of different size onto the source dataset as
# our inputs and outputs
# and batch them up into 20 data points
inputs=source_data.window(10,1,1,True).batch(20,True)
targets=source_data.window(5,1,1,True).batch(20,True)
# zip this into a single training data set
training_set=tf.data.Dataset.zip((inputs,targets))
# a very simple network that takes input of [batch_size,10] and outputs [batch_size,5]
inLayer=tf.keras.layers.Input(shape=(10,),batch_size=20)
outLayer=tf.keras.layers.Dense(5)(inLayer)
model=tf.keras.Model(inputs=inLayer,outputs=outLayer)
model.compile(optimizer='adam',loss='mse')
# this fit call fails
model.fit(training_set)

Other info / logs Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

Traceback (most recent call last):
  File "d:/machinelearning/predictiveaudio/hellogoodbye/testtf.py", line 20, in <module>
    model.fit(training_set)
  File "C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1081, in fit
    tmp_logs = self.train_function(iterator)
  File "C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\def_function.py", line 828, in __call__
    result = self._call(*args, **kwds)
  File "C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\def_function.py", line 871, in _call
    self._initialize(args, kwds, add_initializers_to=initializers)
  File "C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\def_function.py", line 725, in _initialize
    self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access
  File "C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\function.py", line 2976, in _get_concrete_function_internal_garbage_collected  
    graph_function, _ = self._maybe_define_function(args, kwargs)
  File "C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\function.py", line 3371, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\function.py", line 3206, in _create_graph_function
    func_graph_module.func_graph_from_py_func(
  File "C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\framework\func_graph.py", line 990, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\eager\def_function.py", line 634, in wrapped_fn
    out = weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\framework\func_graph.py", line 977, in wrapper
    raise e.ag_error_metadata.to_exception(e)
TypeError: in user code:

    C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\keras\engine\training.py:790 train_function  *
        return step_function(self, iterator)
    C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\keras\engine\training.py:780 step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:1268 run
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:2734 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:3355 _call_for_each_replica
        return fn(*args, **kwargs)
    C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\keras\engine\training.py:773 run_step  **
        outputs = model.train_step(data)
    C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\keras\engine\training.py:739 train_step
        y_pred = self(x, training=True)
    C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\keras\engine\base_layer.py:989 __call__
        input_spec.assert_input_compatibility(self.input_spec, inputs, self.name)
    C:\Users\micro\miniconda3\envs\tf23\lib\site-packages\tensorflow\python\keras\engine\input_spec.py:201 assert_input_compatibility
        raise TypeError('Inputs to a layer should be tensors. Got: %s' % (x,))

    TypeError: Inputs to a layer should be tensors. Got: <tensorflow.python.data.ops.dataset_ops._NestedVariant object at 0x000001BC82925250>

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 2
  • Comments: 16 (3 by maintainers)

Most upvoted comments

The batches should contain a series of values which are a sliding window over the underlying dataset. i.e. if I have a dataset [1,2,3,4,5,6,7,8,9,10], window size 3, batch size 2, I want tensors to come out like: [ [ 1,2,3],[2,3,4] ] [ [3,4,5],[4,5,6] ]

This is surely what window is for? It is to transform a dataset into another dataset, just like batch is? I can’t see what window is useful for if it doesn’t work to window time series data.

Essentially, it seems like when you are producing datasets using dataset transformations, a basic minimum expectation is that when you transform a dataset, the resulting transformed dataset should itself be able to generate input to model.fit?

Maybe it needs a different window method, but realistically, there’s no reason why it shouldn’t be possible to have a transformation which is used by model.fit which goes from a collection of N datasets of size AxB to tensors in the form NxAxB, without having to do flat_map on the structure.

The horrible workaround is to do a flat_map then batch twice. Which at least on my dataset seems slow.

d=tf.data.Dataset.range(10)
w=d.window(3,1,1)
w = w.flat_map(lambda x:x).batch(3).batch(2) #flat map but do nothing to values, then batch twice to get what model.fit expects from windowed datasets