tensorflow: model.fit with tf.data.Dataset.from_generator can't infer shape

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOs 10.13.6
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): v2.0.0-beta1-5101-gc75bb66a99 2.0.0-rc0
Python version: 3.6.4
Bazel version (if compiling from source): N/A
GCC/Compiler version (if compiling from source): N/A
CUDA/cuDNN version: N/A
GPU model and memory: N/A

Describe the current behavior

model.fit when given a tf.data dataset (that uses a from_generator) throws ValueError: as_list() is not defined on an unknown TensorShape.

see https://gist.github.com/matpalm/779c2c67d5ec195845ae2ab01570d883 for full traceback

Describe the expected behavior

model.fit should be able to infer the shape of the input.

Code to reproduce the issue

import tensorflow as tf
from tensorflow.keras.layers import *
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import Adam

inp = Input(shape=(3,))
output = Dense(1, activation='sigmoid')(inp)
model = Model(inp, output)
model.compile(optimizer=Adam(1e-2), loss='binary_crossentropy')

def simple_generator():
    while True:
        yield [0.5, 0.2, -0.3], 0.0
        yield [-0.5, 0.3, -0.1], 1.0
        
dataset = tf.data.Dataset.from_generator(simple_generator,
                                         output_types=(tf.float32,
                                                       tf.float32))
dataset = dataset.batch(4).prefetch(1)

model.fit(dataset)

throws ValueError: as_list() is not defined on an unknown TensorShape.

note: the following works…

for X, y in dataset:
  model.fit(X, y)
  break

Other info / logs

see https://gist.github.com/matpalm/779c2c67d5ec195845ae2ab01570d883 for full traceback

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 8
Comments: 33 (2 by maintainers)

Most upvoted comments

Adding the following after batching allowed me to work around this issue:

def _fixup_shape(images, labels, weights):
    images.set_shape([None, None, None, 3])
    labels.set_shape([None, 19]) # I have 19 classes
    weights.set_shape([None])
    return images, labels, weights
dataset = dataset.map(_fixup_shape)

+20

jpambrun on Nov 6, 2019

Ran into the same issue when trying to use model.fit() with a tf.data.dataset initialized from a generator using 2.0.0 stable.

For now as a workaround model.fit_generator() seems to work on the same dataset object.

caesarnine on Oct 9, 2019

In my case problem was in using many ‘accuracy’ metric which requires IteratorGetNext shape which I didn’t have.

def _fixup_shape(images, labels, weights):
    images.set_shape([None, None, None, 3])
    labels.set_shape([None, 19]) # I have 19 classes
    weights.set_shape([None])
    return images, labels, weights
dataset = dataset.map(_fixup_shape)

@jpambrun’s solution worked for me!

NOTE: it doesn’t matter whether you first do batch or map. But you’ll need two map functions! The first is used for getting data, the other one setting the shape. Doing everything in one function fails, I don’t know why exactly, even if you convert data to Tensor and set_shape before returning it. So I have the following order dataset.batch.map(get_data).map(fix_shape).prefetch and everything goes smoothly.

chrismaliszewski on Jan 26, 2021

Running into the same issue. fit_generator is not an option for us due to need for DistributionStrategy.

ding-c3 on Feb 26, 2020