keras: Attribute Error: Custom Generator object has no attribute 'shape'

I am using a custom data generator, when I try using fit_generator it throws an error saying “object has no attribute ‘shape’”. This is being run on a Google Colab environment.

I followed this simple tutorial for the implementation of the generator: https://medium.com/datadriveninvestor/keras-training-on-large-datasets-3e9d9dbc09d4

Data is being inputted as a list.

Tensorflow version: 1.13 Keras version: 2.2.4

Error: /usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/engine/training.pyc in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch) 1424 use_multiprocessing=use_multiprocessing, 1425 shuffle=shuffle, -> 1426 initial_epoch=initial_epoch) 1427 1428 def evaluate_generator(self,

/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/engine/training_generator.pyc in model_iteration(model, data, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch, mode, batch_size, **kwargs) 113 batch_size=batch_size, 114 epochs=epochs - initial_epoch, –> 115 shuffle=shuffle) 116 117 do_validation = validation_data is not None

/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/engine/training_generator.pyc in convert_to_generator_like(data, batch_size, steps_per_epoch, epochs, shuffle) 375 376 # Create generator from NumPy or EagerTensor Input. –> 377 num_samples = int(nest.flatten(data)[0].shape[0]) 378 if batch_size is None: 379 raise ValueError(‘You must specify batch_size’)

AttributeError: ‘MY_Generator’ object has no attribute ‘shape’

gist: https://gist.github.com/fjur/2815f235f84b8b666107207599482428

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 18

Commits related to this issue

Most upvoted comments

I ran into the same problem. Importing the Sequence class from Tensorflow fixed it for me:

from tensorflow.python.keras.utils.data_utils import Sequence

Thanks that solved it! I was using from keras.utils import Sequence which seemed to be causing issues.

I face with the same issue. If your generator returns a list (something like: [X, Y]), try return a tuple (X, Y). It helps me.

I had the same problem. It worked fine in the jupyter notebook but if I put the exact cell code into a python file and imported the class it would throw the error.

The problem was, I imported keras in the python file with

import keras

Changing it to:

from tensorflor import keras

fixed the problem problem for me.

I guess the naming is ambiguous.

My script kept getting error. While importing the PreProcessing from keras: Use below: from tensorflow.keras.preprocessing.sequence import TimeseriesGenerator Instead of: from keras.preprocessing.sequence import TimeseriesGenerator