keras: Attribute Error: Custom Generator object has no attribute 'shape'
I am using a custom data generator, when I try using fit_generator it throws an error saying “object has no attribute ‘shape’”. This is being run on a Google Colab environment.
I followed this simple tutorial for the implementation of the generator: https://medium.com/datadriveninvestor/keras-training-on-large-datasets-3e9d9dbc09d4
Data is being inputted as a list.
Tensorflow version: 1.13 Keras version: 2.2.4
Error: /usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/engine/training.pyc in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch) 1424 use_multiprocessing=use_multiprocessing, 1425 shuffle=shuffle, -> 1426 initial_epoch=initial_epoch) 1427 1428 def evaluate_generator(self,
/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/engine/training_generator.pyc in model_iteration(model, data, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch, mode, batch_size, **kwargs) 113 batch_size=batch_size, 114 epochs=epochs - initial_epoch, –> 115 shuffle=shuffle) 116 117 do_validation = validation_data is not None
/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/engine/training_generator.pyc in convert_to_generator_like(data, batch_size, steps_per_epoch, epochs, shuffle)
375
376 # Create generator from NumPy or EagerTensor Input.
–> 377 num_samples = int(nest.flatten(data)[0].shape[0])
378 if batch_size is None:
379 raise ValueError(‘You must specify batch_size
’)
AttributeError: ‘MY_Generator’ object has no attribute ‘shape’
gist: https://gist.github.com/fjur/2815f235f84b8b666107207599482428
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 18
Commits related to this issue
- Fix the customized generator according to https://github.com/keras-team/keras/issues/12586 — committed to jeffrey82221/squeezenet_experiment by jeffrey82221 4 years ago
I ran into the same problem. Importing the Sequence class from Tensorflow fixed it for me:
from tensorflow.python.keras.utils.data_utils import Sequence
Thanks that solved it! I was using
from keras.utils import Sequence
which seemed to be causing issues.I face with the same issue. If your generator returns a list (something like: [X, Y]), try return a tuple (X, Y). It helps me.
I had the same problem. It worked fine in the jupyter notebook but if I put the exact cell code into a python file and imported the class it would throw the error.
The problem was, I imported keras in the python file with
import keras
Changing it to:
from tensorflor import keras
fixed the problem problem for me.
I guess the naming is ambiguous.
My script kept getting error. While importing the PreProcessing from keras: Use below:
from tensorflow.keras.preprocessing.sequence import TimeseriesGenerator
Instead of:from keras.preprocessing.sequence import TimeseriesGenerator