keras: Predict_generator problem - could not broadcast input array

Hi there! (I’m up-to-date with the master branch of Keras and using latest TensorFlow on Python 3.4). I’m trying to use predict_generator. Here is my generator (that converts data from scipy sparse to dense matrix):

def batch_predict_generator(X, batch_size):
    number_of_batches = X.shape[0]//batch_size
    while 1:
        for i in range(number_of_batches):
            yield X[i*batch_size:(i+1)*batch_size].todense()

I’m using fit_generator that is similar to this generator to fit my data into MLP network and everything is fine. Here is my network model:

    model = Sequential()
    model.add(Dense(128, batch_input_shape=(None, x_size,)))
    model.add(Activation('relu'))
    model.add(Dropout(0.2))
    model.add(Dense(64))
    model.add(Activation('relu'))
    model.add(Dropout(0.2))
    model.add(Dense(2)) # amount of classes
    model.add(Activation('softmax'))
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=["accuracy"])

batch_size is equal 256, x_size is equal 20573.

However I’m getting the following error when using predict_generator like this: pred_y = model.predict_generator(generator=batch_predict_generator(X_test, batch_size), val_samples=X_test.shape[0])

ValueError: could not broadcast input array from shape (256,2) into shape (238,2)

It weird, because fit_generator and evaluate_generator are working correctly. I’m using it like this

score = model.evaluate_generator(generator=batch_generator(X_test, Y_test, batch_size), val_samples=X_test.shape[0])

Could anybody help me? What wrong with predict_generator? During fitting my model I’m getting this warning:

UserWarning: Epoch comprised more than samples_per_epoch samples, which might affect learning results. Set samples_per_epoch correctly to avoid this warning. warnings.warn('Epoch comprised more than ’

About this issue

Original URL
State: closed
Created 8 years ago
Comments: 20 (1 by maintainers)

Most upvoted comments

I also have this problem with prediction but not with training:

/covnet_v65.py in make_submission()
    571         start = finish
    572         finish = finish+batch_size
--> 573         predicted = model.predict(sample, verbose=2)
    574         print("Predicting Test data with sample size: {}...".format(sample.shape))
    575         predicted = predicted.reshape((sample.shape[0],60))

~/py3/lib/python3.5/site-packages/keras/engine/training.py in predict(self, x, batch_size, verbose)
   1515         f = self.predict_function
   1516         return self._predict_loop(f, ins,
-> 1517                                   batch_size=batch_size, verbose=verbose)
   1518 
   1519     def train_on_batch(self, x, y,

~/py3/lib/python3.5/site-packages/keras/engine/training.py in _predict_loop(self, f, ins, batch_size, verbose)
   1148 
   1149             for i, batch_out in enumerate(batch_outs):
-> 1150                 outs[i][batch_start:batch_end] = batch_out
   1151             if verbose == 1:
   1152                 progbar.update(batch_end)

ValueError: could not broadcast input array from shape (22,1,60) into shape (23,1,60)

civilinformer on Aug 3, 2017

It’s weird. I’ve got the same problem with predict

Traceback (most recent call last): File "test.py", line 220, in <module> test.train(steps=1000, batch_size=1) File "test.py", line 170, in train outs = self.generator.predict(inputs) File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 902, in predict return self.model.predict(x, batch_size=batch_size, verbose=verbose) File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1585, in predict batch_size=batch_size, verbose=verbose) File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1221, in _predict_loop outs[i][batch_start:batch_end] = batch_out ValueError: could not broadcast input array from shape (24,512,512,1) into shape (1,512,512,1)

IWichakam on Jun 7, 2017