keras: ValueError: setting an array element with a sequence while using sparse matrix

Hello!

I found that using sparse matrix would cause ValueError: setting an array element with a sequence

Version:

Keras (1.2.0) tensorflow (0.12.1)

Input data

X <9516x28934 sparse matrix of type '<type 'numpy.float64'>'
	with 946932 stored elements in Compressed Sparse Row format>

y numpy.ndarray  (9516,)

Code

from keras.models import Sequential
model = Sequential()

from keras.layers import Dense, Activation

model.add(Dense(output_dim=64, input_dim=X.shape[1]))
model.add(Activation("relu"))
model.add(Dense(output_dim=2))
model.add(Activation("softmax"))

from keras.utils.np_utils import to_categorical
# would convert to one-hot
y_binary = to_categorical(y)

model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])
model.fit(X, y_binary, nb_epoch=20, batch_size=32)

Log

Epoch 1/20


ValueErrorTraceback (most recent call last)
<ipython-input-37-894211790f97> in <module>()
     14 
     15 model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])
---> 16 model.fit(X, y_binary, nb_epoch=20, batch_size=32)
     17 
     18 # https://github.com/fchollet/keras/issues/4865

/usr/local/lib/python2.7/dist-packages/keras/models.pyc in fit(self, x, y, batch_size, nb_epoch, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, **kwargs)
    662                               shuffle=shuffle,
    663                               class_weight=class_weight,
--> 664                               sample_weight=sample_weight)
    665 
    666     def evaluate(self, x, y, batch_size=32, verbose=1,

/usr/local/lib/python2.7/dist-packages/keras/engine/training.pyc in fit(self, x, y, batch_size, nb_epoch, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch)
   1141                               val_f=val_f, val_ins=val_ins, shuffle=shuffle,
   1142                               callback_metrics=callback_metrics,
-> 1143                               initial_epoch=initial_epoch)
   1144 
   1145     def evaluate(self, x, y, batch_size=32, verbose=1, sample_weight=None):

/usr/local/lib/python2.7/dist-packages/keras/engine/training.pyc in _fit_loop(self, f, ins, out_labels, batch_size, nb_epoch, verbose, callbacks, val_f, val_ins, shuffle, callback_metrics, initial_epoch)
    841                 batch_logs['size'] = len(batch_ids)
    842                 callbacks.on_batch_begin(batch_index, batch_logs)
--> 843                 outs = f(ins_batch)
    844                 if not isinstance(outs, list):
    845                     outs = [outs]

/usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.pyc in __call__(self, inputs)
   1601         session = get_session()
   1602         updated = session.run(self.outputs + [self.updates_op],
-> 1603                               feed_dict=feed_dict)
   1604         return updated[:len(self.outputs)]
   1605 

/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.pyc in run(self, fetches, feed_dict, options, run_metadata)
    764     try:
    765       result = self._run(None, fetches, feed_dict, options_ptr,
--> 766                          run_metadata_ptr)
    767       if run_metadata:
    768         proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.pyc in _run(self, handle, fetches, feed_dict, options, run_metadata)
    935                 ' to a larger type (e.g. int64).')
    936 
--> 937           np_val = np.asarray(subfeed_val, dtype=subfeed_dtype)
    938 
    939           if not subfeed_t.get_shape().is_compatible_with(np_val.shape):

/usr/local/lib/python2.7/dist-packages/numpy/core/numeric.pyc in asarray(a, dtype, order)
    480 
    481     """
--> 482     return array(a, dtype, copy=False, order=order)
    483 
    484 def asanyarray(a, dtype=None, order=None):

ValueError: setting an array element with a sequence.

Though convert sparse matrix to np array can walk through, I think it would be better to support it.

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 17 (3 by maintainers)

Most upvoted comments

@hitchhiker744 X is sparse, not y. Convert X to normal matrix would take much larger memory, so I am suggesting add sparse support.

+12

eromoe on Mar 6, 2017

You should try using the functional API and creating an input layer with sparse=True. That solved an issue for me

maximsch2 on Jun 8, 2017

Is there a tutorial about Keras that … just works?

ebuildy on Jun 5, 2018

Try doing the one-hot encoding of y with scikit learn OneHotEncoder:

from sklearn.preprocessing import OneHotEncoder

enc = OneHotEncoder(sparse=False) # Key here is sparse=False!
y_categorical = enc.fit_transform(y.reshape((y.shape[0]),1))

model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])
model.fit(X, y_categorical, nb_epoch=20, batch_size=32)

hitchhiker744 on Mar 6, 2017

Try using InputLayer(sparse=True) in Sequential model instead of Input(sparse=True) in Functional models.

roywei on Oct 16, 2018

Try With batches, it works fine:

def samples(x_source, y_source, size):
    while True:
        for i in range(0, x_source.shape[0], size):
            j = i + size
            
            if j > x_source.shape[0]:
                j = x_source.shape[0]
                
            yield x_source[i:j].toarray(), y_source[i:j].toarray()
model.fit_generator(samples(x_train, x_train, nb_batch),x_train.shape[0], nb_epoch, verbose=1)

Saves a great a lot on memory nb_batch is the number of batches.

lanking520 on May 30, 2017

I have the same issue with tf 1.0.0 and keras 1.2.2 and following the (I presume otherwise working) example from: http://machinelearningmastery.com/tutorial-first-neural-network-python-keras/

jakubLangr on Mar 4, 2017