dask: Concatenate Array[sparse] raises ValueError

This happens during metadata validation.

In [56]: import scipy.sparse

In [57]: import dask.array as da

In [58]: a = scipy.sparse.random(4, 3, format='csr')

In [59]: meta = scipy.sparse.eye(0).asformat('csr')

In [60]: b = da.from_array(a, meta=meta)

In [61]: b
Out[61]: dask.array<array, shape=(4, 3), dtype=float64, chunksize=(4, 3)>

In [62]: da.concatenate([b], axis=1)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-62-4bf3ad9a8a86> in <module>
----> 1 da.concatenate([b], axis=1).compute()

~/Envs/dask-dev/lib/python3.7/site-packages/dask/array/core.py in concatenate(seq, axis, allow_unknown_chunksizes)
   3232         raise ValueError("Need array(s) to concatenate")
   3233
-> 3234     meta = np.concatenate([meta_from_array(s) for s in seq], axis=axis)
   3235
   3236     # Promote types to match meta

ValueError: zero-dimensional arrays cannot be concatenated

Is this something we want to support? Came up in https://github.com/dask/dask-ml/pull/529

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 1
  • Comments: 16 (16 by maintainers)

Most upvoted comments

FWIW I raised issue ( https://github.com/scipy/scipy/issues/10362 ) to discuss extending NEP 18 to SciPy sparse matrices.