dask: An 'array-like' should not need an 'ndim' property

When upgrading Iris to work with Dask 2.1.0, we observe that an “array-like”, as described in the docs to dask.array.from_array, now seems to also need an ndim property. Previously (version 1, at least), I believe this only required a shape and __getitem__, as described in docs.

As it’s trivial to calculate from shape, this seems like a mistake. Should this be rectified ?

Link showing changes we needed : https://github.com/SciTools/iris/pull/3355/files#r302502604

Example to Reproduce

code

import sys
import numpy as np
import dask
import dask.array as da

class Wrapped(object):
    def __init__(self, shape, dtype=np.float32):
        self.shape = shape
        self.dtype = dtype
    def __getitem__(self, keys):
        return np.zeros(self.shape, dtype=self.dtype).__getitem__(keys)

print('versions:')
print('  python version = {}'.format(sys.version_info))
print('  numpy version = {}'.format(np.__version__))
print('  dask version = {}'.format(dask.__version__))
wrapped = Wrapped((3,4))
lazy = da.from_array(wrapped, chunks=-1)
result = lazy[1:,1:].compute()
print('result = {}'.format(result))

results …

versions:
  python version = sys.version_info(major=3, minor=7, micro=3, releaselevel='final', serial=0)
  numpy version = 1.16.4
  dask version = 1.2.2
result = [[0. 0. 0.]
 [0. 0. 0.]]
versions:
  python version = sys.version_info(major=3, minor=7, micro=3, releaselevel='final', serial=0)
  numpy version = 1.16.4
  dask version = 2.1.0
Traceback (most recent call last):
  File "dask_wrap_example.py", line 18, in <module>
    lazy = da.from_array(wrapped, chunks=-1)
  File "/tmp/persistent/miniconda3/envs/dask2/lib/python3.7/site-packages/dask/array/core.py", line 2683, in from_array
    return Array(dsk, name, chunks, meta=meta, dtype=getattr(x, "dtype", None))
  File "/tmp/persistent/miniconda3/envs/dask2/lib/python3.7/site-packages/dask/array/core.py", line 1000, in __new__
    meta = meta_from_array(meta, dtype=dtype)
  File "/tmp/persistent/miniconda3/envs/dask2/lib/python3.7/site-packages/dask/array/utils.py", line 86, in meta_from_array
    ndim = x.ndim
AttributeError: 'Wrapped' object has no attribute 'ndim'

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 16 (11 by maintainers)

Most upvoted comments

Not really, as we’ve already accommodated the change.

Great

It is more “philosophy” 😉

OK, in that case in the interests of everyone getting back to work I’m inclined to close this issue. Would that be ok with everyone?