dask: An 'array-like' should not need an 'ndim' property
When upgrading Iris to work with Dask 2.1.0, we observe that an “array-like”, as described in the docs to dask.array.from_array
, now seems to also need an ndim
property.
Previously (version 1, at least), I believe this only required a shape
and __getitem__
, as described in docs.
As it’s trivial to calculate from shape, this seems like a mistake. Should this be rectified ?
Link showing changes we needed : https://github.com/SciTools/iris/pull/3355/files#r302502604
Example to Reproduce
code
import sys
import numpy as np
import dask
import dask.array as da
class Wrapped(object):
def __init__(self, shape, dtype=np.float32):
self.shape = shape
self.dtype = dtype
def __getitem__(self, keys):
return np.zeros(self.shape, dtype=self.dtype).__getitem__(keys)
print('versions:')
print(' python version = {}'.format(sys.version_info))
print(' numpy version = {}'.format(np.__version__))
print(' dask version = {}'.format(dask.__version__))
wrapped = Wrapped((3,4))
lazy = da.from_array(wrapped, chunks=-1)
result = lazy[1:,1:].compute()
print('result = {}'.format(result))
results …
versions:
python version = sys.version_info(major=3, minor=7, micro=3, releaselevel='final', serial=0)
numpy version = 1.16.4
dask version = 1.2.2
result = [[0. 0. 0.]
[0. 0. 0.]]
versions:
python version = sys.version_info(major=3, minor=7, micro=3, releaselevel='final', serial=0)
numpy version = 1.16.4
dask version = 2.1.0
Traceback (most recent call last):
File "dask_wrap_example.py", line 18, in <module>
lazy = da.from_array(wrapped, chunks=-1)
File "/tmp/persistent/miniconda3/envs/dask2/lib/python3.7/site-packages/dask/array/core.py", line 2683, in from_array
return Array(dsk, name, chunks, meta=meta, dtype=getattr(x, "dtype", None))
File "/tmp/persistent/miniconda3/envs/dask2/lib/python3.7/site-packages/dask/array/core.py", line 1000, in __new__
meta = meta_from_array(meta, dtype=dtype)
File "/tmp/persistent/miniconda3/envs/dask2/lib/python3.7/site-packages/dask/array/utils.py", line 86, in meta_from_array
ndim = x.ndim
AttributeError: 'Wrapped' object has no attribute 'ndim'
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 16 (11 by maintainers)
Great
OK, in that case in the interests of everyone getting back to work I’m inclined to close this issue. Would that be ok with everyone?