zarr-python: Passing dask array to zarr leads to a TypeError

Minimal, reproducible code sample, a copy-pastable example if possible

import numpy as np
import zarr
import os
import dask.array as da

from ome_zarr.io import parse_url
from ome_zarr.writer import write_image

path = "test_dask_image.zarr"
os.mkdir(path)

mean_val = 10
rng = np.random.default_rng(0)
data = rng.poisson(mean_val, size=(50, 128, 128)).astype(np.uint8)
data = da.from_array(data)  #  <--- !!!THIS IS WHAT BREAKS THE EXAMPLE!!!

# write the image data
store = parse_url(path, mode="w").store
root = zarr.group(store=store)
write_image(image=data, group=root, chunks=(1, 128, 128), axes="zyx")
root.attrs["omero"] = {
    "channels": [{
        "color": "00FFFF",
        "window": {"start": 0, "end": 20},
        "label": "random",
        "active": True,
    }]
}

Problem description

Get an error when trying to write a dask array to a zarr file. I took the proposed example from https://github.com/ome/ome-zarr-py/pull/121 and wrapped the numpy array in a dask array. It appears that the dask array function astype doesn’t support the parameter “order”. The example code works when using a numpy array for me. Here’s the resulting traceback:

Traceback (most recent call last):
  File "/home/sutherland/git/xray_napari/write_zarr_with_dask_failure.py", line 20, in <module>
    write_image(image=data, group=root, chunks=(1, 128, 128), axes="zyx")
  File "/home/sutherland/anaconda3/envs/napari2/lib/python3.9/site-packages/ome_zarr/writer.py", line 178, in write_image
    write_multiscale(image, group, chunks=chunks, fmt=fmt, axes=axes)
  File "/home/sutherland/anaconda3/envs/napari2/lib/python3.9/site-packages/ome_zarr/writer.py", line 105, in write_multiscale
    group.create_dataset(str(path), data=dataset, chunks=chunks)
  File "/home/sutherland/anaconda3/envs/napari2/lib/python3.9/site-packages/zarr/hierarchy.py", line 808, in create_dataset
    return self._write_op(self._create_dataset_nosync, name, **kwargs)
  File "/home/sutherland/anaconda3/envs/napari2/lib/python3.9/site-packages/zarr/hierarchy.py", line 661, in _write_op
    return f(*args, **kwargs)
  File "/home/sutherland/anaconda3/envs/napari2/lib/python3.9/site-packages/zarr/hierarchy.py", line 824, in _create_dataset_nosync
    a = array(data, store=self._store, path=path, chunk_store=self._chunk_store,
  File "/home/sutherland/anaconda3/envs/napari2/lib/python3.9/site-packages/zarr/creation.py", line 377, in array
    z[...] = data
  File "/home/sutherland/anaconda3/envs/napari2/lib/python3.9/site-packages/zarr/core.py", line 1224, in __setitem__
    self.set_basic_selection(selection, value, fields=fields)
  File "/home/sutherland/anaconda3/envs/napari2/lib/python3.9/site-packages/zarr/core.py", line 1319, in set_basic_selection
    return self._set_basic_selection_nd(selection, value, fields=fields)
  File "/home/sutherland/anaconda3/envs/napari2/lib/python3.9/site-packages/zarr/core.py", line 1610, in _set_basic_selection_nd
    self._set_selection(indexer, value, fields=fields)
  File "/home/sutherland/anaconda3/envs/napari2/lib/python3.9/site-packages/zarr/core.py", line 1682, in _set_selection
    self._chunk_setitems(lchunk_coords, lchunk_selection, chunk_values,
  File "/home/sutherland/anaconda3/envs/napari2/lib/python3.9/site-packages/zarr/core.py", line 1871, in _chunk_setitems
    cdatas = [self._process_for_setitem(key, sel, val, fields=fields)
  File "/home/sutherland/anaconda3/envs/napari2/lib/python3.9/site-packages/zarr/core.py", line 1871, in <listcomp>
    cdatas = [self._process_for_setitem(key, sel, val, fields=fields)
  File "/home/sutherland/anaconda3/envs/napari2/lib/python3.9/site-packages/zarr/core.py", line 1924, in _process_for_setitem
    chunk = value.astype(self._dtype, order=self._order, copy=False)
  File "/home/sutherland/anaconda3/envs/napari2/lib/python3.9/site-packages/dask/array/core.py", line 2086, in astype
    raise TypeError(
TypeError: astype does not take the following keyword arguments: ['order']

Version and installation information

Please provide the following:

  • Value of zarr.__version__: 2.10.3
  • Value of numcodecs.__version__: 0.9.1
  • Version of Python interpreter: 3.9.7
  • Operating system (Linux/Windows/Mac): RHEL 8
  • How Zarr was installed (e.g., “using pip into virtual environment”, or “using conda”): using pip into conda env

Also, if you think it might be relevant, please provide the output from pip freeze or conda env export depending on which was used to install Zarr.

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 24 (12 by maintainers)

Commits related to this issue

Most upvoted comments

I’m +1 on redirecting everyone back to use the dask.array.to_zarr method instead of making additional changes here.

Thanks for the info, @michael-sutherland. Looking at the stacktrace again, any objections to migrating this to github.com/ome/ome-zarr-py ?

cc: @sbesson @will-moore