zarr-python: Performance regression in V3

Zarr version

3.0.0

Numcodecs version

0.14.1

Python Version

3.13

Operating System

Linux

Installation

Using uv

Description

This simple workload, which writes out the numbers 1 through 1e9 in 64 separate chunks,

# Using zarr==2.18.2
import numpy as np
import zarr
from zarr._storage.v3 import DirectoryStoreV3

store = DirectoryStoreV3("/tmp/foo.zarr")
arr = zarr.array(np.arange(1024 * 1024 * 1024, dtype=np.float64), chunks=(1024 * 1024 * 16,))
zarr.save_array(store, arr, zarr_version=3, path="/")

run in about 5s on my machine on version 2.18.

The equivalent workload on version 3 takes over a minute:

# Using zarr==3.0
import numpy as np
import zarr
import zarr.codecs
from zarr.storage import LocalStore

store = LocalStore("/tmp/bar.zarr")

compressors = zarr.codecs.BloscCodec(cname='lz4', shuffle=zarr.codecs.BloscShuffle.bitshuffle)

za = zarr.create_array(
    store,
    shape=(1024 * 1024 * 1024,),
    chunks=(1024 * 1024 * 16,),
    dtype=np.float64,
    compressors=compressors,
)

arr = np.arange(1024 * 1024 * 1024, dtype=np.float64)
za[:] = arr

Steps to reproduce

See above

Additional output

No response

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Reactions: 1
  • Comments: 20 (9 by maintainers)

Most upvoted comments

@y4n9squared - thanks very much for your work on this. We are definitely interested in fixing this performance regression and appreciate your input.

@d-v-b is the one who wrote that code, so let’s get his input on it. From my perspective, I am very open to changing the way this is implemented back the the way it used to be.

that sounds like a bug, but maybe it should be spun out into its own issue with a reproducer?

@d-v-b is the one who wrote that code, so let’s get his input on it. From my perspective, I am very open to changing the way this is implemented back the the way it used to be.

Contrary to what git blame suggests, I didn’t write that implementation of all_equal – it appeared in https://github.com/zarr-developers/zarr-python/pull/1826. In any case, we should definitely go with an implementation that has the best all-around performance.

@y4n9squared thanks for the report, we are definitely happy to fix performance regressions here.

one question about the examples you shared upthread: In zarr-python 2.18, the default value for write_empty_chunks is True, but in 3.x it’s False.

Because of this change in default behavior, benchmarking across zarr-python versions without explicitly setting write_empty_chunks might give you misleading results.

For the broader question of benchmarking in the context of throttled or low memory bandwidth, can you recommend any tools for artificially reducing memory bandwidth? I’d like to be able to benchmark these things locally.

Hi @d-v-b , just so we have something concrete to discuss, I opened a draft PR with the changes that I made which showed improvement.