VirtualiZarr: `to_icechunk` doesn't have a `group` option
This signature
should include a group option to allow you to put the dataset somewhere other than at the root of the store hierarchy.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 2
- Comments: 21 (10 by maintainers)
Commits related to this issue
- Add unit tests for non-root groups Fixes #341 — committed to chuckwondo/VirtualiZarr by chuckwondo 2 years ago
- Add unit tests for non-root groups Fixes #341 — committed to chuckwondo/VirtualiZarr by chuckwondo 2 years ago
- Add `group` option to `dataset_to_icechunk` (#383) * Bump ruff pre-commit hook and fix formatting * Bump ruff target version to match min Python version * Add `group` param to `dataset_to_icech... — committed to chuckwondo/VirtualiZarr by chuckwondo 2 years ago
- Add group param to virtualize Dataset accessor Fixes #341 — committed to chuckwondo/VirtualiZarr by chuckwondo 2 years ago
Thank you. I agree. I’ll take that route unless someone strongly advises otherwise.
Sure!
I personally would also make
groupkeyword-only (if we were to rewriteDataset.to_zarrI’d also vote for making all the parameters other thanstorekeyword-only):is much harder to misunderstand than
(I’m intentionally choosing a bad variable name to make my point)
I’d like to take this one, if nobody else has claimed it.
Excellent. Thanks for starting a separate discussion. I’ll add my 2 cents over there.
In the meantime, I’ve plumbed things through as requested, in PR #391
I think changing it to
_dataset_to_icechunkwould be worthwhile.@chuckwondo would you be interested in opening another PR that adds the
groupoption to theto_icechunkmethod in theVirtualiZarrDatasetAccessor: https://github.com/zarr-developers/VirtualiZarr/blob/e3787c55cd242d83257d2bc0ab819aab56dda3cb/virtualizarr/accessor.py#L42-L82This is a convenience method that allows users to use code like:
rather than needing to call
dataset_to_icehunk().I think the best way to test this new code would be to modify the test you previously parameterized to use the
to_icechunk()function on the accessor rather than calling dataset_to_icechunk because this will only add code coverage: https://github.com/zarr-developers/VirtualiZarr/blob/e3787c55cd242d83257d2bc0ab819aab56dda3cb/virtualizarr/tests/test_writers/test_icechunk.py#L42-L80for reference, if
groupwould also be a positional-or-keyword (i.e. not positional-only) parameter with a default – likeappend_dim/last_updated_at– it would be possible to simply usegroupas a keyword argument:(so you don’t have to make it a keyword-only parameter).
That said, keyword-only parameters allow making API use more uniform, which especially for optional parameters really improves the readability of any function calls (in my opinion, at least).
Ideally do whatever xarray.to_zarr does if the group does not already exist.
Yep! Generally we want to make this as similar as possible to xarray.Dataset.to_zarr().
I agree about kw-only args. We’re not too worried about backwards-compatibility at this point as the library is still new and evolving.