xarray: `rename_vars` followed by `swap_dims` and `merge` causes swapped dim to reappear
What happened?
I wanted to rename a dimension coordinate for two datasets before merging: ds = ds.rename_vars(y="z").swap_dims(y="z")
, and the same for the second data set. After merging the datasets, the merged result has the dimension “y” in addition to “z”.
Swapping the order of rename_vars
and swap_dims
before merging works in that “y” does not reappear, but then “z” is listed as a non-dimension coordinate.
Doing rename_vars
followed by swap_dims
/after/ merging gives the result I wanted, but if I merge again, the same issue occurs.
My current solution is to only rename dimension coordinates before saving to netCDF.
What did you expect to happen?
Merging two datasets with the same coordinates and dimensions (but different data variables) should result in a single dataset with all of the data variables from the two datasets and exactly the same coordinates and dimensions.
Minimal Complete Verifiable Example
import numpy as np
import xarray as xr
from xarray.core.utils import Frozen
A = np.arange(4).reshape((2, 2))
B = np.arange(4).reshape((2, 2)) + 4
ds1 = xr.Dataset({"A": (["x", "y"], A), "B": (["x", "y"], B)}, coords={"x": ("x", [1, 2]), "y": ("y", [1, 2])})
ds2 = xr.Dataset({"C": (["x", "y"], A), "D": (["x", "y"], B)}, coords={"x": ("x", [1, 2]), "y": ("y", [1, 2])})
assert ds1.dims == Frozen({"x": 2, "y": 2})
assert ds2.dims == Frozen({"x": 2, "y": 2})
ds1_swap = ds1.rename_vars(y="z").swap_dims(y="z")
ds2_swap = ds2.rename_vars(y="z").swap_dims(y="z")
assert ds1_swap.dims == Frozen({"x": 2, "z": 2})
assert ds2_swap.dims == Frozen({"x": 2, "z": 2})
# merging makes the dimension "y" reappear (I would expect this assertion to fail):
assert xr.merge([ds1_swap, ds2_swap]).dims == Frozen({"x": 2, "z": 2, "y": 2})
# renaming and swapping after the merge causes issues later:
ds12 = xr.merge([ds1, ds2]).rename_vars(y="z").swap_dims(y="z")
ds3 = xr.Dataset({"E": (["x", "z"], A), "F": (["x", "z"], B)}, coords={"x": ("x", [1, 2]), "z": ("z", [1, 2])})
# ds12 and ds3 have the same dimensions:
assert ds12.dims == Frozen({"x": 2, "z": 2})
assert ds3.dims == Frozen({"x": 2, "z": 2})
# but merging brings back "y"
ds123 = xr.merge([ds12, ds3])
assert ds123.dims == Frozen({"x": 2, "z": 2, "y": 2})
# as do other operations:
ds12_as = ds12.assign_coords(x=(ds12.x + 1))
assert ds12_as.sizes == Frozen({"x": 2, "z": 2, "y": 2})
MVCE confirmation
- Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
- Complete example — the example is self-contained, including all data and the text of any traceback.
- Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
- New issue — a search of GitHub Issues suggests this is not a duplicate.
- Recent environment — the issue occurs with the latest version of xarray and its dependencies.
Relevant log output
No response
Anything else we need to know?
No response
Environment
The MVCE works in all venvs I’ve tried including:
INSTALLED VERSIONS
commit: None python: 3.10.13 (main, Nov 10 2023, 15:02:19) [GCC 11.4.0] python-bits: 64 OS: Linux OS-release: 6.5.0-14-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: (‘en_GB’, ‘UTF-8’) libhdf5: 1.12.2 libnetcdf: 4.9.3-development
xarray: 2023.11.0 pandas: 1.5.3 numpy: 1.26.2 scipy: 1.11.4 netCDF4: 1.6.5 pydap: None h5netcdf: 1.3.0 h5py: 3.10.0 Nio: None zarr: None cftime: 1.6.3 nc_time_axis: 1.4.1 iris: None bottleneck: None dask: 2023.12.0 distributed: None matplotlib: 3.8.2 cartopy: 0.22.0 seaborn: 0.13.0 numbagg: None fsspec: 2023.12.1 cupy: None pint: None sparse: 0.15.1 flox: None numpy_groupies: None setuptools: 69.0.2 pip: 23.3.1 conda: None pytest: 7.4.3 mypy: None IPython: 8.18.1 sphinx: None /home/brendan/Documents/inversions/.pymc_venv/lib/python3.10/site-packages/_distutils_hack/init.py:33: UserWarning: Setuptools is replacing distutils. warnings.warn(“Setuptools is replacing distutils.”)
INSTALLED VERSIONS
commit: None python: 3.9.7 (default, Sep 16 2021, 13:09:58) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.81.1.el7.x86_64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: (‘en_GB’, ‘UTF-8’) libhdf5: None libnetcdf: None
xarray: 2024.1.0 pandas: 2.2.0 numpy: 1.26.3 scipy: None netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nc_time_axis: None iris: None bottleneck: None dask: None distributed: None matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None flox: None numpy_groupies: None setuptools: 69.0.3 pip: 23.3.2 conda: None pytest: None mypy: None IPython: 8.18.1 sphinx: None
About this issue
- Original URL
- State: open
- Created 5 months ago
- Comments: 15 (6 by maintainers)
if these two:
do the same thing, could we just make the former use the latter as implementation (obviously adapted to allow renaming multiple dimensions at the same time)? I don’t think we’d even need a deprecation cycle for that.