xcdat: [Bug]:`.spatial.average(lat_bnds=(a,b)` does not appear to be working properly

What happened?

I am a new user to XCDAT so this could definitely be user error. I have a data set that I have am loading on perlmutter with this code

# Load SCREAMv1 regridded files with the Surface related output
directory_prefix='/global/cfs/cdirs/e3sm/terai/SCREAM/v1_production'
f_Jan_SurfVars=xcdat.open_mfdataset(directory_prefix+'/ne1024pg2_ne1024pg2_DY2/output.scream.SurfVars.INSTANT.nmins_x15.20*nc', add_bounds=True, parallel=True)

and then I attempt to take a spatial average with latitude bounds. As a sanity check I took the average for wildly different areas of the globe expecting them to be different, but they end up matching exactly.
I noticed this when plotting the average for more realistic cases and noticing the plots matched.

What did you expect to happen? Are there are possible answers you came across?

I expected the two different lat bounds to produce different answers.

Minimal Complete Verifiable Example (MVCE)

# Load SCREAMv1 regridded files with the Surface related output
directory_prefix='/global/cfs/cdirs/e3sm/terai/SCREAM/v1_production'
f_Jan_SurfVars=xcdat.open_mfdataset(directory_prefix+'/ne1024pg2_ne1024pg2_DY2/output.scream.SurfVars.INSTANT.nmins_x15.20*nc', add_bounds=True, parallel=True)

varname = "T_2m"
filespec = f_Jan_SurfVars
a = np.array(filespec.spatial.average(varname, lat_bounds=(0,10),axis=["X","Y"], weights=v1_weights)[varname].sel(time=slice('2020-01-20 00:00:00','2020-02-28 23:59:59')))
b = np.array(filespec.spatial.average(varname, lat_bounds=(80,90),axis=["X","Y"], weights=v1_weights)[varname].sel(time=slice('2020-01-20 00:00:00','2020-02-28 23:59:59')))
errmax = np.amax(np.abs(a-b))
print("%e, %e, %e" %(a[0],b[0], errmax))

produces:

2.852244e+02, 2.852244e+02, 0.000000e+00

Relevant log output

No response

Anything else we need to know?

No response

Environment

TBH I don’t know how to do this. xcdat.show_versions() throws an error. I am using the e3sm_unified environment on NERSC - perlmutter which I loaded with

source /global/common/software/e3sm/anaconda_envs/load_latest_e3sm_unified_pm-cpu.sh

xarray.show_versions produces:

INSTALLED VERSIONS
------------------
commit: None
python: 3.10.8 | packaged by conda-forge | (main, Nov 22 2022, 08:23:14) [GCC 10.4.0]
python-bits: 64
OS: Linux
OS-release: 5.14.21-150400.24.46_12.0.73-cray_shasta_c
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.12.2
libnetcdf: 4.8.1

xarray: 2022.10.0
pandas: 1.5.2
numpy: 1.23.5
scipy: 1.9.3
netCDF4: 1.6.1
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: None
cftime: 1.6.2
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: 1.3.5
dask: 2022.10.0
distributed: 2022.10.0
matplotlib: 3.6.2
cartopy: 0.21.0
seaborn: 0.12.1
numbagg: None
fsspec: 2022.11.0
cupy: None
pint: 0.20.1
sparse: 0.13.0
flox: None
numpy_groupies: None
setuptools: 65.5.1
pip: 22.3.1
conda: None
pytest: 7.2.0
IPython: 8.7.0
sphinx: None

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 27

Commits related to this issue

Most upvoted comments

I will try to fix this in NCO and release next week.

On Tue, Jul 11, 2023, 22:33 Stephen Po-Chedley @.***> wrote:

@crterai https://urldefense.com/v3/__https://github.com/crterai__;!!CzAuKJ42GuquVTTmVmPViYEvSg!JVl26DVEN79JcRVPe7_AcszdmCnvbUdFCEiwHe6giRQS6afzUg0iGwOq5e-2P4SXjZx8W4l-HX63gHFKqvxgk4k$ / @AaronDonahue https://urldefense.com/v3/__https://github.com/AaronDonahue__;!!CzAuKJ42GuquVTTmVmPViYEvSg!JVl26DVEN79JcRVPe7_AcszdmCnvbUdFCEiwHe6giRQS6afzUg0iGwOq5e-2P4SXjZx8W4l-HX63gHFKEcK0xWk$ – we produced a bug fix for this issue that will be incorporated into the next version of xcdat (not yet released). Could you check to see if this issue is resolved in this candidate/pre-release (see this discussion https://urldefense.com/v3/__https://github.com/xCDAT/xcdat/discussions/517__;!!CzAuKJ42GuquVTTmVmPViYEvSg!JVl26DVEN79JcRVPe7_AcszdmCnvbUdFCEiwHe6giRQS6afzUg0iGwOq5e-2P4SXjZx8W4l-HX63gHFKelz45aE$ ).

— Reply to this email directly, view it on GitHub https://urldefense.com/v3/__https://github.com/xCDAT/xcdat/issues/494*issuecomment-1632081104__;Iw!!CzAuKJ42GuquVTTmVmPViYEvSg!JVl26DVEN79JcRVPe7_AcszdmCnvbUdFCEiwHe6giRQS6afzUg0iGwOq5e-2P4SXjZx8W4l-HX63gHFKbWMZrrA$, or unsubscribe https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ABKWO6LJVES76IIWTG6YDMTXPZOM5ANCNFSM6AAAAAAYXTTRZI__;!!CzAuKJ42GuquVTTmVmPViYEvSg!JVl26DVEN79JcRVPe7_AcszdmCnvbUdFCEiwHe6giRQS6afzUg0iGwOq5e-2P4SXjZx8W4l-HX63gHFKf2HYNqY$ . You are receiving this because you were mentioned.Message ID: @.***>

CF requires coordinates to be monotonically increasing or decreasing, and the bounds on a 1-D coordinate should be consistent with this (so if coordinate increases, the 2nd bound should be greater than first bound. This makes it simple to calculate the cell width. So in you example, [359,1] is incorrect and [-1,1] is correct. (I don’t think CF generally is aware of cyclic coordinates, except maybe time conversions involving hours in a day and days in a year.)

In the special case of a single cell locate at 0 degE spanning [-1,1], it would be legal to store bounds of either [-1,1] or [1,-1], but I think [359,1] would be interpreted as having a width of 358 degrees longitude.

@pochedls - I can give it a try.

I was not sure about the convention but @pochedls found the session in the document. @crterai shared the files on NERSC: /pscratch/sd/t/terai/EAMxx/shareWpochedls/output.scream.SurfVars.INSTANT.nmins_x15.2020-*nc. Thank you for helping on this.