hvplot: hvplot is not using lazy loading for zarr-backed xarray dataset
ALL software version info
xr.__version__
'0.21.1'
hvplot.__version__
'0.7.3'
holoviews.__version__
'1.14.7'
Description of expected behavior and the observed behavior
When I visualize zarr-backed xarray datasets with hvplot, I expect the loading to be lazy. I don’t want to load the entire array into memory. I expect this to happen even without dask being involved.
Instead, I am noticing that hvplot is loading data eagerly.
Complete, minimal, self-contained example code that reproduces the issue
import xarray as xr
import hvplot.xarray
url = "https://ncsa.osn.xsede.org/Pangeo/pangeo-forge/swot_adac/FESOM/surf/fma.zarr"
ds = xr.open_dataset(url, engine='zarr') # note that ds is not chunked but still uses lazy loading
# this will trigger loading the whole array into memory
ds.sst.hvplot(x='lon', y='lat', dynamic=True, rasterize=True)
Note that lazyness can be achieved by chunking the xarray dataset. But I don’t want to have to do that. Dask introduces additional latency that I would like to avoid here.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 19 (12 by maintainers)
I’m on xarray 2022.3.0 and this is sufficient to reproduce:
There is a time dimension.
will lazily grab a single time step, which is all that should be required for a single image plot.