napari: Error while browsing pyramid dataset
🐛 Bug
I was panning and zooming around the IDR COVID gut dataset, minding my own business, when BAM! This happened:
File "/home/jni/miniconda3/envs/f7/lib/python3.7/site-packages/vispy/app/backends/_qt.py", line 825, in paintGL
self._vispy_canvas.events.draw(region=None)
File "/home/jni/miniconda3/envs/f7/lib/python3.7/site-packages/vispy/util/event.py", line 455, in __call__
self._invoke_callback(cb, event)
File "/home/jni/miniconda3/envs/f7/lib/python3.7/site-packages/vispy/util/event.py", line 475, in _invoke_callback
self, cb_event=(cb, event))
File "/home/jni/miniconda3/envs/f7/lib/python3.7/site-packages/vispy/util/event.py", line 471, in _invoke_callback
cb(event)
File "/home/jni/projects/napari/napari/_vispy/vispy_base_layer.py", line 246, in on_draw
shape_threshold=self.node.canvas.size,
File "/home/jni/projects/napari/napari/layers/base/base.py", line 692, in _update_multiscale
self.data_level = data_level
File "/home/jni/projects/napari/napari/layers/image/image.py", line 291, in data_level
self.refresh()
File "/home/jni/projects/napari/napari/layers/base/base.py", line 633, in refresh
self.events.set_data()
File "/home/jni/projects/napari/napari/utils/event.py", line 508, in __call__
self._invoke_callback(cb, event)
File "/home/jni/projects/napari/napari/utils/event.py", line 529, in _invoke_callback
cb_event=(cb, event),
File "/home/jni/projects/napari/napari/utils/event.py", line 523, in _invoke_callback
cb(event)
File "/home/jni/projects/napari/napari/_vispy/vispy_image_layer.py", line 73, in _on_data_change
data = self.downsample_texture(data, self.MAX_TEXTURE_SIZE_2D)
File "/home/jni/projects/napari/napari/_vispy/vispy_image_layer.py", line 167, in downsample_texture
f"Shape of individual tiles in multiscale {data.shape} "
ValueError: Shape of individual tiles in multiscale (1, 19840) cannot exceed GL_MAX_TEXTURE_SIZE 16384. Rendering is currently in 2D mode.
To Reproduce
Steps to reproduce the behavior:
- Download the dataset using the code below
- Open it in napari using the next chunk of code
- pan and zoom for a while
Honestly, I’m not clear on step 3, because everything worked great for a few minutes. I don’t know under what circumstances this error gets actually triggered.
Code to download data (courtesy of @joshmoore):
import requests
import dask.array as da
from dask.diagnostics import ProgressBar
path = 'https://s3.embassy.ebi.ac.uk/idr/zarr/v0.1/9822151.zarr'
try:
datasets = [x['path'] for x in
requests.get(f'{path}/.zattrs').json()["multiscales"][0]["datasets"]]
except KeyError:
datasets = ["0"]
resolutions = [da.from_zarr(path, component=str(i)) for i in datasets]
levels = list(range(len(resolutions)))
pbar = ProgressBar()
for level, data in reversed(list(zip(levels, resolutions))):
with pbar:
data.to_zarr(f'/path/to/data/covid-gut/{level}.zarr')
It’s ~28GB so be prepared to wait. Took about 10h for me, most of which comes from the two highest levels.
Code to view in napari:
import napari
localres = [
da.from_zarr(f'/path/to/data/covid-gut/{level}.zarr')
for level in range(len(resolutions))
]
with napari.gui_qt():
v = napari.view_image(localres)
Expected behavior
Should just keep working like it’s nothing.
Environment
napari: 0.2.12+98.g0f94ec10
Platform: Linux-4.15.0-29-generic-x86_64-with-debian-buster-sid
Python: 3.7.3 | packaged by conda-forge | (default, Jul 1 2019, 21:52:21) [GCC 7.3.0]
Qt: 5.13.0
PyQt5: 5.13.1
NumPy: 1.18.3
SciPy: 1.3.1
Dask: 2.15.0
VisPy: 0.6.4
GL version: 3.0 Mesa 19.2.8
MAX_TEXTURE_SIZE: 16384
Plugins:
- napari-plugin-engine: 0.1.4
- napari-plugins: 0.1.0
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 1
- Comments: 15 (13 by maintainers)
@psobolewskiPhD
zarr is a first-class citizen, ome-zarr is (so far) not. I wonder if ome-zarr should adopt a different file extension… 🤷
In the world where “all” includes a well-curated list of plugins, yes, we can add napari-ome-zarr. I don’t think we are yet in that world, but we could be soon. Something to discuss in community meetings etc.
In the meantime, since this issue is about a crash that no longer occurs, I suggest we close it!
Thanks Josh! In a fresh env I get:
So I guess this is OME zarr and napari builtins can’t handle it? Is there a way to have known this a priori?
because it was associated with that file extension
but erroring is not very graceful, for sure.I installed
napari-ome-zarr
and it worked, maybe we should make that part of[all]
? IMO zarr should be first class citizen?Anyhow, I’ve spent about 30 min zooming and panning this dataset in both normal and async on main (3e3f2ce) and have not been able to reproduce a crash.
Thanks @joshmoore! Yeah I downloaded the datasets willy nilly which is why things were only half working. But my incompetence helped us uncover some bugs in how we handle plugins! =D
See https://github.com/ome/ome-zarr-py/pull/8