napari: Error while browsing pyramid dataset

🐛 Bug

I was panning and zooming around the IDR COVID gut dataset, minding my own business, when BAM! This happened:

  File "/home/jni/miniconda3/envs/f7/lib/python3.7/site-packages/vispy/app/backends/_qt.py", line 825, in paintGL
    self._vispy_canvas.events.draw(region=None)
  File "/home/jni/miniconda3/envs/f7/lib/python3.7/site-packages/vispy/util/event.py", line 455, in __call__
    self._invoke_callback(cb, event)
  File "/home/jni/miniconda3/envs/f7/lib/python3.7/site-packages/vispy/util/event.py", line 475, in _invoke_callback
    self, cb_event=(cb, event))
  File "/home/jni/miniconda3/envs/f7/lib/python3.7/site-packages/vispy/util/event.py", line 471, in _invoke_callback
    cb(event)
  File "/home/jni/projects/napari/napari/_vispy/vispy_base_layer.py", line 246, in on_draw
    shape_threshold=self.node.canvas.size,
  File "/home/jni/projects/napari/napari/layers/base/base.py", line 692, in _update_multiscale
    self.data_level = data_level
  File "/home/jni/projects/napari/napari/layers/image/image.py", line 291, in data_level
    self.refresh()
  File "/home/jni/projects/napari/napari/layers/base/base.py", line 633, in refresh
    self.events.set_data()
  File "/home/jni/projects/napari/napari/utils/event.py", line 508, in __call__
    self._invoke_callback(cb, event)
  File "/home/jni/projects/napari/napari/utils/event.py", line 529, in _invoke_callback
    cb_event=(cb, event),
  File "/home/jni/projects/napari/napari/utils/event.py", line 523, in _invoke_callback
    cb(event)
  File "/home/jni/projects/napari/napari/_vispy/vispy_image_layer.py", line 73, in _on_data_change
    data = self.downsample_texture(data, self.MAX_TEXTURE_SIZE_2D)
  File "/home/jni/projects/napari/napari/_vispy/vispy_image_layer.py", line 167, in downsample_texture
    f"Shape of individual tiles in multiscale {data.shape} "
ValueError: Shape of individual tiles in multiscale (1, 19840) cannot exceed GL_MAX_TEXTURE_SIZE 16384. Rendering is currently in 2D mode.

To Reproduce

Steps to reproduce the behavior:

  1. Download the dataset using the code below
  2. Open it in napari using the next chunk of code
  3. pan and zoom for a while

Honestly, I’m not clear on step 3, because everything worked great for a few minutes. I don’t know under what circumstances this error gets actually triggered.

Code to download data (courtesy of @joshmoore):

import requests
import dask.array as da
from dask.diagnostics import ProgressBar

path = 'https://s3.embassy.ebi.ac.uk/idr/zarr/v0.1/9822151.zarr'
try:
    datasets = [x['path'] for x in
                requests.get(f'{path}/.zattrs').json()["multiscales"][0]["datasets"]]
except KeyError:
    datasets = ["0"]
resolutions = [da.from_zarr(path, component=str(i)) for i in datasets]

levels = list(range(len(resolutions)))

pbar = ProgressBar()
for level, data in reversed(list(zip(levels, resolutions))):
    with pbar:
        data.to_zarr(f'/path/to/data/covid-gut/{level}.zarr')

It’s ~28GB so be prepared to wait. Took about 10h for me, most of which comes from the two highest levels.

Code to view in napari:

import napari

localres = [
    da.from_zarr(f'/path/to/data/covid-gut/{level}.zarr')
    for level in range(len(resolutions))
]

with napari.gui_qt():
    v = napari.view_image(localres)

Expected behavior

Should just keep working like it’s nothing.

Environment

napari: 0.2.12+98.g0f94ec10
Platform: Linux-4.15.0-29-generic-x86_64-with-debian-buster-sid
Python: 3.7.3 | packaged by conda-forge | (default, Jul  1 2019, 21:52:21)  [GCC 7.3.0]
Qt: 5.13.0
PyQt5: 5.13.1
NumPy: 1.18.3
SciPy: 1.3.1
Dask: 2.15.0
VisPy: 0.6.4

GL version:  3.0 Mesa 19.2.8
MAX_TEXTURE_SIZE: 16384

Plugins:
  - napari-plugin-engine: 0.1.4
  - napari-plugins: 0.1.0

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 1
  • Comments: 15 (13 by maintainers)

Most upvoted comments

@psobolewskiPhD

maybe we should make that part of [all]? IMO zarr should be first class citizen?

zarr is a first-class citizen, ome-zarr is (so far) not. I wonder if ome-zarr should adopt a different file extension… 🤷

In the world where “all” includes a well-curated list of plugins, yes, we can add napari-ome-zarr. I don’t think we are yet in that world, but we could be soon. Something to discuss in community meetings etc.

In the meantime, since this issue is about a crash that no longer occurs, I suggest we close it!

Thanks Josh! In a fresh env I get:

ReaderPluginError(
napari.errors.reader_errors.ReaderPluginError: Tried to read https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.1/9822151.zarr with plugin napari, because it was associated with that file extension/because it is the only plugin capable of reading that path, but it gave an error. Try associating a different plugin or installing a different plugin for this kind of file.

So I guess this is OME zarr and napari builtins can’t handle it? Is there a way to have known this a priori? because it was associated with that file extension but erroring is not very graceful, for sure.

I installed napari-ome-zarr and it worked, maybe we should make that part of [all]? IMO zarr should be first class citizen?

Anyhow, I’ve spent about 30 min zooming and panning this dataset in both normal and async on main (3e3f2ce) and have not been able to reproduce a crash.

Thanks @joshmoore! Yeah I downloaded the datasets willy nilly which is why things were only half working. But my incompetence helped us uncover some bugs in how we handle plugins! =D

We’ll get something together to download

See https://github.com/ome/ome-zarr-py/pull/8