napari: Behaviour of `layer.corner_pixels` is inconsistent

🐛 Bug

The behaviour of layer.corner_pixels is inconsistent when working with combinations of 2D, multiscale and 3D images. Only multiscale images displayed in 2D take into account layer._dims_displayed leading to different corner_pixel values depending on the number of displayed dimensions.

Additionally, rolling dimensions when 2D and 3D images are in the viewer leads to incorrect corner_pixels for the 2D layer, which gets one correct set of coordinates and one set to the current slider value.

This inconsistent behaviour makes it difficult to reliably compute the data pixels currently on screen.

To Reproduce

Steps to reproduce the behavior:

import napari
from skimage import data
from scipy.ndimage import zoom
import numpy as np

with napari.gui_qt():
    viewer = napari.Viewer()

    img       = data.camera()
    img_multi = [zoom(img,(1/s,1/s)) for s in (1,2,4)]
    img3D     = np.tile(img[np.newaxis],(8,1,1))

    viewer.add_image(img3D)
    viewer.add_image(img)
    viewer.add_image(img_multi)

Then inspect viewer.layers.corner_pixels before and after rolling dimensions and/or toggling the number of displayed dimensions.

Expected behavior

The layer.corner_pixels attribute consistently displays the top-left and bottom-right pixel coordinates currently being displayed on screen regardless of whether the image is 2D, 3D or multiscale, and regardless of whether or not dimensions have been rolled.

About this issue

Original URL
State: open
Created 3 years ago
Comments: 15 (15 by maintainers)

Most upvoted comments

We recently worked around the lack of channel_axis support for labels in https://github.com/ome/napari-ome-zarr/pull/16 by squeezing out the channel axis (since we assume labels data from OME-Zarr is the same shape as image data). However, if you did have OME-Zarr labels data with size-C being >1 then I think the squeezing would fail and you wouldn’t be able to open in napari. We need to work around this in napari-ome-zarr, either with a channel_axis support, or splitting out channels.

will-moore on Nov 8, 2021

We actually had the same discussion, i.e. that it could make sense to have a channel axis for label images to group segmentations corresponding to different structures of the same underlying data. So yes, it could make sense to add a channel_axis to labels layer. Should I open a separate issue for this? And while this would fix the specific problem we encountered when combining a 5D label array with 4D image arrays, I think the fundamental problem with incorrect corner_pixels computation in some circumstances with layers with different .ndims goes deeper.

VolkerH on Oct 4, 2021

The use case that originally led to this bug being filed was trying to perform some computation on just the pixels currently displayed on screen. This is not possible directly with the current corner_pixels attribute, which was originally added to support the implementation of multiscale layers.

What is needed is some function which, given layer.data, viewer.camera, layer.scale, viewer.dims.ndisplay and viewer.dims.order returns the top-left and bottom-right indices of the bounding box in layer.data containing all pixels on screen (in highest resolution when the layer is multiscale?).

so something like

get_data_in_view(
    layer.data, 
    layer.scale, 
    viewer.camera, 
    viewer.dims.ndisplay, 
    viewer.dims.order) ->  (top_left_corner : Tuple, bottom_right_corner: Tuple)

I’m not sure whether this should be its own utility function or a method on the viewer? Certainly it shouldn’t live on layer as it needs to mostly know about the viewer, not the layer itself.

Special cases/things to consider

layer is multiscale: return coordinates in highest resolution level?
dimensions have been rolled: coordinates should be able to directly index into data - use axis order to transpose the coordinates/ switch them around as required

DragaDoncila on Apr 9, 2021