napari: 3D rendering bug causing access violation

🐛 Bug

On playing with 3D rendering with napari I encountered two problems. Both happen when switching from 2D - slice view (which works in all cases) to 3D view.

Issue 1: Variant 3 in the code snippet below. Screen goes black upon switching to 3D view. No error message. When switching back to 2D view, everything is black. This happens with float32 and uint8 but not float64 dtypes. This is probably the same as #575 .

Issue 2: This is the one related to the issue title: uncomment Variant 1 in the code snippet below. Switch to 3D View. The viewer goes black. Switching back to 2D brings back the view. Traceback is being printed on switching to 3D. I have a feeling this is due to numpy returning a view rather than a copy which somehow causes an access violation.

WARNING: Error drawing visual <Volume at 0x2279140bfc8>
WARNING:vispy:Error drawing visual <Volume at 0x2279140bfc8>
WARNING: Traceback (most recent call last):
  File "c:\Users\Volker\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\ptvsd_launcher.py", line 43, in <module>
    main(ptvsdArgs)
  File "c:\Users\Volker\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\lib\python\ptvsd\__main__.py", line 432, in main     
    run()
  File "c:\Users\Volker\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\lib\python\ptvsd\__main__.py", line 316, in run_file 
    runpy.run_path(target, run_name='__main__')
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "c:\Users\Volker\Dropbox\Github\NapariPlayground\Volume\show_volume.py", line 25, in <module>
    viewer.add_image(vol)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\contextlib.py", line 119, in __exit__
    next(self.gen)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\napari\_qt\event_loop.py", line 21, in gui_qt
    app.exec_()
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\app\backends\_qt.py", line 501, in event
    out = super(QtBaseCanvasBackend, self).event(ev)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\app\backends\_qt.py", line 818, in paintGL
    self._vispy_canvas.events.draw(region=None)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\util\event.py", line 455, in __call__
    self._invoke_callback(cb, event)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\util\event.py", line 475, in _invoke_callback
    self, cb_event=(cb, event))
  << caught exception here: >>
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\OpenGL\latebind.py", line 41, in __call__
    return self._finalCall( *args, **named )
TypeError: 'NoneType' object is not callable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\util\event.py", line 471, in _invoke_callback
    cb(event)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\scene\canvas.py", line 217, in on_draw
    self._draw_scene()
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\scene\canvas.py", line 266, in _draw_scene
    self.draw_visual(self.scene)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\scene\canvas.py", line 304, in draw_visual
    node.draw()
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\scene\visuals.py", line 99, in draw
    self._visual_superclass.draw(self)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\visuals\visual.py", line 443, in draw
    self._vshare.index_buffer)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\visuals\shaders\program.py", line 101, in draw
    Program.draw(self, *args, **kwargs)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\gloo\program.py", line 533, in draw
    canvas.context.flush_commands()
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\gloo\context.py", line 176, in flush_commands
    self.glir.flush(self.shared.parser)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\gloo\glir.py", line 572, in flush
    self._shared.flush(parser)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\gloo\glir.py", line 494, in flush
    parser.parse(self._filter(self.clear(), parser))
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\gloo\glir.py", line 819, in parse
    self._parse(command)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\gloo\glir.py", line 787, in _parse
    ob.set_data(*args)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\gloo\glir.py", line 1640, in set_data
    glTexSubImage3D(self._target, 0, x, y, z, format, gtype, data)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\gloo\glir.py", line 1607, in glTexSubImage3D
    width, height, depth, format, type, pixels)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\OpenGL\latebind.py", line 45, in __call__
    return self._finalCall( *args, **named )
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\OpenGL\wrapper.py", line 868, in wrapperCall
    result = wrappedOperation( *cArguments )
OSError: exception: access violation reading 0x00000227A5C97000
WARNING:vispy:Traceback (most recent call last):
  File "c:\Users\Volker\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\ptvsd_launcher.py", line 43, in <module>
    main(ptvsdArgs)
  File "c:\Users\Volker\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\lib\python\ptvsd\__main__.py", line 432, in main     
    run()
  File "c:\Users\Volker\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\lib\python\ptvsd\__main__.py", line 316, in run_file 
    runpy.run_path(target, run_name='__main__')
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "c:\Users\Volker\Dropbox\Github\NapariPlayground\Volume\show_volume.py", line 25, in <module>
    viewer.add_image(vol)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\contextlib.py", line 119, in __exit__
    next(self.gen)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\napari\_qt\event_loop.py", line 21, in gui_qt
    app.exec_()
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\app\backends\_qt.py", line 501, in event
    out = super(QtBaseCanvasBackend, self).event(ev)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\app\backends\_qt.py", line 818, in paintGL
    self._vispy_canvas.events.draw(region=None)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\util\event.py", line 455, in __call__
    self._invoke_callback(cb, event)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\util\event.py", line 475, in _invoke_callback
    self, cb_event=(cb, event))
  << caught exception here: >>
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\OpenGL\latebind.py", line 41, in __call__
    return self._finalCall( *args, **named )
TypeError: 'NoneType' object is not callable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\util\event.py", line 471, in _invoke_callback
    cb(event)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\scene\canvas.py", line 217, in on_draw
    self._draw_scene()
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\scene\canvas.py", line 266, in _draw_scene
    self.draw_visual(self.scene)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\scene\canvas.py", line 304, in draw_visual
    node.draw()
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\scene\visuals.py", line 99, in draw
    self._visual_superclass.draw(self)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\visuals\visual.py", line 443, in draw
    self._vshare.index_buffer)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\visuals\shaders\program.py", line 101, in draw
    Program.draw(self, *args, **kwargs)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\gloo\program.py", line 533, in draw
    canvas.context.flush_commands()
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\gloo\context.py", line 176, in flush_commands
    self.glir.flush(self.shared.parser)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\gloo\glir.py", line 572, in flush
    self._shared.flush(parser)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\gloo\glir.py", line 494, in flush
    parser.parse(self._filter(self.clear(), parser))
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\gloo\glir.py", line 819, in parse
    self._parse(command)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\gloo\glir.py", line 787, in _parse
    ob.set_data(*args)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\gloo\glir.py", line 1640, in set_data
    glTexSubImage3D(self._target, 0, x, y, z, format, gtype, data)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\vispy\gloo\glir.py", line 1607, in glTexSubImage3D
    width, height, depth, format, type, pixels)
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\OpenGL\latebind.py", line 45, in __call__
    return self._finalCall( *args, **named )
  File "C:\Users\Volker\Anaconda3\envs\napari_new\lib\site-packages\OpenGL\wrapper.py", line 868, in wrapperCall
    result = wrappedOperation( *cArguments )
OSError: exception: access violation reading 0x00000227A5C97000
ERROR: Invoking <bound method SceneCanvas.on_draw of <SceneCanvas (PySide2) at 0x227867db348>> for DrawEvent
ERROR:vispy:Invoking <bound method SceneCanvas.on_draw of <SceneCanvas (PySide2) at 0x227867db348>> for DrawEvent
WARNING: Error drawing visual <Volume at 0x2279140bfc8>
WARNING:vispy:Error drawing visual <Volume at 0x2279140bfc8>
ERROR: Invoking <bound method SceneCanvas.on_draw of <SceneCanvas (PySide2) at 0x227867db348>> repeat 2
ERROR:vispy:Invoking <bound method SceneCanvas.on_draw of <SceneCanvas (PySide2) at 0x227867db348>> repeat 2
WARNING: Error drawing visual <Volume at 0x2279140bfc8>
WARNING:vispy:Error drawing visual <Volume at 0x2279140bfc8>

To Reproduce

Steps to reproduce the behavior:

Download sample volume https://www.dropbox.com/s/93ykwmonofq7171/droso_ovarioles.tif?dl=0
Uncomment one of the different Variants in the code snippet below. Variant 1 produces traceback shown above. I first thought maybe this is because numpy returned a view, but that does not seem to be the case. Maybe it has to do with exact array dimensions ?

import napari
from skimage.io import imread
import numpy as np 


vol = imread(".\Data\droso_ovarioles.tif")


# Variant 1
# with subsampling ... this crashes in OpenGL\wrapper.py
vol = vol[::2,::2,::2].astype(np.float64)
print(vol.flags)
# Variant 2
# no subsampling, but changing dtype to float64
# vol = vol.astype(np.float64)


# Variant 3
# original dtype
# do nothing
print(vol.dtype) # it is float32 for the dtype


with napari.gui_qt(): 
    viewer = napari.Viewer()
    viewer.add_image(vol)

Expected behavior

3D Rendering of volume as in Variant 2

Environment

napari Version (e.g., 1.0): 0.2.0
OS (e.g., Linux): Win 10
Python version: 3.7.4
Any other relevant information:

Additional context

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 53 (38 by maintainers)

Most upvoted comments

Ok I did more research and I think everyone is right. 😉

This discussion is about row alignment of the image provided by the user in the host memory.

Yes, but we are describing how the data is organized in the client (CPU-side) memory so that the GPU can read it correctly and most efficiently. So there are some low-level hardware and memory transfer side effects to this, but we have nothing more to go on than “And if you are interested, most GPUs like chunks of 4 bytes” in the khronos wiki.

I think you are right on the byte alignment part. I think I misread whatever stackoverflow or wiki page I had read about the data format being included in the alignment calculation. I think vispy, or the code it borrowed from was written assuming all textures were uint8. So, the code should definitely be updated to work with other data types.

I re-read the code that normalizes the shape of the data when it gets set on a gloo Texture object and I missed something important. Let’s say for example you have a 2D texture. If you give it a 2D array, then it assumes that this is a Luminance “image” (a single channel of data) and it will add the extra dimension. If you give it an RGB array (rows, cols, 3) then it will keep the 3 in the last dimension. I had thought it was always adding a (1,) to the shape. This makes the alignment calculations make much more sense when dealing with RGB/A data.

So I think you are right @cgohlke that we need to be taking itemsize in to account when getting the alignment. I think we still should determine the highest possible alignment based on these values though if it effects performance at all. I do think we (CC @kmuehlbauer) are making an assumption that new GPUs would prefer 8 instead of 4.

I’ll try to find time to update my PR to use itemsize in the next couple days hopefully.

djhoese on Nov 18, 2019

I would be surprised if setting the alignment to the highest possible value has any significant effect on performance. It would be trivial for the OpenGL driver to calculate that value given pixel format and type, and GL_UNPACK_ values.

Regardless, does my change fix this test case on your system?

Yes. However, since GL_UNPACK_ALIGNMENT is a byte alignment it should be

alignment = self._get_alignment(data.shape[-2] * data.shape[-1] * data.itemsize)

No?

cgohlke on Nov 18, 2019

vispy-only (no napari code) example code

diff --git a/examples/basics/scene/volume.py b/examples/basics/scene/volume.py
index c3f6b856..c3d7a222 100644
--- a/examples/basics/scene/volume.py
+++ b/examples/basics/scene/volume.py
@@ -36,6 +36,7 @@ from vispy.visuals.transforms import STTransform

 # Read volume
 vol1 = np.load(io.load_data_file('volume/stent.npz'))['arr_0']
+vol1 = np.random.random((68, 296, 393)).astype(np.float32)
 vol2 = np.load(io.load_data_file('brain/mri.npz'))['data']
 vol2 = np.flipud(np.rollaxis(vol2, 1))

cgohlke on Nov 12, 2019

Vispy 0.6.3 is out on PyPI! Waiting for conda-forge to pick it up and all that stuff.

djhoese on Nov 27, 2019

Attempted fix in https://github.com/vispy/vispy/pull/1758

djhoese on Nov 18, 2019

GL_UNPACK_ALIGNMENT is a byte alignment

No,

I checked the OpenGL red and blue books and all they mention are row alignments in units of bytes for GL_UNPACK_ALIGNMENT.

you’re assuming a lot about GPU design

This discussion is about row alignment of the image provided by the user in the host memory.

cgohlke on Nov 18, 2019

Maybe I am missing something, but it seems vispy wrongly calculates the alignment to 8 bytes. The rows of a numpy float32 array are word-aligned (4 bytes). On this system there’s no crash when using a 4 bytes alignment (the OpenGL default).

cgohlke on Nov 12, 2019

I will check whether the fixes to the float32-issue in vispy fixed this as well. Will have to wait until tomorrow though.

VolkerH on Oct 27, 2019