visit: OpenPMD reader crashes viewer due to VTK dataset parsing bug

@markcmiller86 Thank you so much for your kind reply and bug fixes. Awfully sorry for delay in responding.

I just tested against VisIt of version 3.3.3 with your great fixes applied, and it works fine as shown in the below figure. However, when I tested 3.4.1 with the fixes applied using the same OpenPMD file as previously uploaded, “Pseudocolor” and “Volume” plots crashed the application with the following error as shown in the following. The same error also appears when drawing the sliced 3D Pseudocolor plot. Interestingly, drawing “Contour” and “Histogram” plots seem to work without any problem.

I reproduced the error of VisIt 3.4.1 with debug flag on and attached the logs in the following.

VisIt 3.3.3 OpenPMD Fix applied

Running: gui3.3.3
Running: viewer3.3.3 -geometry 1415x928+505+54 -borders 26,4,4,4 -shift 0,0 -preshift 4,26 -defer -host 127.0.0.1 -port 5600
Running: mdserver3.3.3 -host 127.0.0.1 -port 5601
Error opening plugin file: /home/u3942590/.local/opt/visit/3.3.3/linux-x86_64_gcc-8.5/3.3.3/linux-x86_64/plugins/databases/libMIDXDatabase.so (/home/u3942590/.local/opt/visit/3.3.3/linux-x86_64_gcc-8.5/3.3.3/linux-x86_64/plugins/databases/libMIDXDatabase.so: undefined symbol: _ZN3MPI8Datatype4FreeEv)
Running: engine_ser3.3.3 -idle-timeout 480 -threads 14 -host 127.0.0.1 -port 5600
 

Visit 3.4.1 Plot Errors:

Running: gui3.4.1 -debug
Warning: debug level not specified, assuming 1
Running: viewer3.4.1 -debug -geometry 1121x766+415+27 -borders 26,4,4,4 -shift 0,0 -preshift 4,26 -defer -host 127.0.0.1 -port 5600
Warning: debug level not specified, assuming 1
Warning: debug level not specified, assuming 1
Running: mdserver3.4.1 -debug 1 1121x766+415+27 -host 127.0.0.1 -port 5601
Error opening plugin file: /opt/pkgs/VisIt/3.4.1/linux-x86_64_gcc-8.5/3.4.1/linux-x86_64/plugins/databases/libMIDXDatabase.so (/opt/pkgs/VisIt/3.4.1/linux-x86_64_gcc-8.5/3.4.1/linux-x86_64/plugins/databases/libMIDXDatabase.so: undefined symbol: _ZN3MPI8Datatype4FreeEv)
Running: engine_ser3.4.1 -idle-timeout 480 -threads 8 -debug 1 1121x766+415+27 -host 127.0.0.1 -port 5600
2024-05-22 16:28:18.813 (  38.126s) [        F441E000]      vtkDataReader.cxx:3367   ERR| vtkRectilinearGridReader (0x5ec9310): Cannot read field header! for file: 
VisIt's viewer exited abnormally! Aborting the Graphical User Interface. VisIt's developers may be reached via our GitHub discussions, https://github.com/visit-dav/visit/discussions

Visit 3.4.1 Logs: Log.zip

Best regards, Wei Lin

_Originally posted by @wlin8633 in https://github.com/visit-dav/visit/discussions/19514#discussioncomment-9518762_

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 16 (16 by maintainers)

Most upvoted comments

@markcmiller86 apologies, not sure what happened, except I was trying a lot of different things this morning. I recompiled and re-ran my simple test. I get the crash, too, with vtkDataReader saying it cannot read the ‘field header’. This is probably due to the fact that ‘Field’ is a keyword in vtk files (on par with scalars,vectors,normals).

@markcmiller86 I tried with rect3d.silo, defined an expr named field that aliased d. Visit crashed as soon as I plotted field with a pc plot.

VisIt 3.4.1 on macOS

@wlin8633 another work-around is to define an alias expression and plot that instead. So, use the expression system and define some expression (without the word field in the new expression’s name).

I have discovered the problem. I suspect it is a bug in VTK. Below, I paste the contents of a vtk file from our test suite that VisIt 3.4.1 can read…

# vtk DataFile Version 3.0
vtk output
ASCII
DATASET RECTILINEAR_GRID
DIMENSIONS 2 2 2
X_COORDINATES 2 float
-0.9 0.9 
Y_COORDINATES 2 float
-0.9 0.9 
Z_COORDINATES 2 float
-0.9 0.9 
CELL_DATA 1
FIELD FieldData 1
cellLabel 1 1 char
100 
POINT_DATA 8
SCALARS nodeLabels char
LOOKUP_TABLE default
65 66 67 68 69 70 71 72 

However, if I change the name of the SCALARS field from nodeLabels to Fields/E/x (the name of a variable in OpenPMD files), as in…

# vtk DataFile Version 3.0
vtk output
ASCII
DATASET RECTILINEAR_GRID
DIMENSIONS 2 2 2
X_COORDINATES 2 float
-0.9 0.9 
Y_COORDINATES 2 float
-0.9 0.9 
Z_COORDINATES 2 float
-0.9 0.9 
CELL_DATA 1
FIELD FieldData 1
cellLabel 1 1 char
100 
POINT_DATA 8
SCALARS Fields/E/x char
LOOKUP_TABLE default
65 66 67 68 69 70 71 72 

This VTK file will crash the engine in exactly the same place…

2024-05-28 23:38:17.029 (   0.649s) [         1029046]      vtkDataReader.cxx:3367   ERR| vtkRectilinearGridReader (0x7ff78e237fb0): Cannot read field header! for file: /Users/miller86/visit/visit/34rc/build/test1.vtk

The slashes in the SCALARS variable name make no difference. The string Fields_E_x will crash things too. Anything with Field will crash things. Almost certainly it is an issue in VTK’s reader code…which we will upstream to VTK shortly.

A short term fix for OpenPMD reader is to adjust that string to something else like Variable. However, if you have existing Python CLI code, such a change would break that code.