netcdf-c: nc_create_par triggers HDF5 error stack message related to H5FDunregister
To report a non-security related issue, please provide:
-
the version of the software with which you are encountering an issue v4.9.3-rc1
-
environmental information (i.e. Operating System, compiler info, java version, python version, etc.) Frontier (ORNL), Cray clang version 17.0.0 (b59b7a8e9169719529cf5ab440f3c301e515d047)
-
a description of the issue with the steps needed to reproduce it
A call to status = nc_create_par(output_path, NC_NETCDF4 | NC_CLOBBER, MPI_COMM_WORLD, MPI_INFO_NULL, &ncid);
is returning the HDF5 error message
HDF5-DIAG: Error detected in HDF5 (1.15.0):
#000: /ccs/home/brtnfld/packages/hdf5/src/H5FD.c line 368 in H5FDunregister(): not a file driver
major: Invalid arguments to routine
minor: Inappropriate type
It does not cause an error; it is just a distraction since all the ranks print the message. It seems to be triggered by H5FD_http_finalize
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 19 (13 by maintainers)
Commits related to this issue
- Integrating fix suggested by jhendersonHDF in support of https://github.com/Unidata/netcdf-c/issues/2990 — committed to WardF/netcdf-c by WardF 2 years ago
This was closed as part of the PR merges that went in; I recognize it may be premature and will re-open if folk are still observing this. We’ve incorporated #3012 which was testing for the issue (thanks @edwardhartnett) and #3013 which incorporated a fix (thanks @jhendersonHDF), after which the tests went from failing to passing.
I’m still seeing MPI related issues with the most recent Ubuntu, but I will open a new issue.
@WardF @edhartnett Hi all, I believe what may be going on here is an ordering issue where HDF5 has already closed the ID used for the HTTP VFD before getting to the part where the
H5FD_http_finalizefunction (which callsH5FDunregister) is called. So, the VFD is holding on to what it thinks is a valid ID, but it actually isn’t. Normally, the termination callback of a VFD (in this case,H5FD_http_term) is where one would unregister and reset the ID for a VFD, but it looks like the termination callback for the HTTP VFD does nothing and the ID isn’t unregistered until later on when the HDF5 dispatch code callsH5FD_http_finalizedirectly. The easiest fix would probably be to add a call toH5FD_http_finalizeto check that the ID is valid before callingH5FDunregister, as in:https://github.com/Unidata/netcdf-c/blob/main/libhdf5/H5FDhttp.c#L259-L261
Alternatively, the code to unregister the ID could be moved to
H5FD_http_terminstead ofH5FD_http_finalize, but I don’t know if that would break something from the netCDF-C perspective.I confirmed that it also happens with 1.14.4-3.