netcdf-c: nc_create_par triggers HDF5 error stack message related to H5FDunregister

To report a non-security related issue, please provide:

  • the version of the software with which you are encountering an issue v4.9.3-rc1

  • environmental information (i.e. Operating System, compiler info, java version, python version, etc.) Frontier (ORNL), Cray clang version 17.0.0 (b59b7a8e9169719529cf5ab440f3c301e515d047)

  • a description of the issue with the steps needed to reproduce it

A call to status = nc_create_par(output_path, NC_NETCDF4 | NC_CLOBBER, MPI_COMM_WORLD, MPI_INFO_NULL, &ncid);

is returning the HDF5 error message

HDF5-DIAG: Error detected in HDF5 (1.15.0):
  #000: /ccs/home/brtnfld/packages/hdf5/src/H5FD.c line 368 in H5FDunregister(): not a file driver
    major: Invalid arguments to routine
    minor: Inappropriate type

It does not cause an error; it is just a distraction since all the ranks print the message. It seems to be triggered by H5FD_http_finalize

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 19 (13 by maintainers)

Commits related to this issue

Most upvoted comments

This was closed as part of the PR merges that went in; I recognize it may be premature and will re-open if folk are still observing this. We’ve incorporated #3012 which was testing for the issue (thanks @edwardhartnett) and #3013 which incorporated a fix (thanks @jhendersonHDF), after which the tests went from failing to passing.

I’m still seeing MPI related issues with the most recent Ubuntu, but I will open a new issue.

@WardF @edhartnett Hi all, I believe what may be going on here is an ordering issue where HDF5 has already closed the ID used for the HTTP VFD before getting to the part where the H5FD_http_finalize function (which calls H5FDunregister) is called. So, the VFD is holding on to what it thinks is a valid ID, but it actually isn’t. Normally, the termination callback of a VFD (in this case, H5FD_http_term) is where one would unregister and reset the ID for a VFD, but it looks like the termination callback for the HTTP VFD does nothing and the ID isn’t unregistered until later on when the HDF5 dispatch code calls H5FD_http_finalize directly. The easiest fix would probably be to add a call to H5FD_http_finalize to check that the ID is valid before calling H5FDunregister, as in:

    if (H5FD_HTTP_g && (H5Iis_valid(H5FD_HTTP_g) > 0))
         H5FDunregister(H5FD_HTTP_g);

https://github.com/Unidata/netcdf-c/blob/main/libhdf5/H5FDhttp.c#L259-L261

Alternatively, the code to unregister the ID could be moved to H5FD_http_term instead of H5FD_http_finalize, but I don’t know if that would break something from the netCDF-C perspective.

I confirmed that it also happens with 1.14.4-3.