netcdf-c: OPeNDAP+HTTPS: certificates not found

With the new NetCDF 4.8.1 updated in Julia, we have an issue to accessing OPeNDAP resources over HTTPs. It seems that no certificate authority is loaded by default. In the new version of NetCDF 4.8.1 we also link against a new version of libCURL (now 7.81.0 previously 7.73.0).

  • the version of the software with which you are encountering an issue NetCDF 4.8.1

  • environmental information (i.e. Operating System, compiler info, java version, python version, etc.)

Ubuntu 20.04 and x86_64-linux-gnu-gcc (GCC) 4.8.5 (compilation is done within a docker container)

  • a description of the issue with the steps needed to reproduce it

nc_open (called by NCDataset) with a HTTPS url results in the following error:

julia> using NCDatasets
julia> NCDataset("https://erddap.ifremer.fr/erddap/griddap/SDC_GLO_CLIM_TS_V2_1")
Error:curl error: SSL peer certificate or SSH remote key was not OK
curl error details: 
Warning:oc_open: Could not read url
ERROR: NetCDF error: Opening path https://erddap.ifremer.fr/erddap/griddap/SDC_GLO_CLIM_TS_V2_1: NetCDF: I/O failure (NetCDF error code: -68)
Stacktrace:
 [1] nc_open(path::String, mode::UInt16)
[...]

If I define a ~/.ncrc file with the following content HTTP.SSL.CAINFO, then OPENDAP over HTTPS works:

HTTP.SSL.CAINFO=/etc/ssl/certs/ca-certificates.crt

I am wondering if somebody can shed some light how it worked before (as in NetCDF 4.7.4/libcurl 7.73.0 such configuration file was not needed) and if it is possible to restore the previous behavior.

As noted by @visr, some libraries (like GDAL, Proj4) expose a function to set this path programmatically. This solution would also be ideal for NetCDF in Julia.

(Any insight on how this is solved in e.g. anaconda or R would be very useful.)

Reference: https://github.com/Alexander-Barth/NCDatasets.jl/issues/173 https://github.com/JuliaPackaging/Yggdrasil/issues/4843

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 23 (11 by maintainers)

Commits related to this issue

Most upvoted comments

Thanks for the reminder. I plan to expand that API to a little better access to the rc file: e… insert, find, remove

Good point. As a developer, I sometimes forget to take the naive user into account. In include/ncrc.h there is a technically internal operation to insert a new key/value pair into the internal copy of .ncrc,

EXTERNL int NC_rcfile_insert(const char* key, const char* value, const char* hostport, const char* path);

You might try it as an experiment to see if it works for your problem. You can set the last two parameters to NULL. You will also need to declare the function signature in your code. I will try to think of the proper way to expose equivalent functionality.