netcdf-c: Bad data read or conversion to double on compressed 32-bit floats?

Hi,

I’m experiencing this on Ubuntu 18.04 LTS with gcc version 7.5.0 and various versions of the netcdf library, including 4.7.4, 4.7.3, and 4.6.1 (the oldest one I have around).

The attached netcdf file has variable ‘phycocyanin’ with the following characteristics from ncdump -s:

    float phycocyanin(time) ;
            phycocyanin:_FillValue = -9999.f ;
            phycocyanin:units = "RFU" ;
            phycocyanin:long_name = "Phycocyanin" ;
            phycocyanin:_Storage = "chunked" ;
            phycocyanin:_ChunkSizes = 1440 ;
            phycocyanin:_DeflateLevel = 4 ;
            phycocyanin:_Shuffle = "true" ;
            phycocyanin:_Endianness = "little" ;

‘ncdump’ reports all the data as missing:

phycocyanin = _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, (etc.)

When I read the data into a floating point array using the C library using nc_get_var_float() then I get the expected values of -9999.0. However when I read it into a double precision array with nc_get_var_double() then I get strange values. Here is the output of the little test program that I’ve appended below:

Sizeof float: 4 double: 8 ======== DOUBLE ======== -559239646634519513659653226496.000000 -559239646634519513659653226496.000000 -559239646634519513659653226496.000000 -559239646634519513659653226496.000000 -559239646634519513659653226496.000000 -559239646634519513659653226496.000000 -559239646634519513659653226496.000000 -559239646634519513659653226496.000000 -559239646634519513659653226496.000000 -559239646634519513659653226496.000000
======== FLOAT ======== -9999.000000 -9999.000000 -9999.000000 -9999.000000 -9999.000000 -9999.000000 -9999.000000 -9999.000000 -9999.000000 -9999.000000

This seems like a bug, unless I’m overlooking something obvious in the test code (always possible). The nc_get_var_double() call is supposed to convert the netcdf file’s values into double precision and store them in the provided double precision array, right? That’s my understanding anyway.

Test program:

#include <stdio.h>
#include <stdlib.h>
#include "netcdf.h"

void main( int argc, char *argv[] )
{
        int     err, ncid, varid;
        size_t  nt;
        double  *ddat;
        float   *fdat;

        nt = 1440;      /* just hardcode for test */

        printf( "Sizeof float: %ld  double: %ld\n", sizeof(float), sizeof(double) );

        err = nc_open("mendota_buoy.2018-11-08.nc", 0, &ncid );
        if( err != 0 ) {
                printf( "err open = %d\n", err );
                exit(-1);
                }

        err = nc_inq_varid( ncid, "phycocyanin", &varid );
        if( err != 0 ) {
                printf( "err inq_varid = %d\n", err );
                exit(-1);
                }

        /* Make room for both double and floating dat */
        ddat = (double *)malloc( sizeof(double) * nt );
        fdat = (float  *)malloc( sizeof(float ) * nt );

        /* Read into double array. Supposed to convert to double...? */
        err = nc_get_var_double( ncid, varid, ddat );
        if( err != 0 ) {
                printf( "err get_var_double = %d\n", err );
                exit(-1);
                }

        printf( "======== DOUBLE ========\n" );
        for( int ii=0; ii<10; ii++ )
                printf( "%lf  ", ddat[ii] );

        printf( "\n" );

        /* === Do same thing, but for float === */

        err = nc_get_var_float( ncid, varid, fdat );
        if( err != 0 ) {
                printf( "err get_var_float = %d\n", err );
                exit(-1);
                }

        printf( "======== FLOAT ========\n" );
        for( int ii=0; ii<10; ii++ )
                printf( "%f  ", fdat[ii] );

        printf( "\n" );
}

mendota_buoy.2018-11-08.zip