spack: Problems building HDF5 with Intel OpenMPI

Is anyone else have trouble building HDF5 with Intel and OpenMPI? It works fine outside of Spack but with Spack it crashes during make. It also works with Spack with any other compiler or any other MPI library, just not Intel and OpenMPI. Here is the configuration I’m using:

spack install hdf5 %intel +szip -cxx +mpi ^openmpi+tm+psm

The Intel compiler is 16.0.1, OpenMPI 1.10.2, and HDF5 1.10.0 (although I think this problem still occurred with the previous HDF5 version).

When built outside of Spack, everything looks good:

checking for Fortran INTEGER KINDs... {1,2,4,8}
checking for Fortran REAL KINDs... {4,8,16}
checking for Fortran REALs maximum decimal precision... 34
checking sizeof of native KINDS...
checking for Fortran INTEGER KINDs... {1,2,4,8}
checking for Fortran REAL KINDs... {4,8,16}
checking for Fortran REALs maximum decimal precision... 34
checking sizeof of available INTEGER KINDs... {1,2,4,8}
checking sizeof of available REAL KINDs... {4,8,16}
checking if Fortran compiler supports intrinsic C_LONG_DOUBLE... yes
checking if Fortran C_LONG_DOUBLE is different from C_DOUBLE... yes
checking for Fortran interoperable KINDS with C... {4,8,16}

When built with Spack, I’m seeing the following warning message during configure:

Error
configure: WARNING: Failed to run program to determine available KINDs
checking sizeof of native KINDS... configure: WARNING: Fortran program fails to build or run!

Error
configure: WARNING: Failed to run program to determine available KINDs
checking sizeof of available INTEGER KINDs... {}
checking sizeof of available REAL KINDs... {}
checking if Fortran compiler supports intrinsic C_LONG_DOUBLE... yes
checking if Fortran C_LONG_DOUBLE is different from C_DOUBLE... yes
configure: WARNING:
           Fortran REAL(KIND=) is  Bytes, but no corresponding C float type exists of that size
                     !!! Fortran interfaces will not be generated for REAL(KIND=) !!!

checking for Fortran interoperable KINDS with C...

This eventually leads to errors in make:

H5match_types.c(155): error: expected an expression
    int IntKinds[] = H5_FORTRAN_INTEGER_KINDS;
                                             ^

Logs outside of Spack: config.log soft-build.out

Logs when using Spack: config.log spack-build.out

About this issue

  • Original URL
  • State: closed
  • Created 11 years ago
  • Comments: 30 (29 by maintainers)

Commits related to this issue

Most upvoted comments

Alternatively, in https://groups.google.com/forum/#!topic/spack/5jVVwd1OeqQ, James recommends adding this directory as an extra_rpaths in compilers.yaml. I’ve never tried this before, but it may work if you don’t have write access. Worth a shot if you’re in a rush.

@svenevs Just for completeness: you can still use .cfg files and put them wherever you want. In that case you need to set ICCCFG & Co. to point the Intel compilers to the configuration files. See here.

Excellent, I went with the compilers.yaml approach since we’d have to jump through some annoying hoops to create the .cfg files down there (only IT has root privs…). So for completeness, in my specific case, the relevant section of compilers.yaml is:

- compiler:
    environment: {}
    extra_rpaths: [/usr/local/intel/composer_xe_2015.3.187/compiler/lib/intel64]
    flags: {}
    modules: []
    operating_system: rhel6
    paths:
      cc: /usr/local/intel/bin/icc
      cxx: /usr/local/intel/bin/icpc
      f77: /usr/local/intel/bin/ifort
      fc: /usr/local/intel/bin/ifort
    spec: intel@15.0.3
    target: x86_64

There may be some unexpected errors in the future, from the above we had

$ locate libimf.so
/usr/local/intel/composer_xe_2015.3.187/compiler/lib/intel64/libimf.so
/usr/local/intel/composer_xe_2015.3.187/compiler/lib/mic/libimf.so
/usr/local/intel/itac/9.0.3.051/intel64/bin/rtlib/libimf.so

I chose the intel64 folder and things are seemingly happy. The issue is the pesky mic folder. On this cluster we have the old version of the Xeon Phi cards which use a #pragma offload(target:mic) programming model. These were the first generation, where the host compiler and Phi compiler are different. Anybody else reading this should hopefully not even have a mic compiler (assuming newer cards and/or more up to date intel compiler) described in the google forum link Adam provides above 😉

Thanks so much for consolidating all of the information @adamjstewart 😍

Ahhhhhh ok I’ll have to get el jefe to make those then, I don’t have write access there. I thought the *.cfg files were supposed to be going under spack somewhere, thanks!

@svenevs I think this discussion is even more clear: https://groups.google.com/d/topic/spack/NxyNTAZyMQg/discussion

Basically, Intel’s compilers do not know where Intel’s libraries live, they rely on things like LD_LIBRARY_PATH to locate them. As you noted, Spack unsets LD_LIBRARY_PATH before building to clean the environment, so we can’t rely on it. The recommended way to get around this is to create icc.cfg, icpc.cfg, and ifort.cfg files to tell the compilers where their libraries can be found. These files need to be located in the same directory as the compilers themselves. Note that prefix/bin contains these compilers, but these are just symlinks. The config files need to be in the same directory as the actual compilers.

Also note that this problem only occurs when using an externally installed Intel compiler. If you let Spack install your compilers, it will solve this problem for you. Check out rpath_configuration in both intel and intel-parallel-studio for the details.

The installation directories change with each version, but it should look something like:

# Find icc directory
$ which icc
/usr/local/intel/compilers_and_libraries_2015.3.187/bin/icc
# Follow symlinks to find real icc directory
$ readlink -f /usr/local/intel/compilers_and_libraries_2015.3.187/bin/icc
/usr/local/intel/compilers_and_libraries_2015.3.187/linux/bin/intel64/icc
# Create icc.cfg, icpc.cfg, and ifort.cfg files in this directory containing:
$ cat /usr/local/intel/compilers_and_libraries_2015.3.187/linux/bin/intel64/icc.cfg
-Xlinker -rpath -Xlinker /usr/local/intel/compilers_and_libraries_2015.3.187/linux/compiler/lib/intel64_lin

In the long run I think we should keep compile-time tests (i.e. those who require to have a stage available) separated from run-time tests (i.e. those who are just a sanity check of the installed software). None of the two imo should change the hash of the installed spec.

OpenMPI could depends_on('intel-mpi-benchmarks', when='+test'), but I’m not sure how I feel about that. Ideally, it would be cool to have a separate test() function in every Spack package, and whenever you run spack install -t or spack test, it would run those test commands. It could include simple things like make test as well as more complex things like benchmarks.

We have an openmpi module compiled with intel here and I remember being able to solve this problem by adding specific variants. If I remember well the issue was related to dlopen and hwloc openmpi option. The problem is that it gives no error at build time but it has problems at runtime. That was the reason to ask for a simple code that test mpi. I was thinking to try Intel® MPI Benchmarks that seems to be downloadable does it make sense to make a spack package of it that can depends on mpi and add a test function? ( hints wanted 😉