spack: External cray-mpich provides incorrect spec['mpi'].prefix.include path

Steps to reproduce

This is observed on the spock system, a cray early-access machine at ORNL. It causes several mpi dependent package builds to fail including TAU, which I will use as an example.

packages.yaml looks like:

packages:
    mpich:
      buildable: false
      externals:
      - spec: mpich@8.1.7
        modules:
        - cray-mpich/8.1.7

The TAU package sets up its mpi configuration like this:

            options.append("-mpiinc=%s" % spec['mpi'].prefix.include)
            options.append("-mpilib=%s" % spec['mpi'].prefix.lib)

Which translates to this in the actual configure invocation: '-mpiinc=/opt/cray/pe/mpich/8.1.7/gtl/include' '-mpilib=/opt/cray/pe/mpich/8.1.7/gtl/lib'

However, /opt/cray/pe/mpich/8.1.7/gtl/include does not exist. The include files for this mpi install are located in compiler specific subdiriectories of /opt/cray/pe/mpich/8.1.7/ofi and /opt/cray/pe/mpich/8.1.7/ucx. (e.g. /opt/cray/pe/mpich/8.1.7/ofi/gnu/9.1/include)

This makes the system-provided mpich impossible to use as an external, at least for packages that use spec[‘mpi’].prefix.include to get the include path.

Error message

No response

Information on your system

  • Spack: 0.16.2-4019-80713e234c
  • Python: 3.6.13
  • Platform: cray-sles15-zen2
  • Concretizer: original

General information

  • I have run spack debug report and reported the version of Spack/Python/Platform
  • I have searched the issues of this repo and believe this is not a duplicate
  • I have run the failing commands in debug mode and reported the output

About this issue

  • Original URL
  • State: open
  • Created 5 years ago
  • Comments: 28 (28 by maintainers)

Most upvoted comments

I also have been having a problem building on the Cray. After seeing this I added a modules setting (I had the prefix) to the externals file and now I can build our MPI collectors on a Cray. The paths to the MPI installation were missing.