spack: Picks wrong MPI for mfem but not other packages

In my packages.py I have

    openmpi:
        paths:
            openmpi@2.0.1: /usr/local
        version: [system]
        buildable: False        

and

    all:
        providers:
            # prefer OpenMPI over MPICH by default
            mpi: [openmpi, mpich]

Now in a clean spack directory (after git clean -x -f -d) I get

$ ./bin/spack install mpi
==> openmpi is externally installed in /usr/local
~/Src/spack (develop=) 
$ ./bin/spack install hypre
==> Installing hypre
==> netlib-lapack is externally installed in /usr
==> openmpi is externally installed in /usr/local
==> Fetching http://computation.llnl.gov/project/linear_solvers/download/hypre-2.11.1.tar.gz
.... Builds hypre fine

~/Src/spack (develop=) 
$ ./bin/spack install mfem
==> Installing mfem
==> Installing mvapich2
==> Installing libpciaccess
==> Installing util-macros
Starts the massive project of installing mvapich2 and its dependencies.

Spack is fantasizing that it should use some wacked out MPI mvapich2 instead of the default. This happens with mfem but not with hypre, petsc, superlu_dist, mumps …

This is the develop branch on my Mac.

Not if you use ./bin/spack install mfem+mpi then it does use the OpenMPI. mfem/package.py does have a default dependence on mpi

variant('mpi', default=True, description='Activate support for MPI')

so there is something wrong with how Spack selects the MPI when it is not provided on the command line for the install.

About this issue

  • Original URL
  • State: closed
  • Created 10 years ago
  • Comments: 18 (18 by maintainers)

Most upvoted comments

@BarrySmith I cannot reproduce this problem on my Mac from the develop branch, unfortunately (see attached) log.txt . But I think there is progress to be made here:

  1. You can figure out what Spack is going to do using spack spec -l instead of spack install. So first… on the version of Spack you are CURRENTLY running, please share the outputs of spack spec -l hypre and spack spec -l mfem. I expect they will be consistent with your results above on spack install, but will give more detail.

  2. Please indicate the exact version of Spack you are using by: a) Share the result of git log | head. If that revision is not available in the main Spack repo, please indicate which Spack fork it can be downloaded from. b) Show that EVERYTHING is in your Spack is checked into Git by running git status.

  3. When you run with a particular configuration of packages.yaml, also share with us the results of spack config get packages. This will indicate the configuration that Spack is ACTUALLY working with, after it merges the various packages.yaml files. Please include this output, not just snippets of your packages.yaml file (there might be something you’re doing somewhere in the config that is subverting you).

  4. Use the following in your packages.yaml file instead of what you shared above (I assume you meant packages.yaml, not packages.py):

    openmpi:
        paths:
            openmpi@2.0.1: /usr/local
        version: [2.0.1]
        buildable: False   

See if that makes any difference.

  1. Try turning off buildable: False, see if it uses openmpi now (probably a lower version).

My Best Guess

MPI is unusual for a virtual dependency because it comes in many versions. openmpi@2.0.1 implements the virtual dependency mpi@3.1. See in openmpi/package.py:

    provides('mpi@:2.2', when='@1.6.5')
    provides('mpi@:3.0', when='@1.7.5:')
    provides('mpi@:3.1', when='@2.0.0:')

If Spack wants to use mpi@3.1, it will be happy to use the openmpi@2.0.1 that you provided it. If Spack thinks it needs a lower version of mpi — say, mpi@3.0 — then it will know it can’t use openmpi@2.0.1. Instead, it needs to go back to openmpi@1.7.5. But you set openmpi to buildable: False, so Spack is not allowed to build openmpi@1.75. It then falls back on its next option, which is mpich.

My best guess is that the information requested above will flush out the bug. I think there’s something else in your packages.yaml file that is forcing an MPI version lower than mpi@3.1 to be selected. And if I’m wrong in this guess… the above requested info should provide everything needed to reconstruct this bug on another Mac.