spack: spec.py _replace_with dictionary error

From what I can tell, ‘mpi’ is getting replaced with my concrete choice ‘mpich@3’ and so ‘mpi’ is removed from the dependencies list. But this happens for at least three different packages (hypre, superlu-dist, hdf5), and ‘mpi’ is no longer in the dictionary.

sarich@cetuslac1 spack]$ ./bin/spack spec petsc
Input spec
------------------------------
  petsc

Normalized
------------------------------
  petsc
      ^blas
      ^lapack
      ^python@2.6:2.7
          ^bzip2
          ^ncurses
          ^openssl
              ^zlib
          ^readline
          ^sqlite

Concretized
------------------------------
Traceback (most recent call last):
  File "./bin/spack", line 202, in <module>
    main()
  File "./bin/spack", line 179, in main
    return_val = command(parser, args)
  File "/gpfs/mira-home/sarich/spack/lib/spack/spack/cmd/spec.py", line 57, in spec
    spec.concretize()
  File "/gpfs/mira-home/sarich/spack/lib/spack/spack/spec.py", line 1329, in concretize
    self._expand_virtual_packages(),
  File "/gpfs/mira-home/sarich/spack/lib/spack/spack/spec.py", line 1295, in _expand_virtual_packages
    spec._replace_with(replacement)
  File "/gpfs/mira-home/sarich/spack/lib/spack/spack/spec.py", line 1180, in _replace_with
    del dependent._dependencies[self.name]
KeyError: 'mpi'

My packages.yaml file:

packages:
  mpich@3:
    paths:
      mpich@3%gcc@4.4.7: /soft/compilers/wrappers/gcc
    buildable: False

  netcdf:
    paths:
      netcdf@4.3.3%gcc@4.4.7: /soft/libraries/netcdf/4.3.3-f4.4.1/cnk-gcc/V1R2M2-20150514
    buildable: False

  hdf5:
    paths:
      hdf5@1.8.14%gcc@4.4.7: /soft/libraries/hdf5/1.8.14/cnk-gcc/V1R2M2-20150515
    buildable: False


  all:
    compiler: [gcc@4.4.7]
    providers:
      mpi: [mpich]
      blas: [openblas]

About this issue

  • Original URL
  • State: closed
  • Created 10 years ago
  • Comments: 31 (31 by maintainers)

Most upvoted comments

@BarrySmith: The potential larger issue here is with the way Spack treats external packages. We are trying to support them (which is risky TBH, as so much can go wrong) because you need them on a lot of HPC machines.

When you write:

packages:
  mpich:
    paths:
      mpich@3%gcc@4.4.7: /soft/compilers/wrappers/gcc
    buildable: False

Your’e telling Spack that the MPI installed in /soft/compilers/wrappers/gcc will indeed satisfy the mpich@3%gcc@4.4.7 build spec. This is not true, since /soft/compilers/wrappers/gcc is the soft compiler install directory, and it’s not how mpich would actually install.

I would claim that the layout changes required for soft wrappers are really a different build configuration, as you’re changing the contract of how you use MPI as a dependency.

In Spack, the contract between MPI and other packages is handled with the mpicc variable and friends, which are set up by the various MPI packages. You could tweak the mpich package with a variant, say +anl, that would set things up properly for an install like this. You’d need to mess with setup_dependent_package in the mpich Spack package, and you’d need to have it set the MPI compiler location conditionally to the /soft/compilers/wrappers/gcc/mpicc location when the +anl variant is enabled. Then you could write:

packages:
  mpich:
    paths:
      mpich@3%gcc@4.4.7+anl: /soft/compilers/wrappers/gcc
    buildable: False

And Spack would know about the difference. But if we can avoid dealing with this by using the vendor wrappers that come with the IBM driver on BG/Q, that would be ideal. What do the soft wrappers add? I thought at ANL they just provided some tmp caching for faster builds – something Spack does already.

@BarrySmith: Spack has to support both its own installed versions of MPI and MPI installations at facilities. I don’t think it’s a huge assumption to assume that MPI will be installed the way it was intended to be installed by the vendor, especially when the “vendor” of this particular MPI is ANL.

Also, we’re not precluding wrappers – Spack works fine with the MPI compiler wrappers as installed by MPICH and OpenMPI’s own installation systems. What we don’t work with and what I don’t want to support is nonstandard installation layouts where every facility can invent their own. At that point I don’t think it’s MPICH anymore – it’s something else the facility made that they should really upstream if they intend for it to be used by lots of people.

That said, I realize that HPC is an insular community with a lot of weird stuff, so maybe we need support for soft the same way we support modules (maybe). But in this instance I suspect that the compiler wrapper in /soft/compilers/wrappers/mpicc actually points at some reasonable mpicc someplace else, and I suspect this would work if you just used the real MPI prefix.