spack: spec['mpi'].mpicc doesn't work for MPI in intel-parallel-studio
I failed to build mdtest with Intel Parallel Studio due to “‘Spec’ object has no attribute ‘mpicc’” issue. mdtest is an MPI program to test metadata performance on filesystem.
The command to reproduce this issue and the error message is listed below.
$ spack install mdtest %intel ^intel-parallel-studio+mpi
==> Installing mdtest
==> intel-parallel-studio is externally installed in /lustre/spack/tools/linux-centos7-x86_64/gcc-4.8.5/intel-parallel-studio-cluster.2017.1-4uy7iwkuftkrg5wx2sg5zi6r5355rqbe
==> Using cached archive: /tmp/rpm/spack_source_cache/mdtest/mdtest-1.9.3.tar.gz
==> Warning: Fetching from mirror without a checksum!
This package is normally checked out from a version control system, but it has been archived on a spack mirror.
This means we cannot know a checksum for the tarball in advance. Be sure that your connection to this mirror is secure!.
==> Already staged mdtest-1.9.3-zwdgfm4vkhmcmqxhla4xhagbywnz7eqm in /home/rpm/spack/var/spack/stage/mdtest-1.9.3-zwdgfm4vkhmcmqxhla4xhagbywnz7eqm
==> No patches needed for mdtest
==> Building mdtest [Package]
==> Executing phase : 'install'
==> Error: AttributeError: 'Spec' object has no attribute 'mpicc'
/home/rpm/spack/var/spack/repos/builtin/packages/mdtest/package.py:40, in install:
39 def install(self, spec, prefix):
>> 40 filter_file('$(CC.$(OS))', spec['mpi'].mpicc, 'Makefile', string=True)
41 make('mdtest')
42 mkdirp(prefix.bin)
43 install('mdtest', prefix.bin)
See build log for details:
/tmp/rpm/spack_stage/spack-stage/spack-stage-XbDO_t/mdtest/spack-build.out
mdtest’s build process is organized in Makefiile shipped with it. What Spack does is to replace the “$(CC.$(OS))'” string with the selected MPI’s mpicc specified in spec['mpi'].mpicc, then run make. The embarrassing thing is that I tested it under %gcc ^onmpi only when I submit simul package, leaving %intel ^intel-parallel-studio untested.
According to Spack’s doc http://spack.readthedocs.io/en/latest/packaging_guide.html?highlight=mpic , spec['mpi'].mpicc should get the MPICC compiler for intel-parallel-studio. Please correct me if I use spec['mpi'] in the wrong way.
Link to mdtest package https://github.com/LLNL/spack/blob/develop/var/spack/repos/builtin/packages/mdtest/package.py .
About this issue
- Original URL
- State: closed
- Created 10 years ago
- Comments: 16 (16 by maintainers)
@weijianwen i would look at
mpichand see how everything is done there and what is missing inintel-parallel-studiopackage. Then just copy-paste and add some ifs for intel/gcc compilers.yes, i think this is how it should be done. This is also the same for things like single-threaded Atlas and multithreaded Atlas – they can probably be stored within a single prefix, but we don’t want to do it because we want a single package be specific, in that case have
+openmpflag and thereby return correct libraries.In your case you would have two installations of intel-mpi, the one with intel will return intel MPI wrappers, the one with GCC will return GCC wrappers.
In spack, that’s the best way to handle this, I think. We don’t assume that a package can dynamically swap compiler backends (because most don’t), so we install two versions in cases like this.
Is the wrapper in Intel MPI significantly different from the wrapper in mpich? The important part for spack builds is that the wrapper accepts MPICH_CC, MPICH_CXX, etc. variables. If it doesn’t, then setup_dependent_package will need to figure out how best to get the MPI wrappers to use the spack wrappers.
The
intel-parallel-studiopackage needs to definempiccand related attributes in asetup_dependent_packagemethod. Parallel studio is based on MPICH, so you can probably take thesetup_dependent_packagefrommpichto make this work. You probably want to set the attributes and environment variables conditionally, depending on whether parallel studio is built with+mpi.This brings up two things we could think about.
First, with @alalazo’s spec interface and the mpicc concention, virtual dependencies are starting to have contracts associated with them in spack. e.g., “if I provide mpi, I will have an mpicc attribute”. Seems like something that could be declared and checked statically.
Second, we’d need a place to put that contract. Having a package placeholder for a vdep is something we’ve talked about but haven’t actually merged. A placeholder package for
mpi,blas, etc. seem like good places to put sanity checks (and/or something more declarative), to enforce contracts like this one As new MPI/whatever implementations as they are added.Thoughts?