spack: spack load: don't load multiple variants of same packages and don't load packages that depend on different versions of already-loaded packages
Assume I have installed two versions of OpenMPI with matching versions of Boost by running the following:
spack install boost +mpi ^openmpi@1.10.3+slurm+verbs+pmi
spack install boost +mpi ^openmpi@1.8.8+slurm+verbs+pmi
Now if a user does the obvious thing
spack load openmpi
spack load boost
they end up with all matching modules loaded:
Currently Loaded Modulefiles:
1) spack 4) boost-1.62.0-gcc-4.9.4-bb3jpvq
2) openmpi-1.8.8-gcc-4.9.4-gyda6cg 5) boost-1.62.0-gcc-4.9.4-5kz43yc
3) openmpi-1.10.3-gcc-4.9.4-mnqdpir
This is of course never the intended action. To get the correct set of modules, a user will need to run commands like these instead:
spack load openmpi@1.10.3
spack load boost ^openmpi@1.10.3
spack find --deps can tell me what dependencies a package is linked to, but constructing dependency specs is not something you’d want to bother a novel user with if you can avoid it (also, they shouldn’t need to care whether they’re using OpenMPI 1.10.1 or 1.10.3). I think this could be made easier on the spack load level:
- Since loading the same package in multiple versions or with multiple different dependencies never makes sense, first of all, all spack-generated module files should declare conflicts against all modules with the same name.
- It should be possible to specify a default version that will be loaded if no version number is specified (e.g.
spack load openmpicould translate intospack load openmpi@1.10.3orspack load openmpi@1.8.8depending on configuration). Similar to howspack install openmpiwill always install the most recent version, except it should be configurable to a different version. Maybe etc/spack/packages.yaml can already be used for this purpose. - If a dependency module (e.g. openmpi@1.10.3) is already loaded, then loading the dependent without further dependency specification (e.g.
spack load boost) should automatically load the version of the dependent that is linked against the loaded dependency (in this example, boost ^openmpi@1.10.3)
About this issue
- Original URL
- State: closed
- Created 10 years ago
- Comments: 24 (24 by maintainers)
Commits related to this issue
- spack load exits with 1 if module does not exist or is not installed fixes #2215 fixes #2570 fixes #6676 fixes #7281 closes #3827 This PR reverts the use of `spack module loads` in favor of `spack m... — committed to epfl-scitas/spack by alalazo 8 years ago
- spack load exits with 1 if module does not exist or is not installed fixes #2215 fixes #2570 fixes #6676 fixes #7281 closes #3827 This PR reverts the use of `spack module loads` in favor of `spack m... — committed to epfl-scitas/spack by alalazo 8 years ago
- spack load exits with 1 if module does not exist or is not installed fixes #2215 fixes #2570 fixes #6676 fixes #7281 closes #3827 This PR reverts the use of `spack module loads` in favor of `spack m... — committed to epfl-scitas/spack by alalazo 8 years ago
- spack load exits with 1 if module does not exist or is not installed fixes #2215 fixes #2570 fixes #6676 fixes #7281 closes #3827 This PR reverts the use of `spack module loads` in favor of `spack m... — committed to epfl-scitas/spack by alalazo 8 years ago
- spack load exits with 1 if module does not exist or is not installed fixes #2215 fixes #2570 fixes #6676 fixes #7281 closes #3827 This PR reverts the use of `spack module loads` in favor of `spack m... — committed to spack/spack by alalazo 8 years ago
@alalazo, sorry for the late replies…
This is mainly a concern when you need multiple libraries (e.g. Scalapack and FFTW) that depend on a common library (e.g. MPI) or when you both need a library (e.g. Boost) and one of its dependencies (e.g. Python). Assuming I have the following packages installed:
boost @1.62 +python ^python@2.7boost @1.62 +python ^python@3.4boost @1.62 +python ^python@3.5boost @1.65 +python ^python@2.7boost @1.65 +python ^python@3.5boost @1.65 +python ^python@3.6Then I’d like declare that each boost module conflicts with all python modules except for the one that it was built against. If I don’t have these conflicts declared, then users who accidentally load
boost @1.62 +python ^python@2.7andpython@3.6and try to use Boost.Python in their own code get linker errors. If this was caught by the module system, the error would be clearer to the user.Similarly, assuming I have the following packages installed:
netlib-scalapack ^openmpi@3netlib-scalapack ^openmpi@2fftw ^openmpi@3fftw ^openmpi@2Then I’d like to see an error message if
fftw ^openmpi@3andnetlib-scalapack ^openmpi@2are loaded together.I guess this mostly affects Python and MPI as these don’t allow you to use multiple versions of the same library from within the same process due to their complex internal state.
You may be right. Adding python and mpi to the hierarchy should suffice. I’m currently trying that.