spack: Installation issue: intel: Can't build Intel lmod file from module-based Intel install
Steps to reproduce the issue
$ spack install intel@2021.3.0
==> intel@2021.3.0 : has external module in ['comp/gcc/10.1.0', 'comp/intel/2021.3.0']
==> Warning: cannot perform the requested write operation on module files [Trying to source non-existing file: /usr/local/intel/oneapi/2021/compiler/2021.3.0/linux/compilers_and_libraries/linux/bin/compilervars.sh]
==> Warning: cannot perform the requested write operation on module files [Trying to source non-existing file: /usr/local/intel/oneapi/2021/compiler/2021.3.0/linux/compilers_and_libraries/linux/bin/compilervars.sh]
[+] /usr/local/intel/oneapi/2021/compiler/2021.3.0/linux (external intel-2021.3.0-hdgcisraeacu42xktiqns4w5raaqizir)
Information on your system
- Spack: 0.17.1-1070-5208fcf1ea
- Python: 3.9.9
- Platform: linux-sles12-skylake_avx512
- Concretizer: clingo
Additional information
This is an odd one. I’m trying to use Intel 2021.3 on a cluster where Intel 2021.3 has already been installed by the sysadmin. Normally, I’d do:
ml comp/gcc/10.1.0 comp/intel/2021.3.0
and I’d get it.
So with spack, in my compilers.yaml I have (truncated for this issue, and gcc 4.8 is the “base” gcc on this system):
compilers:
- compiler:
spec: gcc@4.8
paths:
cc: /usr/bin/gcc
cxx: /usr/bin/g++
f77: /usr/bin/gfortran
fc: /usr/bin/gfortran
flags: {}
operating_system: sles12
target: x86_64
modules: []
environment: {}
extra_rpaths: []
- compiler:
spec: intel@2021.3.0
paths:
cc: /usr/local/intel/oneapi/2021/compiler/2021.3.0/linux/bin/intel64/icc
cxx: /usr/local/intel/oneapi/2021/compiler/2021.3.0/linux/bin/intel64/icpc
f77: /usr/local/intel/oneapi/2021/compiler/2021.3.0/linux/bin/intel64/ifort
fc: /usr/local/intel/oneapi/2021/compiler/2021.3.0/linux/bin/intel64/ifort
flags: {}
operating_system: sles12
target: x86_64
modules:
- comp/gcc/10.1.0
- comp/intel/2021.3.0
environment: {}
extra_rpaths: []
I have a packages.yaml:
packages:
...
intel:
buildable: false
externals:
- spec: intel@2021.3.0%gcc@4.8
modules:
- comp/gcc/10.1.0
- comp/intel/2021.3.0
...
And when I try to install, spack installs, but then fails at the Lua module creation phase. As seen above, it’s looking for a compilervars.sh file that doesn’t exist.
Now, as a test, I essentially duplicated all the above with intel-oneapi-compilers as I didn’t quite know what 2021.3 actually is in spack speak. When i do that I get:
$ spack install intel-oneapi-compilers@2021.3.0
==> intel-oneapi-compilers@2021.3.0 : has external module in ['comp/gcc/10.1.0', 'comp/intel/2021.3.0']
==> Warning: cannot perform the requested write operation on module files [Trying to source non-existing file: /usr/local/intel/oneapi/2021/compiler/2021.3.0/linux/compiler/2021.3.0/env/vars.sh]
==> Warning: cannot perform the requested write operation on module files [Trying to source non-existing file: /usr/local/intel/oneapi/2021/compiler/2021.3.0/linux/compiler/2021.3.0/env/vars.sh]
[+] /usr/local/intel/oneapi/2021/compiler/2021.3.0/linux (external intel-oneapi-compilers-2021.3.0-kucwu2giszpwh4daajsxfq5zabcz7f2p)
Now this is slightly better as there is a vars.sh file in the install on my system, but it’s at:
/usr/local/intel/oneapi/2021/compiler/2021.3.0/env/vars.sh
not:
/usr/local/intel/oneapi/2021/compiler/2021.3.0/linux/compiler/2021.3.0/env/vars.sh
Is there something I need to add to modules.yaml to tell it where vars.sh lives?
Maintainers: @rscohn2
General information
- I have run
spack debug reportand reported the version of Spack/Python/Platform - I have run
spack maintainers <name-of-the-package>and @mentioned any maintainers - I have uploaded the build log and environment files
- I have searched the issues of this repo and believe this is not a duplicate
About this issue
- Original URL
- State: open
- Created 5 years ago
- Comments: 21 (21 by maintainers)
@kgerheiser: There was some discussion here: https://github.com/spack/spack/pull/28066. It would be great if you can do it.
Hey @mathomp4 , I was looking into a similar issue because I wanted to implement a
spack external find intel-oneapi-mpi. There are two related issues going on, but basically the information fromsetup_run_environmentis also put into your modules, which includes sourcing some files. Spack isn’t great at generating meta modules.First, just kind of reiterating was @rscohn2 said for your test case. Aside from
modules.yaml,setup_run_environmentalso populates your modules like this:https://github.com/spack/spack/blob/93377942d17d356aaae793c5d665a3e48ab7649e/lib/spack/spack/build_systems/oneapi.py#L97-L98
Where component path is:
https://github.com/spack/spack/blob/93377942d17d356aaae793c5d665a3e48ab7649e/lib/spack/spack/build_systems/oneapi.py#L39-L41
And your prefix is:
/usr/local/intel/oneapi/2021/compiler/2021.3.0/linuxI’m guessing the prefix is formed from truncating the
/binwhere the compiler is located. I don’t know the exact mechanism.Now, for your original issue, you have to look at the
intelpackage (different than intel-oneapi-compilers). Thesetup_run_environmentsourcesbin/compilevars.shhttps://github.com/spack/spack/blob/93377942d17d356aaae793c5d665a3e48ab7649e/lib/spack/spack/build_systems/intel.py#L643