spack: Problems with Lmod modulefile names post-PR#3206

We’re using Spack on CentOS 7 systems. We build our things nightly to stay on top of changes.

The recent changes in #3206 changed the names of the directories in which our modules end up. We splice a spack tree into our users environments with a shell script that used to refer to the Core directory with something like this:

_APPS_LMOD_CORE_DIR=${APPS_DIR}/share/spack/lmod/linux-centos7-x86_64/Core

which worked well enough since all of our systems are x86_64.

Now I have two problems (and I’m not even using regular expressions):

  1. the Core modules end up in linux-centos7-haswell but there’s no good way for us to guess the haswell bit, it seems to be the arch of the system on which the system compiler was built.

    I poked around at the gcc command line a bit, but don’t see any info about how it was built

    $ /usr/bin/gcc -dumpmachine
    x86_64-redhat-linux
    $ /usr/bin/gcc -print-multiarch
    
    $
    
  2. We use the system gcc (4.8.5) to build gcc@8.2.0 and use that to build the rest of the things

    Unfortunately, the Core module prepends this bit onto the MODULEPATH:

    $ grep MODULEPATH share/spack/lmod/linux-centos7-haswell/Core/gcc/8.2.0-q5tss7s.lua
    prepend_path("MODULEPATH", "/home/ELIDED/temp/spack/share/spack/lmod/linux-centos7-haswell/gcc/8.2.0")
    

    but that directory doesn’t exist, all the things that we’ve built with the compiler we built with the system compiler end up in linux-centos7-skylake_avx512.

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 31 (31 by maintainers)

Commits related to this issue

Most upvoted comments

I think I’m hitting the same issue using the develop branch (b929774). I git pull’ed develop yesterday for the first time in about a month. After that, I installed a package and noticed that the module and install path is now linux_rhel7_skylake instead of linux_rhel7_x86_64. The system I’m working on has a pre-skylake intel processor.

update:

Adding targets: [x86_64] in packages.yaml under the packages: heading appears to have informed the output of spec. A subsequent install went through without any obvious issues.

packages:
  all:
    ...
    target: [x86_64]
    ...

I know that it’s been marked “off topic” (😈), but I think that @alalazo’s solution for @anne-glerum, which said:

@anne-glerum You can use:

$ spack install -a target=<name>

should have instead said

$ spack uninstall -a target=<name>

although the attached discussion might have made that clear.

Does that make sense to you?

Totally. Thanks!

@odoublewen and I were chatting out of band.

spack arch digs into /proc/cpuinfo (on linux) to figure out what the host architecture is.

Is there some way to ask what the target is for a given compiler, e.g. spack arch %gcc4.8.5 (that doesn’t actually work, but you get the idea)? That would give us programatic way to figure out what to expect Lmod modulefile names to look like and etc.

Now I am uncomfortable marking this and the couple of messages above off-topic as well.

Ah, the ripple effect in action. Apologies. I’ll mark them off topic, but (as usual with GUI’s…), I can’t find the button. Don’t see it under the three dots, or anywhere else obvious… Clue, please?

I can revert that if you think they are relevant.

Not necessary, I just had never seen that before. I’m a firm believer in “There’s no such thing as a dumb question.” and sometimes being declared “off topic” can feel close to that.

It might be friendly to make a note that they being marked off topic and are (or aren’t) worthy of a separate discussion…

Am I correct that there’s still a problem with generating Core module files when we use a newer compiler that gives Spack access to fancier architectures, or is there something that we’re doing wrong in our modulefile configuration?

I think you’re correct - the unit test for lmod modules don’t have any case of multiple architectures and that very likely requires some adjustment to the code.

let me know if you think something is unclear or should be added to the user docs or to the packager docs.

@alalazo – they seem complete, in particular the user docs explain our situation.

I believe that there is a hierarchical structure to the architectures; for those of us who don’t live and breath CPUS, is there a way to print them out that shows that structure (don’t see any variants of --known-targets that does it…)?

@alalazo – sounds good.

I’ll fine tune @odoublewen’s comment though, my biggest problem with the hierarchical modules is that the Core module file adds the wrong directory to MODULEPATH, so that loading the Core module does not make the other modules accessible. (point #2 in summary).

@anne-glerum – Did you clean out all of the stuff you built as darwin-mojave-broadwell before you built the new things as darwin-mojave-x86_64? If the old versions are still hanging around they might be confusing the issue.