spack: Installation issue: `llvm` main branch fails to build via Spack

Steps to reproduce the issue

$ spack install llvm@main+omp_debug+cuda cuda_arch=80
[...]
[+] /p/scratch/cjzam11/kitayama1/projects/spack/opt/spack/linux-centos8-zen2/gcc-10.3.0/py-setuptools-50.3.2-daadfdnmkswqtoa5thp5c2zzezbk6nxp
[+] /p/scratch/cjzam11/kitayama1/projects/spack/opt/spack/linux-centos8-zen2/gcc-10.3.0/z3-4.8.9-lapegzcygwpjonwb34dhmjwzz5jvh4mp
==> Installing llvm-main-oi5oojnysupwxabpy6uyo6cziysntbrg
==> No binary for llvm-main-oi5oojnysupwxabpy6uyo6cziysntbrg found: installing from source
==> Warning: Missing a source id for llvm@main
==> Warning: Missing a source id for cuda@11.0.221
==> No patches needed for llvm
==> llvm: Executing phase: 'cmake'
==> llvm: Executing phase: 'build'
==> llvm: Executing phase: 'install'
==> Error: ProcessError: Command exited with status 2:
    'make' '-j16'

18 errors found in build log:
     52500    [  9%] Generating amdgcn_smid.gfx906.bc
     52501    [  9%] Generating task.gfx900.bc
     52502    [ 10%] Generating amdgcn_smid.gfx900.bc
     52503    [ 10%] Generating task.gfx803.bc
     52504    [ 10%] Generating task.gfx801.bc
     52505    /bin/sh: CLANG_TOOL-NOTFOUND: command not found
  >> 52506    make[2]: *** [libomptarget/deviceRTLs/amdgcn/CMakeFiles/libomptar
              get-amdgcn-gfx906.dir/build.make:300: libomptarget/deviceRTLs/amd
              gcn/task.gfx906.bc] Error 127
     52507    make[2]: *** Waiting for unfinished jobs....
     52508    /bin/sh: CLANG_TOOL-NOTFOUND: command not found
  >> 52509    make[2]: *** [libomptarget/deviceRTLs/amdgcn/CMakeFiles/libomptar
              get-amdgcn-gfx906.dir/build.make:118: libomptarget/deviceRTLs/amd
              gcn/amdgcn_smid.gfx906.bc] Error 127
     52510    /bin/sh: CLANG_TOOL-NOTFOUND: command not found
  >> 52511    make[1]: *** [CMakeFiles/Makefile2:1035: libomptarget/deviceRTLs/
              amdgcn/CMakeFiles/libomptarget-amdgcn-gfx906.dir/all] Error 2
  >> 52512    make[2]: *** [libomptarget/deviceRTLs/amdgcn/CMakeFiles/libomptar
              get-amdgcn-gfx900.dir/build.make:300: libomptarget/deviceRTLs/amd
              gcn/task.gfx900.bc] Error 127
     52513    make[1]: *** Waiting for unfinished jobs....
     52514    make[2]: *** Waiting for unfinished jobs....
     52515    /bin/sh: CLANG_TOOL-NOTFOUND: command not found
  >> 52516    make[2]: *** [libomptarget/deviceRTLs/amdgcn/CMakeFiles/libomptar
              get-amdgcn-gfx803.dir/build.make:300: libomptarget/deviceRTLs/amd
              gcn/task.gfx803.bc] Error 127
     52517    /bin/sh: CLANG_TOOL-NOTFOUND: command not found
  >> 52518    make[1]: *** [CMakeFiles/Makefile2:1087: libomptarget/deviceRTLs/
              amdgcn/CMakeFiles/libomptarget-amdgcn-gfx803.dir/all] Error 2
  >> 52519    make[2]: *** [libomptarget/deviceRTLs/amdgcn/CMakeFiles/libomptar
              get-amdgcn-gfx900.dir/build.make:118: libomptarget/deviceRTLs/amd
              gcn/amdgcn_smid.gfx900.bc] Error 127
  >> 52520    make[1]: *** [CMakeFiles/Makefile2:1061: libomptarget/deviceRTLs/
              amdgcn/CMakeFiles/libomptarget-amdgcn-gfx900.dir/all] Error 2
     52521    [ 11%] Generating task.gfx701.bc
     52522    /bin/sh: CLANG_TOOL-NOTFOUND: command not found
  >> 52523    make[2]: *** [libomptarget/deviceRTLs/amdgcn/CMakeFiles/libomptar
              get-amdgcn-gfx801.dir/build.make:300: libomptarget/deviceRTLs/amd
              gcn/task.gfx801.bc] Error 127
     52524    make[2]: *** Waiting for unfinished jobs....
     52525    /bin/sh: CLANG_TOOL-NOTFOUND: command not found
  >> 52526    make[2]: *** [libomptarget/deviceRTLs/amdgcn/CMakeFiles/libomptar
              get-amdgcn-gfx701.dir/build.make:300: libomptarget/deviceRTLs/amd
              gcn/task.gfx701.bc] Error 127
     52527    make[2]: *** Waiting for unfinished jobs....
     52528    [ 11%] Generating amdgcn_smid.gfx801.bc
     52529    [ 11%] Generating amdgcn_smid.gfx701.bc
     52530    [ 12%] Generating amdgcn_locks.gfx801.bc
     52531    [ 14%] Generating amdgcn_locks.gfx701.bc
     52532    [ 14%] Generating target_impl.gfx801.bc
     52533    [ 14%] Generating target_impl.gfx701.bc
     52534    /bin/sh: CLANG_TOOL-NOTFOUND: command not found
  >> 52535    make[2]: *** [libomptarget/deviceRTLs/amdgcn/CMakeFiles/libomptar
              get-amdgcn-gfx801.dir/build.make:118: libomptarget/deviceRTLs/amd
              gcn/amdgcn_smid.gfx801.bc] Error 127
     52536    /bin/sh: CLANG_TOOL-NOTFOUND: command not found
  >> 52537    make[2]: *** [libomptarget/deviceRTLs/amdgcn/CMakeFiles/libomptar
              get-amdgcn-gfx701.dir/build.make:118: libomptarget/deviceRTLs/amd
              gcn/amdgcn_smid.gfx701.bc] Error 127
     52538    /bin/sh: CLANG_TOOL-NOTFOUND: command not found
  >> 52539    make[2]: *** [libomptarget/deviceRTLs/amdgcn/CMakeFiles/libomptar
              get-amdgcn-gfx801.dir/build.make:105: libomptarget/deviceRTLs/amd
              gcn/amdgcn_locks.gfx801.bc] Error 127
     52540    /bin/sh: CLANG_TOOL-NOTFOUND: command not found
  >> 52541    make[2]: *** [libomptarget/deviceRTLs/amdgcn/CMakeFiles/libomptar
              get-amdgcn-gfx701.dir/build.make:105: libomptarget/deviceRTLs/amd
              gcn/amdgcn_locks.gfx701.bc] Error 127
     52542    /bin/sh: CLANG_TOOL-NOTFOUND: command not found
     52543    /bin/sh: CLANG_TOOL-NOTFOUND: command not found
  >> 52544    make[2]: *** [libomptarget/deviceRTLs/amdgcn/CMakeFiles/libomptar
              get-amdgcn-gfx701.dir/build.make:287: libomptarget/deviceRTLs/amd
              gcn/target_impl.gfx701.bc] Error 127
  >> 52545    make[2]: *** [libomptarget/deviceRTLs/amdgcn/CMakeFiles/libomptar
              get-amdgcn-gfx801.dir/build.make:287: libomptarget/deviceRTLs/amd
              gcn/target_impl.gfx801.bc] Error 127
  >> 52546    make[1]: *** [CMakeFiles/Makefile2:1139: libomptarget/deviceRTLs/
              amdgcn/CMakeFiles/libomptarget-amdgcn-gfx701.dir/all] Error 2
  >> 52547    make[1]: *** [CMakeFiles/Makefile2:1113: libomptarget/deviceRTLs/
              amdgcn/CMakeFiles/libomptarget-amdgcn-gfx801.dir/all] Error 2
     52548    [ 14%] Built target libomp-needed-headers
     52549    [ 15%] Linking CXX shared library ../../libomptarget.rtl.x86_64.s
              o
     52550    [ 15%] Built target omptarget.rtl.x86_64
     52551    [ 16%] Linking CXX shared library ../libomptarget.so
     52552    [ 16%] Linking CXX shared library ../../libomptarget.rtl.cuda.so
     52553    [ 16%] Built target omptarget.rtl.cuda

Information on your system

Additional information

  • spack-build-out.txt
  • spack-build-env.txt

General information

  • I have run spack debug report and reported the version of Spack/Python/Platform
  • I have run spack maintainers <name-of-the-package> and @mentioned any maintainers
  • I have uploaded the build log and environment files
  • I have searched the issues of this repo and believe this is not a duplicate

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 30 (29 by maintainers)

Most upvoted comments

@naromero77 , as the llvm package co-maintainer, is the main branch build completely out of support scope? while I think it is useful to address the issues found in main builds. Just curious to know your thoughts.

The Spack packages are supported by the community. Neither @trws or I have specific funding earmarked to support the LLVM Spack package. We do this out of our “spare time.” If someone else from the community initiate PR, we can review it. However, IMO supporting the main branch would be difficult. Most of the Spack packages only support officially releases. So, we have no plans to support main in the Spack package. @trws should jump in and correct anything I say.

I think in upstream main, the AMDGPU target can not be built reliably at this moment.

The problem is the following code in the llvm installation file:

if "+all_targets" not in spec:  # all is default on cmake

            targets = ["NVPTX", "AMDGPU"]
            if spec.version < Version("3.9.0"):
                # Starting in 3.9.0 CppBackend is no longer a target (see
                # LLVM_ALL_TARGETS in llvm's top-level CMakeLists.txt for
                # the complete list of targets)
                targets.append("CppBackend")
            
           if spec.target.family in ("XXX"):
                targets.append("XXX")
          elif ...
          cmake_args.append(define("LLVM_TARGETS_TO_BUILD", targets))

This makes that independently of your spec, when not all targets are requested (default spack option but not cmake one), both NVPTX, AMDGPU will be still compiled. And unfortunately AMDGPU compilation always fails (at least for me). Therefore, I need to remove it from this list in any of my spack installations before attempting the llvm installation.

Both NVPTX, AMDGPU, CppBackend, should be explicit options somehow. I don’t see requesting the specs the user wants to compile and if needed providing a default spec corresponding to the machine spec only.

Edit: I will publish this as separate issue anyway as is only part of the problem reported here.

@naromero77 , as the llvm package co-maintainer, is the main branch build completely out of support scope? while I think it is useful to address the issues found in main builds. Just curious to know your thoughts.

The Spack packages are supported by the community. Neither @trws or I have specific funding earmarked to support the LLVM Spack package. We do this out of our “spare time.” If someone else from the community initiate PR, we can review it. However, IMO supporting the main branch would be difficult. Most of the Spack packages only support officially releases. So, we have no plans to support main in the Spack package. @trws should jump in and correct anything I say.

That’s right. Neither of us are “official support” for this. I think we both want it to be as usable and useful as possible, main branch included, but only have so much bandwidth for this as a side project. If there is an issue with the spack package that does not appear when building directly, that’s what we’re here to fix.