spack: "Installation issue: openblas: default blas/lapack 'openblas' can't build on RHEL6 with gcc@7.2.0 built by Spack
snl-atdm-issue
Steps to reproduce the issue
$ ./atdm-spack/spack/bin/spack install openblas %gcc@7.2.0
with build errors starting with:
==> Installing openblas
==> Searching for binary cache of openblas
==> Finding buildcaches in /ascldap/users/rabartl/Spack.base3/atdm-spack/atdm-spack-tarball-mirror/build_cache
==> No binary for openblas found: installing from source
==> Using cached archive: /home/rabartl/Spack.base3/atdm-spack/spack/var/spack/cache/openblas/openblas-0.3.5.tar.gz
==> Staging archive: /home/rabartl/Spack.base3/atdm-spack/spack/var/spack/stage/openblas-0.3.5-ase7rloe2w5cvr7egf4fglyx5kw2egrl/v0.3.5.tar.gz
==> Created stage in /home/rabartl/Spack.base3/atdm-spack/spack/var/spack/stage/openblas-0.3.5-ase7rloe2w5cvr7egf4fglyx5kw2egrl
==> No patches needed for openblas
==> Building openblas [MakefilePackage]
==> Executing phase: 'edit'
==> Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
'make' '-j32' 'CC=/home/rabartl/Spack.base3/atdm-spack/spack/lib/spack/env/gcc/gcc' 'FC=/home/rabartl/Spack.base3/atdm-spack/spack/lib/spack/env/gcc/gfortran' 'MAKE_NB_JOBS=0' 'USE_OPENMP=0' 'USE_THREAD=0' 'libs' 'netlib' 'shared'
577 errors found in build log:
1348 /home/rabartl/Spack.base3/atdm-spack/spack/lib/spack/env/gcc/gcc -O2 -DMAX_STACK_ALLOC=2048 -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_AVX512 -DNO_WARMUP -DMAX_CPU_NUMBER=32 -DMAX_PARALLEL_NUMBER=1
-DVERSION=\"0.3.5\" -mavx2 -DASMNAME=dtrmm_kernel_LN -DASMFNAME=dtrmm_kernel_LN_ -DNAME=dtrmm_kernel_LN_ -DCNAME=dtrmm_kernel_LN -DCHAR_NAME=\"dtrmm_kernel_LN_\" -DCHAR_CNAME=\"dtrmm_kernel_LN\" -D
NO_AFFINITY -I.. -DDOUBLE -UCOMPLEX -c -DTRMMKERNEL -DDOUBLE -UCOMPLEX -DLEFT -UTRANSA ../kernel/x86_64/dtrmm_kernel_4x8_haswell.c -o dtrmm_kernel_LN.o
1349 /home/rabartl/Spack.base3/atdm-spack/spack/lib/spack/env/gcc/gcc -O2 -DMAX_STACK_ALLOC=2048 -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_AVX512 -DNO_WARMUP -DMAX_CPU_NUMBER=32 -DMAX_PARALLEL_NUMBER=1
-DVERSION=\"0.3.5\" -mavx2 -DASMNAME=dtrmm_kernel_LT -DASMFNAME=dtrmm_kernel_LT_ -DNAME=dtrmm_kernel_LT_ -DCNAME=dtrmm_kernel_LT -DCHAR_NAME=\"dtrmm_kernel_LT_\" -DCHAR_CNAME=\"dtrmm_kernel_LT\" -D
NO_AFFINITY -I.. -DDOUBLE -UCOMPLEX -c -DTRMMKERNEL -DDOUBLE -UCOMPLEX -DLEFT -DTRANSA ../kernel/x86_64/dtrmm_kernel_4x8_haswell.c -o dtrmm_kernel_LT.o
1350 /home/rabartl/Spack.base3/atdm-spack/spack/lib/spack/env/gcc/gcc -O2 -DMAX_STACK_ALLOC=2048 -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_AVX512 -DNO_WARMUP -DMAX_CPU_NUMBER=32 -DMAX_PARALLEL_NUMBER=1
-DVERSION=\"0.3.5\" -mavx2 -DASMNAME=dtrmm_kernel_RN -DASMFNAME=dtrmm_kernel_RN_ -DNAME=dtrmm_kernel_RN_ -DCNAME=dtrmm_kernel_RN -DCHAR_NAME=\"dtrmm_kernel_RN_\" -DCHAR_CNAME=\"dtrmm_kernel_RN\" -D
NO_AFFINITY -I.. -DDOUBLE -UCOMPLEX -c -DTRMMKERNEL -DDOUBLE -UCOMPLEX -ULEFT -UTRANSA ../kernel/x86_64/dtrmm_kernel_4x8_haswell.c -o dtrmm_kernel_RN.o
1351 /home/rabartl/Spack.base3/atdm-spack/spack/lib/spack/env/gcc/gcc -O2 -DMAX_STACK_ALLOC=2048 -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_AVX512 -DNO_WARMUP -DMAX_CPU_NUMBER=32 -DMAX_PARALLEL_NUMBER=1
-DVERSION=\"0.3.5\" -mavx2 -DASMNAME=dtrmm_kernel_RT -DASMFNAME=dtrmm_kernel_RT_ -DNAME=dtrmm_kernel_RT_ -DCNAME=dtrmm_kernel_RT -DCHAR_NAME=\"dtrmm_kernel_RT_\" -DCHAR_CNAME=\"dtrmm_kernel_RT\" -D
NO_AFFINITY -I.. -DDOUBLE -UCOMPLEX -c -DTRMMKERNEL -DDOUBLE -UCOMPLEX -ULEFT -DTRANSA ../kernel/x86_64/dtrmm_kernel_4x8_haswell.c -o dtrmm_kernel_RT.o
1352 /home/rabartl/Spack.base3/atdm-spack/spack/lib/spack/env/gcc/gcc -c -O2 -DMAX_STACK_ALLOC=2048 -Wall -m64 -DF_INTERFACE_GFORT -fPIC -DNO_AVX512 -DNO_WARMUP -DMAX_CPU_NUMBER=32 -DMAX_PARALLEL_NUMBER
=1 -DVERSION=\"0.3.5\" -mavx2 -DASMNAME=dtrsm_kernel_LN -DASMFNAME=dtrsm_kernel_LN_ -DNAME=dtrsm_kernel_LN_ -DCNAME=dtrsm_kernel_LN -DCHAR_NAME=\"dtrsm_kernel_LN_\" -DCHAR_CNAME=\"dtrsm_kernel_LN\"
-DNO_AFFINITY -I.. -DDOUBLE -UCOMPLEX -DTRSMKERNEL -UCOMPLEX -DDOUBLE -DUPPER -DLN -UCONJ ../kernel/x86_64/../generic/trsm_kernel_LN.c -o dtrsm_kernel_LN.o
1353 ../kernel/x86_64/dgemm_kernel_4x8_haswell.S: Assembler messages:
>> 1354 ../kernel/x86_64/dgemm_kernel_4x8_haswell.S:1759: Error: no such instruction: `vpermpd $ 0xb1,%ymm0,%ymm0'
>> 1355 ../kernel/x86_64/dgemm_kernel_4x8_haswell.S:1759: Error: no such instruction: `vpermpd $ 0x1b,%ymm0,%ymm0'
The STDOUT from spack is given in:
The the build log file:
atdm-spack/spack/var/spack/stage/openblas-0.3.5-ase7rloe2w5cvr7egf4fglyx5kw2egrl/OpenBLAS-0.3.5/spack-build.out
is:
Platform and user environment
Please report your OS here:
$ uname -a
Linux crf450.srn.sandia.gov 2.6.32-696.30.1.el6.x86_64 #1 SMP Fri May 18 11:50:44 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -d
Description: Red Hat Enterprise Linux Workstation release 6.9 (Santiago)
Here are the compilers (where gcc@7.2.0 was built by spack itself with gcc@4.4.7 on the system).
Additional information
Sometimes the issue benefits from additional details. In these cases there are a few things we can suggest doing. First of all, you can post the full output of:
./atdm-spack/spack/bin/spack spec --install-status openblas %gcc@7.2.0
Input spec
--------------------------------
- openblas%gcc@7.2.0
Concretized
--------------------------------
==> Error: String contains no specs:
to show people whether Spack installed a faulty software or if it was not able to build it at all.
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 25 (8 by maintainers)
Commits related to this issue
- Fix build error in OpenBLAS on rhel6 Use a newer assembler to allow avx2 instructions. This is the same error that was [reported here](https://github.com/JuliaLang/julia/issues/7653) and [here](https... — committed to sethrj/spack by sethrj 7 years ago
- Revert fix for #12235 — committed to sethrj/spack by sethrj 7 years ago
@sethrj, thanks for the tip on
packages: all: target: [x86_64]. I was having to putarch=x86_64on all of the specs. Spack is large and complex.@bartlettroscoe From what I understand, gcc is looking for gas on the PATH, unless you compile it explicitly with binutils.
It’s possible the build will succeed if you add a depends_on(“binutils”) in openblas. Openblas shouldn’t depend on binutils, I bet it will compile fine with intel compiler or llvm.
The real problem is that “gcc -mavx2” will tell the build system that you can compile avx2 instructions, but it doesn’t check that the assembler it uses will support the instruction set.
The correct fix, should be for spack to provide an “as” wrapper as part of the “compiler” when providing a gcc toolchain. My workaround is to load the binutils package as a module from the compiler definition.