spack: openblas@0.3.24 target=neoverse_n1: kernel/arm64/sgemm_kernel_sve_v2x8.S:873: Error: selected processor does not support `dup z7.s,w18'

Steps to reproduce the issue

openblas@0.3.24 fails to build for target=neoverse_n1 using:

  • spack@develop (72ed8711a74ccf8c4c15325ae8d5bd469e3adcb1 from Fri Oct 6 18:42:47 2023 +0200)
  • Ubuntu 20.04, ARM Neoverse N1
  • GCC 11.4.0
Concretization
 -   openblas@0.3.24%gcc@11.4.0~bignuma~consistent_fpcsr+fortran~ilp64+locking+pic+shared build_system=makefile symbol_suffix=none threads=none arch=linux-ubuntu20.04-neoverse_n1
 -       ^perl@5.38.0%gcc@11.4.0+cpanm+opcode+open+shared+threads build_system=generic patches=714e4d1 arch=linux-ubuntu20.04-neoverse_n1
[+]          ^berkeley-db@18.1.40%gcc@11.4.0+cxx~docs+stl build_system=autotools patches=26090f4,b231fcc arch=linux-ubuntu20.04-neoverse_n1
[+]              ^gnuconfig@2022-09-17%gcc@11.4.0 build_system=generic arch=linux-ubuntu20.04-neoverse_n1
 -           ^bzip2@1.0.8%gcc@11.4.0~debug~pic+shared build_system=generic arch=linux-ubuntu20.04-neoverse_n1
 -               ^diffutils@3.9%gcc@11.4.0 build_system=autotools arch=linux-ubuntu20.04-neoverse_n1
[+]                  ^libiconv@1.17%gcc@11.4.0 build_system=autotools libs=shared,static arch=linux-ubuntu20.04-neoverse_n1
 -           ^gdbm@1.23%gcc@11.4.0 build_system=autotools arch=linux-ubuntu20.04-neoverse_n1
 -               ^readline@8.2%gcc@11.4.0 build_system=autotools patches=bbf97f1 arch=linux-ubuntu20.04-neoverse_n1
 -                   ^ncurses@6.4%gcc@11.4.0~symlinks+termlib abi=none build_system=autotools arch=linux-ubuntu20.04-neoverse_n1
[+]                      ^pkgconf@1.9.5%gcc@11.4.0 build_system=autotools arch=linux-ubuntu20.04-neoverse_n1
[+]          ^zlib-ng@2.1.3%gcc@11.4.0+compat+opt build_system=autotools patches=299b958,ae9077a,b692621 arch=linux-ubuntu20.04-neoverse_n1

Error:

$> spack install openblas@0.3.24
...
==> Installing openblas-0.3.24-aw2blghbqnzq4b5cxu6u2ztv5s4dbyhg [12/12]
==> No binary for openblas-0.3.24-aw2blghbqnzq4b5cxu6u2ztv5s4dbyhg found: installing from source
==> Fetching https://github.com/OpenMathLib/OpenBLAS/releases/download/v0.3.24/OpenBLAS-0.3.24.tar.gz
==> No patches needed for openblas
==> openblas: Executing phase: 'edit'
==> openblas: Executing phase: 'build'
...
cc1: warning: switch '-mcpu=neoverse-n1' conflicts with '-march=armv8.4-a+sve' switch
../kernel/arm64/sgemm_kernel_sve_v2x8.S: Assembler messages:
../kernel/arm64/sgemm_kernel_sve_v2x8.S:873: Error: selected processor does not support `dup z7.s,w18'
../kernel/arm64/sgemm_kernel_sve_v2x8.S:874: Error: selected processor does not support `cntw x19'
../kernel/arm64/sgemm_kernel_sve_v2x8.S:878: Error: selected processor does not support `ptrue p0.s'
../kernel/arm64/sgemm_kernel_sve_v2x8.S:914: Error: selected processor does not support `dup z16.s,#0'
../kernel/arm64/sgemm_kernel_sve_v2x8.S:914: Error: selected processor does not support `dup z17.s,#0'
../kernel/arm64/sgemm_kernel_sve_v2x8.S:914: Error: selected processor does not support `dup z18.s,#0'
../kernel/arm64/sgemm_kernel_sve_v2x8.S:914: Error: selected processor does not support `dup z19.s,#0'
../kernel/arm64/sgemm_kernel_sve_v2x8.S:914: Error: selected processor does not support `dup z20.s,#0'
../kernel/arm64/sgemm_kernel_sve_v2x8.S:914: Error: selected processor does not support `dup z21.s,#0'
...
  >> 5785    ../kernel/arm64/strmm_kernel_sve_v1x8.S:981: Error: selected processor does not support `incw x9'
  >> 5786    ../kernel/arm64/strmm_kernel_sve_v1x8.S:982: Error: selected processor does not support `whilelt p1.s,x9,x0'
  >> 5787    ../kernel/arm64/strmm_kernel_sve_v1x8.S:983: Error: selected processor does not support `cntp x15,p0,p1.s'
  >> 5788    make[1]: *** [Makefile.L3:964: strmm_kernel_LN_NEOVERSEV1.o] Error 1
  >> 5789    make[1]: *** [Makefile.L3:974: strmm_kernel_LT_NEOVERSEV1.o] Error 1
  >> 5790    make: *** [Makefile:188: libs] Error 1

See build log for details:
  /tmp/root/spack-stage/spack-stage-openblas-0.3.24-aw2blghbqnzq4b5cxu6u2ztv5s4dbyhg/spack-build-out.txt

Error message

Error message
... see above

Information on your system

  • Spack: 0.21.0.dev0 (492a8111b92a49e3ca918a2a41d336b60dd58c2d)
  • Python: 3.8.10
  • Platform: linux-ubuntu20.04-neoverse_n1
  • Concretizer: clingo

Additional information

spack-build-env.txt spack-build-out.txt

@haampie @eschnett @martin-frbg

General information

  • I have run spack debug report and reported the version of Spack/Python/Platform
  • I have run spack maintainers <name-of-the-package> and @mentioned any maintainers
  • I have uploaded the build log and environment files
  • I have searched the issues of this repo and believe this is not a duplicate

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Comments: 26 (12 by maintainers)

Commits related to this issue

Most upvoted comments

Hello,

I could reproduce on an Ampere Altra Q8030 (Neoverse N1). I first install a gcc 14.2.0, load it, spack compiler find it; then install a HPL with it. It fails on latest develop branch with OpenBLAS 0.3.27 issuing similar errors as described above: targetting arm+sve.

I could get a working installation by disabling dynamic dispatch and enabling the experimental CMake build system. E.g. with HPL : spack install hpl %gcc@14.2.0 ^openblas build_system=cmake ~dynamic_dispatch

This should let the CMake of OpenBLAS decide what to compile for,

Also, note that just using ~dynamic_dispatch (using the default makefile build system) still issues DYNAMIC_ARCH=1.

==> [2024-08-19-15:57:57.129359] 'make' '-j20' '-s' 'CC=/home_nfs/bmorvana/test/spack/spack/lib/spack/env/gcc/gcc' 'FC=/home_nfs/bmorvana/test/spack/spack/lib/spack/env/gcc/gfortran' 'MAKE_NB_JOBS=0' 'ARCH=arm64' 'DYNAMIC_ARCH=1' 'DYNAMIC_OLDER=1' 'TARGET=GENERIC' 'USE_LOCKING=1' 'USE_OPENMP=0' 'USE_THREAD=0' 'RANLIB=ranlib' 'all'

The logic around here is most likely the culprit : https://github.com/spack/spack/blob/ac9398ed2144d9c364c771e638b9e098420b96e9/var/spack/repos/builtin/packages/openblas/package.py#L412-L422

Best.

The workaround @martin-frbg proposed doesn’t work because the wrapper internally pass the -mcpu flag to gcc. When I commented out a line preextend flags_list SPACK_TARGET_ARGS from the wrapper (lib/spack/env/gcc/gcc), the compilation finished successfully.

I have a similar error trying to build openblas on a Raspberry Pi 5 (Raspberry Pi OS). I’m attaching the spack out and env files. spack-build-out.txt spack-build-env.txt

The combination it complains about is a bit different:

cc1: warning: switch '-mcpu=cortex-a72' conflicts with '-march=armv8.2-a' switch

But then I get the same sort of errors with selected processor does not support ....

maybe try adding something like CFLAGS := $(filter-out -march=neoverse_n1,$(CFLAGS)) somewhere near the start of OpenBLAS’ toplevel makefile

oh wait - how does the combination of -mcpu=neoverse-n1 and -march=armv8.4-a+sve come to be ? Obviously the N1 is not supposed to do SVE, and the DYNAMIC_ARCH compilation has progressed to the V1 while apparently retaining compiler options from the preceding iteration. This is not supposed to happen (and even if we managed to accumulate a frankensteinian string of compiler options from each cpu in the DYNAMIC_ARCH set, it should conclude with -mtune=neoverse-v1 (makefile.arm64 using mtune in lieu of mcpu) if it already has armv8.4-a+sve )