spack: OpenBLAS build error on macOS

Summary

The installation for hypre fails for spack on Mac OS 10.13.4. The failure occurs when openblas 0.2.20 installation is attempted. This failure occurs for both gcc7.3 and intel 18.0.2. The problem is not spack directly but really openblas-0.2.20 which tries to use assembler commands that do not exist for this system. I don’t give a hoot about openblas since I really want to use the system versions of blas and lapack. The problem is that the spack default configuration for hypre tries to build openblas. When I build hypre by hand, the configure options --with-blas and --with-lapack will use the system files in /usr/lib, which works. I cannot for the life of me figure out from the spack documentation how to tell it to use these options, much less how to write a package.yaml specialization to tell all packages to use the system versions of blas and lapack. I tried using blas : [system] and lapack : [system] in my user packages.yaml and this didn’t help. I also tried removing these lines entires. In either case spack still tries to build openblas.

Expected Result

spack install hypre %gcc@7.3.0 and spack install hypre %intel@18.0.2

work

Actual Result

The command spack install hypre %gcc7.3.0 fails with the error message ==> Installing openblas ==> Using cached archive: /Users/jgrove/spack/var/spack/cache/openblas/openblas-0.2.20.tar.gz ==> Staging archive: /Users/jgrove/spack/var/spack/stage/openblas-0.2.20-cqu36bmxblsej2t4vro52tvfxhhmwbg5/v0.2.20.tar.gz ==> Created stage in /Users/jgrove/spack/var/spack/stage/openblas-0.2.20-cqu36bmxblsej2t4vro52tvfxhhmwbg5 ==> Applied patch make.patch ==> Building openblas [MakefilePackage] ==> Executing phase: ‘edit’ ==> Executing phase: ‘build’ ==> Error: ProcessError: Command exited with status 2: ‘make’ ‘CC=/Users/jgrove/spack/lib/spack/env/gcc/gcc’ ‘FC=/Users/jgrove/spack/lib/spack/env/gcc/gfortran’ ‘MAKE_NO_J=1’ ‘USE_OPENMP=0’ ‘USE_THREAD=0’ ‘libs’ ‘netlib’ ‘shared’

1 error found in build log: 2337 …/kernel/x86_64/saxpy_microk_sandy-2.c:82:no such instruction: vaddps %ymm11, %ymm7,%ymm15' 2338 ../kernel/x86_64/saxpy_microk_sandy-2.c:83:no such instruction: vmovups %ymm12, -128(%rdx,%rax,4)’ 2339 …/kernel/x86_64/saxpy_microk_sandy-2.c:84:no such instruction: vmovups %ymm13, -96(%rdx,%rax,4)' 2340 ../kernel/x86_64/saxpy_microk_sandy-2.c:85:no such instruction: vmovups %ymm14, -64(%rdx,%rax,4)’ 2341 …/kernel/x86_64/saxpy_microk_sandy-2.c:86:no such instruction: vmovups %ymm15, -32(%rdx,%rax,4)' 2342 ../kernel/x86_64/saxpy_microk_sandy-2.c:87:no such instruction: vzeroupper’

2343 make[1]: *** [Makefile.L1:640: saxpy_k.o] Error 1 2344 make[1]: Leaving directory ‘/private/var/folders/jl/9jpdtxpj67v2z2kn9x6bh0d8000c82/T/jgrove/spack-stage/sp ack-stage-8O8TWh/OpenBLAS-0.2.20/kernel’ 2345 make: *** [Makefile:139: libs] Error 1

See build log for details: /Users/jgrove/spack/var/spack/stage/openblas-0.2.20-cqu36bmxblsej2t4vro52tvfxhhmwbg5/OpenBLAS-0.2.20/spack-build.out

This is really a problem with OpenBLAS-0.2.20 which apparently does not properly support Macs. I don’t want to use OpenBLAS anyway, just to tell hypre to use the system versions of blas and lapack. I expect this issue occurs for all packages that try to use blas or lapack, directly or indirectly and the correct configuration in my opinion is that for Mac darwin, use the system provided versions of these libraries.

Steps to reproduce the issue

On a Mac running OS 10.13.4 run

$ spack install hypre %gcc@7.3.0
and/or
$ spack install hypre %intel@18.0.2
...

Information on your system

This includes:

  1. Late 2013 MacPro running OS 10.13.4
  2. I tried modifying packages.yaml as follow but this had no effect
packages:
  all:
    compiler: [gcc, intel, pgi, clang, xl, nag]
    providers:
      awk: [gawk]
      blas: [system]
      daal: [intel-daal]
      elf: [elfutils]
      gl: [mesa, opengl]
      golang: [gcc]
      ipp: [intel-ipp]
      java: [jdk]
      lapack: [system]
      mkl: [intel-mkl]
      mpe: [mpe2]
      mpi: [openmpi, mpich]
      opencl: [pocl]
      openfoam: [openfoam-com, openfoam-org, foam-extend]
      pil: [py-pillow]
      pkgconfig: [pkgconf, pkg-config]
      scalapack: [netlib-scalapack]
      szip: [libszip, libaec]
      tbb: [intel-tbb]
      jpeg: [libjpeg-turbo, libjpeg]

About this issue

  • Original URL
  • State: closed
  • Created 9 years ago
  • Comments: 16 (10 by maintainers)

Most upvoted comments

Increasingly, things just don’t build on Mac with GCC. I had Spack build a GCC for me, and then set up my compilers.yaml with it, but JUST FOR FORTRAN:

compilers:
- compiler:
    environment: {}
    extra_rpaths: []
    flags: {}
    modules: []
    operating_system: sierra
    paths:
      cc: /usr/bin/clang
      cxx: /usr/bin/clang++
      f77: /Users/rpfische/spack/gcc/opt/spack/darwin-sierra-x86_64/clang-9.0.0-apple/gcc-6.4.0-ph64x3ujobquzhno56opmg3ig7mnntku/bin/gfortran
      fc: /Users/rpfische/spack/gcc/opt/spack/darwin-sierra-x86_64/clang-9.0.0-apple/gcc-6.4.0-ph64x3ujobquzhno56opmg3ig7mnntku/bin/gfortran
    spec: clang@9.0.0-apple
    target: x86_64
- compiler:
    environment: {}
    extra_rpaths: []
    flags: {}
    modules: []
    operating_system: sierra
    paths:
      cc: /Users/rpfische/spack/gcc/opt/spack/darwin-sierra-x86_64/clang-9.0.0-apple/gcc-6.4.0-ph64x3ujobquzhno56opmg3ig7mnntku/bin/gcc
      cxx: /Users/rpfische/spack/gcc/opt/spack/darwin-sierra-x86_64/clang-9.0.0-apple/gcc-6.4.0-ph64x3ujobquzhno56opmg3ig7mnntku/bin/g++
      f77: /Users/rpfische/spack/gcc/opt/spack/darwin-sierra-x86_64/clang-9.0.0-apple/gcc-6.4.0-ph64x3ujobquzhno56opmg3ig7mnntku/bin/gfortran
      fc: /Users/rpfische/spack/gcc/opt/spack/darwin-sierra-x86_64/clang-9.0.0-apple/gcc-6.4.0-ph64x3ujobquzhno56opmg3ig7mnntku/bin/gfortran
    spec: gcc@6.4.0
    target: x86_64

At the risk of some loss of optimization, the openblas package for GCC 7.3.0 on macs can be built using cpu_target=GENERIC “spack install openblas %gcc@7.3.0 cpu_target=GENERIC”. This at least compiles. I am now having trouble getting openblas to build with the intel compiler on Macs. Here the issue is that it is not loading Fortran properly when it compiles a dynamic library. I am still trying to figure this out and will likely submit another issue on this problem. All in all, it looks like the veclibfort package is a more reasonable way to get blas/lapack on Macs than openblas.

John W. Grove CCS-2 MS D413, PO Box 1663 Los Alamos National Laboratory Los Alamos, NM 87545 (505) 667-0723 jgrove@lanl.gov From: Elizabeth Fischer notifications@github.com Reply-To: spack/spack reply@reply.github.com Date: Monday, April 23, 2018 at 11:23 AM To: spack/spack spack@noreply.github.com Cc: “John W. Grove” jgrove@lanl.gov, Mention mention@noreply.github.com Subject: Re: [spack/spack] OpenBLAS build error on macOS (#7803)

On Mon, Apr 23, 2018 at 1:07 PM, JohnWGrove notifications@github.com wrote:

Thanks Elizabeth: I will look at this link. GCC 7.3.0 is in fact the current gcc release https://gcc.gnu.org/ as of January 25, the development branch is gcc 8. I appreciate your help.

Yes you’re right, gcc7 is no longer in dev. But some important things still don’t work with it.

– Elizabeth

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/spack/spack/issues/7803#issuecomment-383654740, or mute the threadhttps://github.com/notifications/unsubscribe-auth/Af6iu-fUdn50dvKFAIfBJAit2MxEQMwLks5trg3zgaJpZM4TZGbm.

I believe the system BLAS/LAPACK on macOS is called veclibfort?

that’s not entirely correct. veclibfort fixes some fortran-related things in macOS blas/lapack:

vecLibFort is lightweight but flexible “shim” designed to rectify the incompatibilities between the Accelerate/vecLib BLAS and LAPACK libraries shipped with Mac OS X and FORTRAN code compiled with modern compilers such as GNU Fortran.

But yes, the only thing you need to do is

packages:
    all:
        providers:
            blas: [veclibfort]
            lapack: [veclibfort]

You don’t need to tell Spack that veclibfort is a system-provided package (it is not!).