spack: Build error: kokkos-kernels +openmp+cuda arch=zen2

Steps to reproduce

The basic command is

spack install trilinos@13.2.0+cuda+hdf5~mpi+openmp cuda_arch=72 ^python~pyexpat ^kokkos-nvcc-wrapper~mpi

but attached is the script used to run that, as it sets other variables. .txt was added to all file names so that github would accept them. I also attach the build-out.txt file, which contains the full list of errors.

An example error is

/home/bilder/contrib/gcc-9.4.0/lib/gcc/x86_64-pc-linux-gnu/9.4.0/include/avx512fintrin.h(9146): error: identifier “__builtin_ia32_rndscaless_round” is undefined

which may indicate that trilinos is trying to include avx512 instructions that don’t exist or are not implemented?

runspack.sh.txt setupspack.sh.txt spack.yaml.txt spack-build-out.txt .

Error message

cd /home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-build-qqmz7rt/packages/kokkos-kernels/src && /home/research/cary/opt/linux-rocky8-zen2/gcc-9.4.0/kokkos-nvcc-wrapper-3.2.00-4n5jp35hi6jheb7giutnfl5rieynvt74/bin/nvcc_wrapper -Dkokkoskernels_EXPORTS -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-build-qqmz7rt -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-build-qqmz7rt/packages/kokkos-kernels/src -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-src/packages/kokkos-kernels/src -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-src/packages/kokkos-kernels/src/impl -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-build-qqmz7rt/packages/kokkos-kernels/src/impl -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-src/packages/kokkos-kernels/src/impl/tpls -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-src/packages/kokkos-kernels/src/blas -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-src/packages/kokkos-kernels/src/blas/impl -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-src/packages/kokkos-kernels/src/sparse -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-src/packages/kokkos-kernels/src/sparse/impl -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-src/packages/kokkos-kernels/src/graph -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-src/packages/kokkos-kernels/src/graph/impl -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-src/packages/kokkos-kernels/src/batched -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-src/packages/kokkos-kernels/src/common -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-build-qqmz7rt/packages/kokkos/algorithms/src -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-src/packages/kokkos/algorithms/src -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-build-qqmz7rt/packages/kokkos/containers/src -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-src/packages/kokkos/containers/src -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-build-qqmz7rt/packages/kokkos/core/src -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-src/packages/kokkos/core/src -I/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-build-qqmz7rt/packages/kokkos -I/home/research/cary/opt/linux-rocky8-zen2/gcc-9.4.0/hwloc-2.6.0-ois5nhxvr7jurardayfyrrfmjs3y3cfj/include -I/home/research/cary/opt/linux-rocky8-zen2/gcc-9.4.0/amdlibflame-3.1-pzioontfzjlj6mysxmqosynzdzdrdau4/include -I/home/research/cary/opt/linux-rocky8-zen2/gcc-9.4.0/amdblis-3.1-s7uqgliwjsadqlbqcvsji4owtcqugmot/include -march=znver2 -mtune=znver2  -Xcompiler -fopenmp -expt-extended-lambda -Wext-lambda-captures-this -arch=sm_72  -O2 -g -DNDEBUG -fPIC -std=c++14 -MD -MT packages/kokkos-kernels/src/CMakeFiles/kokkoskernels.dir/impl/generated_specializations_cpp/trmm/Blas3_trmm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o -MF CMakeFiles/kokkoskernels.dir/impl/generated_specializations_cpp/trmm/Blas3_trmm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o.d -o CMakeFiles/kokkoskernels.dir/impl/generated_specializations_cpp/trmm/Blas3_trmm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o -c /home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-build-qqmz7rt/packages/kokkos-kernels/src/impl/generated_specializations_cpp/trmm/Blas3_trmm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp
/home/bilder/contrib/gcc-9.4.0/lib/gcc/x86_64-pc-linux-gnu/9.4.0/include/avx512fintrin.h(9146): error: identifier "__builtin_ia32_rndscaless_round" is undefined

/home/bilder/contrib/gcc-9.4.0/lib/gcc/x86_64-pc-linux-gnu/9.4.0/include/avx512fintrin.h(9155): error: identifier "__builtin_ia32_rndscalesd_round" is undefined

/home/bilder/contrib/gcc-9.4.0/lib/gcc/x86_64-pc-linux-gnu/9.4.0/include/avx512fintrin.h(14797): error: identifier "__builtin_ia32_rndscaless_round" is undefined

/home/bilder/contrib/gcc-9.4.0/lib/gcc/x86_64-pc-linux-gnu/9.4.0/include/avx512fintrin.h(14806): error: identifier "__builtin_ia32_rndscalesd_round" is undefined

/home/bilder/contrib/gcc-9.4.0/lib/gcc/x86_64-pc-linux-gnu/9.4.0/include/avx512dqintrin.h(1365): error: identifier "__builtin_ia32_fpclassss" is undefined

/home/bilder/contrib/gcc-9.4.0/lib/gcc/x86_64-pc-linux-gnu/9.4.0/include/avx512dqintrin.h(1372): error: identifier "__builtin_ia32_fpclasssd" is undefined

6 errors detected in the compilation of "/home/research/cary/spack/stage/spack-stage-trilinos-13.2.0-qqmz7rt72db76rkdysb3lx6y3cimzkcg/spack-build-qqmz7rt/packages/kokkos-kernels/src/impl/generated_specializations_cpp/trsm/Blas3_trsm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp".
make[2]: *** [packages/kokkos-kernels/src/CMakeFiles/kokkoskernels.dir/build.make:1731: packages/kokkos-kernels/src/CMakeFiles/kokkoskernels.dir/impl/generated_specializations_cpp/trsm/Blas3_trsm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o] Error 2

Information on your system

$ spack debug report

  • Spack: 0.17.1-668-e0d8964453
  • Python: 3.6.8
  • Platform: linux-rocky8-zen2
  • Concretizer: clingo

General information

  • I have run spack debug report and reported the version of Spack/Python/Platform
  • I have searched the issues of this repo and believe this is not a duplicate
  • I have run the failing commands in debug mode and reported the output

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 18 (7 by maintainers)

Most upvoted comments

Trilinos now building. Thanks, @sethrj.

I see now that I created the environment initially using

spack env create -d vsimall

and that created that directory with a default spack.yaml. That file has in “view: true”. So if that is not a good thing to do, maybe the default creation should change?

I’ll put some of these comments into the spack/slack.

spack-build-out.txt

The above is the spack-build-out from the failed kokkos-kernels build.

Adding arch=x86_64 to the kokkos-kernels spec unleashed a whole bunch of building, all the way down through cmake, but ultimately it succeeded. LMK if you need anything else. For now I will just use arch=x86_64. Thx…

I removed “+hdf5”, I assume a typo. Then I get one arch warning and the same errors:

     570    nvcc warning : The 'compute_35', 'compute_37', 'compute_50', 'sm_35
            ', 'sm_37' and 'sm_50' architectures are deprecated, and may be rem
            oved in a future release (Use -Wno-deprecated-gpu-targets to suppre
            ss warning).
  >> 571    /home/bilder/contrib/gcc-9.4.0/lib/gcc/x86_64-pc-linux-gnu/9.4.0/in
            clude/avx512fintrin.h(9146): error: identifier "__builtin_ia32_rnds
            caless_round" is undefined
     572
  >> 573    /home/bilder/contrib/gcc-9.4.0/lib/gcc/x86_64-pc-linux-gnu/9.4.0/in
            clude/avx512fintrin.h(9146): error: identifier "__builtin_ia32_rnds
            caless_round" is undefined