spack: Installation issue: trilinos~kokkos fails

Steps to reproduce the issue

$ spack install trilinos~kokkos

Annoyingly, the above command fails. I must actually run

$ spack install trilinos~kokkos~tpetra~amesos2~ifpack2

5 errors found in build log:
     2659                     from /tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-src/packages/muelu/src/Interface/MueLu_Pa
             rameterListInterpreter_def.hpp:74,
     2660                     from /tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-build-maqit3j/packages/muelu/src/Interfac
             e/MueLu_ParameterListInterpreter.hpp:2,
     2661                     from /tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-src/packages/muelu/adapters/xpetra/MueLu_
             CreateXpetraPreconditioner.hpp:24,
     2662                     from /tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-src/packages/muelu/adapters/xpetra/MueLu_
             RefMaxwell_def.hpp:98,
     2663                     from /tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-build-maqit3j/packages/muelu/src/MueLu_Re
             fMaxwell.hpp:2,
     2664                     from /tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-src/packages/muelu/adapters/aztecoo/MueLu
             _AztecEpetraOperator.cpp:8:
  >> 2665    /tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-src/packages/muelu/src/Graph/PairwiseAggregation/MueLu_NotayAgg
             regationFactory_def.hpp:55:10: fatal error: KokkosKernels_Handle.hpp: No such file or directory
     2666       55 | #include "KokkosKernels_Handle.hpp"
     2667          |          ^~~~~~~~~~~~~~~~~~~~~~~~~~
     2668    compilation terminated.
     2669    In file included from /tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-build-maqit3j/packages/muelu/src/MueLu_No
             tayAggregationFactory.hpp:2,
     2670                     from /tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-src/packages/muelu/src/Interface/MueLu_Fa
             ctoryFactory_decl.hpp:109,
     2671                     from /tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-build-maqit3j/packages/muelu/src/Interfac
             e/MueLu_FactoryFactory.hpp:1,
     2672                     from /tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-src/packages/muelu/src/Interface/MueLu_Pa
             rameterListInterpreter_def.hpp:74,
     2673                     from /tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-build-maqit3j/packages/muelu/src/Interfac
             e/MueLu_ParameterListInterpreter.hpp:2,
     2674                     from /tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-src/packages/muelu/adapters/xpetra/MueLu_
             CreateXpetraPreconditioner.hpp:24,
     2675                     from /tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-src/packages/muelu/adapters/epetra/MueLu_
             CreateEpetraPreconditioner.cpp:14:
  >> 2676    /tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-src/packages/muelu/src/Graph/PairwiseAggregation/MueLu_NotayAgg
             regationFactory_def.hpp:55:10: fatal error: KokkosKernels_Handle.hpp: No such file or directory
     2677       55 | #include "KokkosKernels_Handle.hpp"
     2678          |          ^~~~~~~~~~~~~~~~~~~~~~~~~~
     2679    compilation terminated.
  >> 2680    make[2]: *** [packages/muelu/adapters/CMakeFiles/muelu-adapters.dir/build.make:98: packages/muelu/adapters/CMakeFiles/muelu-adapters.dir/epetra/MueLu_Crea
             teEpetraPreconditioner.cpp.o] Error 1
     2681    make[2]: *** Waiting for unfinished jobs....
  >> 2682    make[2]: *** [packages/muelu/adapters/CMakeFiles/muelu-adapters.dir/build.make:111: packages/muelu/adapters/CMakeFiles/muelu-adapters.dir/aztecoo/MueLu_Az
             tecEpetraOperator.cpp.o] Error 1
     2683    make[2]: Leaving directory '/tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-build-maqit3j'
  >> 2684    make[1]: *** [CMakeFiles/Makefile2:4775: packages/muelu/adapters/CMakeFiles/muelu-adapters.dir/all] Error 2
     2685    make[1]: Leaving directory '/tmp/kellyt/spack-stage/spack-stage-trilinos-13.0.1-maqit3jlyebeuwg4f6ggnk3l55icglm6/spack-build-maqit3j'
     2686    make: *** [Makefile:185: all] Error 2

cc maintainers: @keitat @kuberry @sethrj

Information on your system

$ spack debug report
* **Spack:** 0.16.2-4032-85f10adaff
* **Python:** 3.6.8
* **Platform:** linux-rhel8-broadwell
* **Concretizer:** original

Additional information

spack-build-env.txt spack-build-out.txt

General information

  • I have run spack debug report and reported the version of Spack/Python/Platform
  • I have run spack maintainers <name-of-the-package> and @mentioned any maintainers
  • I have uploaded the build log and environment files
  • I have searched the issues of this repo and believe this is not a duplicate

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 21 (19 by maintainers)

Most upvoted comments

So a quick recap. The recipe does now work for @KineticTheory, once they add ~muelu. MueLu does have a required dependency on Kokkos in Tribits that is currently listed as optional.

I can follow up with the MueLu team to see whether they are intending to support a Tpetra+Kokkos free build or not. If they are not, then PR #25703 should be merged, disabling an Epetra only build path for MueLu. If they are trying to, then I’ll need to look into whether it is some other configuration option in the recipe causing the issue or a bug on their end.

https://github.com/trilinos/Trilinos/issues/9648

I am shocked this even configures.

We’ve been using that line for several years w/o issue!

Maybe we should move toward toggles for first and second generation package groups (epetra and all its solvers, tpetra and all its solvers) instead of adding more variants and propagating tribits logic into the spack recipe?

Follow up: Based on issue https://github.com/trilinos/Trilinos/pull/9532, I was referred to PR https://github.com/trilinos/Trilinos/pull/9532.

MueLu is going full Kokkos. I suggest we approve PR https://github.com/spack/spack/pull/25703.

We’ve been using that line for several years w/o issue!

So you have been building the Epetra-only stack in Trilinos? (That is what you get if you disable Kokkos and Tpetra basically.) I don’t think there is much, if any, automated testing to ensure that the Epetra-only stack works in Trilinos these days (or this configuration would still be working). It is just the opposite, they want the Epetra stack to go away and to get everyone to move over to the newer Kokkos/Tpetra-based stack. So they likely want you to be building trilinos~epetra if anything. But I really should not be commenting on official Trilinos policy. I will let the other Trilinos developers CCed on this issue do that.

I am shocked this even configures.