blt: linking issue with openmpi on arch linux (blt_mpi_smoke fails to link)
reported by an ascent user, encountered in conduit build.
Is likely a cmake issue, not a blt issue per se.
gcc 9.2.0 on arch linux spack built openmpi-3.1.4 BLT Version: 0.2.5 CMake Version: 3.14.5
[ 6%] Linking CXX executable ../../../tests/blt_mpi_smoke
cd BUILD_LOC/spack/var/spack/stage/spack-stage-conduit-master-kxqo2mlz4yzti5ougeq2m2fhlzaa73be/spack-src/spack-build/blt/tests/smoke && BUILD_LOC/spack/opt/spack/linux-arch-sandybridge/gcc-9.2.0/cmake-3.14.5-wcfeiamw55g4vwlceklvxyibctne2lgb/bin/cmake -E cmake_link_script CMakeFiles/blt_mpi_smoke.dir/link.txt --verbose=1
/usr/bin/g++ -O2 -g -DNDEBUG -rdynamic
-Wl,-rpath -Wl,BUILD_LOC/spack/opt/spack/linux-arch-sandybridge/gcc-9.2.0/hwloc-1.11.11-ljswr5og475ralakqrelzykauwnfw3lf/lib
-Wl,BUILD_LOC/spack/opt/spack/linux-arch-sandybridge/gcc-9.2.0/openmpi-3.1.4-7tuxmnjnzj5swmus2shzh3jep2prtzuv/lib
-LBUILD_LOC/spack/opt/spack/linux-arch-sandybridge/gcc-9.2.0/hwloc-1.11.11-ljswr5og475ralakqrelzykauwnfw3lf/lib
-pthread
CMakeFiles/blt_mpi_smoke.dir/blt_mpi_smoke.cpp.o
-o ../../../tests/blt_mpi_smoke
-Wl,-rpath,BUILD_LOC/spack/opt/spack/linux-arch-sandybridge/gcc-9.2.0/openmpi-3.1.4-7tuxmnjnzj5swmus2shzh3jep2prtzuv/lib
BUILD_LOC/spack/opt/spack/linux-arch-sandybridge/gcc-9.2.0/openmpi-3.1.4-7tuxmnjnzj5swmus2shzh3jep2prtzuv/lib/libmpi.so
BUILD_LOC/spack/opt/spack/linux-arch-sandybridge/gcc-9.2.0/openmpi-3.1.4-7tuxmnjnzj5swmus2shzh3jep2prtzuv/lib/libmpi_cxx.so
BUILD_LOC/spack/opt/spack/linux-arch-sandybridge/gcc-9.2.0/openmpi-3.1.4-7tuxmnjnzj5swmus2shzh3jep2prtzuv/lib/libmpi_usempif08.so
BUILD_LOC/spack/opt/spack/linux-arch-sandybridge/gcc-9.2.0/openmpi-3.1.4-7tuxmnjnzj5swmus2shzh3jep2prtzuv/lib/libmpi_usempi_ignore_tkr.so
BUILD_LOC/spack/opt/spack/linux-arch-sandybridge/gcc-9.2.0/openmpi-3.1.4-7tuxmnjnzj5swmus2shzh3jep2prtzuv/lib/libmpi_mpifh.so
Error:
ld: BUILD_LOC/spack/opt/spack/linux-arch-sandybridge/gcc-9.2.0/openmpi-3.1.4-7tuxmnjnzj5swmus2shzh3jep2prtzuv/lib: file not recognized: Is a directory
Looks like a malformed rpath command, the linker is simply passed a directory:
-Wl,BUILD_LOC/spack/opt/spack/linux-arch-sandybridge/gcc-9.2.0/openmpi-3.1.4-7tuxmnjnzj5swmus2shzh3jep2prtzuv/lib
Notice later it is also passed what looks like the expected rpath command:
-Wl,-rpath,BUILD_LOC/spack/opt/spack/linux-arch-sandybridge/gcc-9.2.0/openmpi-3.1.4-7tuxmnjnzj5swmus2shzh3jep2prtzuv/lib
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 21 (21 by maintainers)
I recalled there were some changes + recommend Matt update and I don’t think that resolved the problem.
I suspect the openmpi compiler wrapper
showme
is feeding bad link lines to CMake, and those might be hard, but not impossible to tactically fix.Usually when folks hit his problem, we tell them to use mpich, end of story.
In our new case, we need openmpi since there is some container magic at play that requires it.