mfem: HYPRE+HIP Runtime Error
I’m trying to compile with HIP & HYPRE. Compiling with just HIP works fine, but I’d like to use HypreBoomerAMG
.
I get the following runtime error when running ex1p
. Any help would be much appreciated.
Options used:
--mesh ../data/star.mesh
--order 1
--no-static-condensation
--no-partial-assembly
--device cpu
--visualization
Device configuration: cpu
Memory configuration: host-std
Number of finite element unknowns: 82561
Memory access fault by GPU node-1 (Agent handle: 0x24fa560) on address 0x7f3f67a0e000. Reason: Page not present or supervisor privilege.
[jezebel:1158855] *** Process received signal ***
[jezebel:1158855] Signal: Aborted (6)
[jezebel:1158855] Signal code: (-6)
[jezebel:1158855] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x143c0)[0x7f4103a4b3c0]
[jezebel:1158855] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7f40e7a7703b]
[jezebel:1158855] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7f40e7a56859]
[jezebel:1158855] [ 3] /opt/rocm-5.0.0/hip/lib/../../lib/libhsa-runtime64.so.1(+0x72847)[0x7f40e75d1847]
[jezebel:1158855] [ 4] /opt/rocm-5.0.0/hip/lib/../../lib/libhsa-runtime64.so.1(+0x7545b)[0x7f40e75d445b]
[jezebel:1158855] [ 5] /opt/rocm-5.0.0/hip/lib/../../lib/libhsa-runtime64.so.1(+0x18757)[0x7f40e7577757]
[jezebel:1158855] [ 6] /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609)[0x7f4103a3f609]
[jezebel:1158855] [ 7] /lib/x86_64-linux-gnu/libc.so.6(clone+0x43)[0x7f40e7b53163]
[jezebel:1158855] *** End of error message ***
Aborted (core dumped)
I’m using the master
branch of MFEM & HYPRE v2.24.0.
HYRPE config
./configure \
--with-hip \
--disable-fortran \
CC=/home/wcdawn/bin/openmpi-4.1.2/bin/mpicc \
CXX=/home/wcdawn/bin/openmpi-4.1.2/bin/mpicxx \
--with-MPI-include='/home/wcdawn/bin/openmpi-4.1.2/include' \
--with-MPI-libs='mpi mpi_cxx' \
--with-MPI-lib-dirs='/home/wcdawn/bin/openmpi-4.1.2/lib'
MFEM config
make config \
MFEM_USE_HIP=YES HIP_ARCH=gfx1030 \
MFEM_USE_MPI=YES \
MPICC=/home/wcdawn/bin/openmpi-4.1.2/bin/mpicc \
MPICXX=/home/wcdawn/bin/openmpi-4.1.2/bin/mpicxx \
HYPRE_DIR=@MFEM_DIR@/../hypre-2.24.0/src/hypre \
METIS_DIR=@MFEM_DIR@/../metis-5.1.0 MFEM_USE_METIS_5=YES
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 21 (17 by maintainers)
Hi folks. There is certainly an issue with HYPRE at present that I will try to address when I can. The issue is that the Navi gaming cards (gfx1030 indicates an RDNA2 card, so something like a 6900XT) run with warp/wavefront sizes of 32. Currently, HYPRE on AMD GPUs is setup for warp/wavefront size of 64. I’ll post a note here when we update HYPRE to support wavefront size 32 on AMD GPUs.