mfem: HYPRE+GPU Runtime Error

I have compiled MFEM minimally using METIS, HYPRE (with CUDA), and CUDA. When I run mpirun -np 1 ./ex1p -d cuda, I get the following error.

CUDA error: (cudaGetLastError()) failed with error:
 --> invalid device function
 ... in function: void mfem::CuWrap1D(int, DBODY&&) [with int BLCK = 256; DBODY = __nv_dl_wrapper_t<__nv_dl_tag<mfem::Vector& (mfem::Vector::*)(double), &mfem::Vector::operator=, 1>, double*, double>&]
 ... in file: linalg/../general/forall.hpp:309

I can provide the scripts I used to configure each of the libraries if that would be helpful. I suspect this may be because I’m using CUDA 11.

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 16 (16 by maintainers)

Most upvoted comments

@tzanio Thanks for the quick reply!

Using hypre-2.22.1 with mfem:master seems to work with ex1p. It does look like there is a new HYPRE API break in the latest commit here that affects triangular solvers but this should probably be in a different issue.

wcdawn on Aug 31, 2021

Can you try running this in a debugger (with MFEM_DEBUG=YES) and post the stack trace after the segfault?

Also, what is the exact CUDA version you use? For example, what does nvcc --version show?

v-dobrev on Aug 6, 2021