mfem: "CUSPARSE_SPMV_CSR_ALG1" is undefined

Compiling mfem with cuda support on ORNL Summit using the cuda 11.2.0 module.

Full error mssage

$ nvcc -Xcompiler=-fPIC  -g -Xcompiler=-Wall -std=c++14 -x=cu --expt-extended-lambda -arch=sm_70 -ccbin mpicxx \
-I/ccs/home/jfaibussow/petsc/arch-olcf-summit-streams-debug/include \
-I/ccs/home/jfaibussow/petsc/arch-olcf-summit-streams-debug/include  -I/ccs/home/jfaibussow/petsc/include \
-I/ccs/home/jfaibussow/petsc/arch-olcf-summit-streams-debug/include -I/sw/summit/cuda/11.2.0/include  \
-c linalg/complex_operator.cpp -o linalg/complex_operator.o

linalg/sparsemat.cpp(694): error: identifier "CUSPARSE_SPMV_CSR_ALG1" is undefined

Version info

Output of make info: mfemInfo.txt

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Mon_Nov_30_19:09:01_PST_2020
Cuda compilation tools, release 11.2, V11.2.67
Build cuda_11.2.r11.2/compiler.29373293_0

$ echo $CUDA_DIR
/sw/summit/cuda/11.2.0

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 1
Comments: 20 (20 by maintainers)

Most upvoted comments

I’ll open a PR tomorrow morning (for real this time!).

Jacobfaib on Aug 2, 2021

@Jacobfaib do you want to propose a PR with the above suggested change?

Yeah sure, I’ll check that the cusparse version check works on the machines I’ve got access to.

Jacobfaib on Jul 4, 2021

I’d say use the condition CUSPARSE_VERSION >= 11400 where CUSPARSE_VERSION is defined in cusparse.h as

#define CUSPARSE_VERSION (CUSPARSE_VER_MAJOR * 1000 + \
                          CUSPARSE_VER_MINOR *  100 + \
                          CUSPARSE_VER_PATCH)

Judging by the posts above, this should work on both Lassen and Summit with their respective installs of CUDA 11.2.0.

If this is not the exact cutoff version, we can adjust again later, if this comes up again.

v-dobrev on Jun 29, 2021

Alrighty so after getting in touch with the ORNL folks they mentioned that:

Cuda 11.3.0 documentation, where CUSPARSE_CSRMV_ALG1 was deprecated. In the 11.2.0 documentation (https://docs.nvidia.com/cuda/archive/11.2.0/cusparse/index.html#cusparse-generic-function-spmv) you can see that CUSPARSE_CSRMV_ALG1 is the only one present and CUSPARSE_SPMV_CSR_ALG1 is not defined at all. That missing definition is probably what you are running into in this case.

With that I inspected the respective $CUDA_[HOME|DIR]/include/cusparse.h for both Lassen and Summit and lo and behold

Lassen:

#define CUSPARSE_VER_MAJOR 11
#define CUSPARSE_VER_MINOR 4
#define CUSPARSE_VER_PATCH 1
#define CUSPARSE_VER_BUILD 1152

Summit:

#define CUSPARSE_VER_MAJOR 11
#define CUSPARSE_VER_MINOR 3
#define CUSPARSE_VER_PATCH 1
#define CUSPARSE_VER_BUILD 68

So maybe its not the cuda version that needs checking but the cusparse version.

Jacobfaib on Jun 17, 2021

Could you check if replacing #if CUDA_VERSION >= 11020 with #if CUDA_VERSION >= 11021 in linalg/sparsemat.cpp solves this issue?

This works.

Jacobfaib on Jun 16, 2021