spack: CUDA toolkit is missing a libcuda.so.1 symlink to stub lib
[!TIP] Workaround is to use
ldflags=-Wl,--allow-shlib-undefinedin Spack 0.22.0.dev0 and above. In the future we could look to do this by default.
Steps to reproduce
When you link a shared lib against the cuda stub lib
gcc -o libf.so f.o -lcuda
you get a libf.so with DT_NEEDED specified as libcuda.so.1, because that is the SONAME of the stub lib.
If you then link an executable against this library, it fails:
gcc -o app app.o -lf
because the linker has to locate transitive deps (just like the runtime loader would), and cannot find libcuda.so.1 because it does not exist in the stub lib dir.
ld: warning: libcuda.so.1, needed by libf.so, not found (try using -rpath or -rpath-link)
The hint about -rpath and -rpath-link does not apply, because there is no file libcuda.so.1 anywhere.
This would be solved if the CUDA stub libs dir contained:
libcuda.so.1 -> libcuda.so
It’s likely that NVIDIA folks did not consider this use case, they may have purposefully omitted this symlink to avoid that the stub lib gets used at runtime, but glanced over the fact that sometimes you need to link against a library linked against a stub lib, for which libcuda.so.1 is required.
About this issue
- Original URL
- State: open
- Created 3 years ago
- Reactions: 5
- Comments: 24 (15 by maintainers)
Or the login node of a cluster, which doesn’t exposes the drivers of the GPUs because they’re only available on the compute nodes. This setup is far from being unusual.
Ok, you should be able to set
ldflags=-Wl,--allow-shlib-undefinednow that d946c37cbb7a41acbb2b34fc98078acc96533705 is on develop.Yesterday I was able to compile
on a login node without a GPU (and without
libcuda.so.1)Sorry, to be clear I wasnt disagreeing. The issue is common, but for people to reproduce they need to have the full context