dlib: CUDA cuDNN CMake try_compile tests fail w/ Xcode clang=8.0.0

I’m walking through dlib DNN CUDA and cuDNN setup for a Mac OS X 10.12 with a preferred Xcode 8.1 (or newer) toolchain.

clang --version:

Apple LLVM version 8.0.0 (clang-800.0.42.1)

Apparently, Xcode/clang doesn’t like the CUDA and cuDNN CMake try_compile tests. In particular, the -std=c++11 CUDA_NVCC_FLAGS flag triggers an error. Removing the flag makes it happy.

An earlier post ( https://github.com/davisking/dlib/issues/356 ) suggests a clang-omp compiler can be used as a workaround on OS X, but the latest Xcode toolchain seems to work fine with a few mods. I’d prefer to work this into the current CMake if possible.

Here is a pointer to the relevant code in the WIP fork: https://github.com/headupinclouds/dlib/commit/1b8bb49ec7527817e4b43d88f3949544ff9246f1

#list(APPEND CUDA_NVCC_FLAGS "-arch=sm_30;-std=c++11;-D__STRICT_ANSI__;-D_MWAITXINTRIN_H_INCLUDED;-D_FORCE_INLINES")
# -std=c++11 fails on OSX/Xcode clang=8.0.0
list(APPEND CUDA_NVCC_FLAGS "-arch=sm_30;-D__STRICT_ANSI__;-D_MWAITXINTRIN_H_INCLUDED;-D_FORCE_INLINES")

I think we can probably exclude the flag for detected XCODE builds:

if(NOT XCODE) # possibly more specific (AND CLANG_SOMETHING)
   list(APPEND CUDA_NVCC_FLAGS "-std=c++11")
endif()

After that, I ran into an issue related to a previously documented CUDA + OpenMP requirement, which doesn’t seem to be the case for Xcode/clang. An initial workaround in my test fork is here:

https://github.com/headupinclouds/dlib/commit/72b5464ae083dfed8660ca5c587108d6f330bb71

Essentially adding NOT XCODE to omit openmp_libraries linking seems to be sufficient:

if (NOT openmp_libraries AND NOT MSVC AND NOT XCODE) 

If such changes would be merged upstream, I can dig a little deeper and put a revise pr.xcode.cuda.fix PR together for review and further discussion. Let me know if that makes sense. Thanks!