clangd: System include extraction unable to get system include from arm-none-eabi-gcc

@Falven I just use --query-driver=** and do not care about slashes, different compilers in compile_commands.json, etc.

@ArcsinX Do you also sleep with your windows and doors unlocked/open? 😃

aha! i was just trying to point out the slash mismatch, but you beat me to it. we’ll try to get this handled more graciously on clangd side as well (i.e. look for a file with os executable suffix if we can’t find the driver at first try), but it won’t be available before clangd-13 (which is due in 6 months).

So for a faster solution you might want to contact the tool generating the compile_commands.json to see why they don’t keep the executable suffix. This is the first time we’re facing this issue, so I guess most other generators in windows must be preserving the suffix.

However, looking at the @Falven’s last comment, it seems the mixed slashes are not really a problem, just the .exe, which is something we can probably add, at least for now. It would be good if clangd would indeed be more permissive in this area, though.

Agreed, we definitely should and I’ll prepare a patch for the .exe issue. Thanks for swift action on your side as well!

Also if you are shipping your own clangd with the ide (i.e. not just making use of the major llvm releases, but building from close to HEAD), you can just wait for us to land the fix, which should happen in less than a week.

Hi @kadircet, I’m partly responsible for the code that generated @Falven’s compile_commands.json files 😃

So for a faster solution you might want to contact the tool generating the compile_commands.json to see why they don’t keep the executable suffix.

Looking at our code, it seems that the suffix was never there in the first place. The compilation commands are generated from configuration files that specify templates for compilation commands, but these are cross-platform, so they just specify the command to execute (optionally qualified with a path) and we pass that on to the platform as-is (through go.Cmd), which apparently accepts the path without .exe or adds it somewhere.

This is also the reason we have mixed slashes: The config files just use forward slashes, but some parts are interpolated into the templates by our toolling, so those use native slashes. So far, this has not been a problem, Windows (for the cmd) and gcc (for e.g. include paths) accept the mixed paths just fine. This also means that the mixed slashes are hard for us to fix, since our tooling does not really know which arguments are paths and which are not (there might also be slashes in non-path arguments that should not be touched).

However, looking at the @Falven’s last comment, it seems the mixed slashes are not really a problem, just the .exe, which is something we can probably add, at least for now. It would be good if clangd would indeed be more permissive in this area, though.

Ok. So if I fix all of the non-windows paths, and append the .exe extension to both the compile_commands.json and --query-driver command, it works!

[
 {
  "directory": "c:\\Users\\falve\\.vscode\\extensions\\vsciot-vscode.vscode-arduino-0.4.2\\out\\src",
  "arguments": [
   "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\bin\\arm-none-eabi-g++.exe",
   "-mcpu=cortex-m0plus",
   "-mthumb",
   "-c",
   "-g",
   "-Os",
   "-w",
   "-std=gnu++11",
   "-ffunction-sections",
   "-fdata-sections",
   "-fno-threadsafe-statics",
   "-nostdlib",
   "--param",
   "max-inline-insns-single=500",
   "-fno-rtti",
   "-fno-exceptions",
   "-MMD",
   "-DF_CPU=48000000L",
   "-DARDUINO=10607",
   "-DARDUINO_SAMD_MKRWIFI1010",
   "-DARDUINO_ARCH_SAMD",
   "-DUSE_ARDUINO_MKR_PIN_LAYOUT",
   "-D__SAMD21G18A__",
   "-DUSB_VID=0x2341",
   "-DUSB_PID=0x8054",
   "-DUSBCON",
   "-DUSB_MANUFACTURER=\"Arduino LLC\"",
   "-DUSB_PRODUCT=\"Arduino MKR WiFi 1010\"",
   "-DUSE_BQ24195L_PMIC",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS\\4.5.0\\CMSIS\\Include\\",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS-Atmel\\1.2.0\\CMSIS\\Device\\ATMEL\\",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated-avr-comp",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010",
   "c:\\source\\repos\\clangd-arduino\\build\\sketch\\clangd-arduino.ino.cpp",
   "-o",
   "c:\\source\\repos\\clangd-arduino\\build\\sketch\\clangd-arduino.ino.cpp.o"
  ],
  "file": "c:\\source\\repos\\clangd-arduino\\build\\sketch\\clangd-arduino.ino.cpp"
 },
 {
  "directory": "c:\\Users\\falve\\.vscode\\extensions\\vsciot-vscode.vscode-arduino-0.4.2\\out\\src",
  "arguments": [
   "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\arm-none-eabi-gcc\\7-2017q4\\bin\\arm-none-eabi-g++.exe",
   "-mcpu=cortex-m0plus",
   "-mthumb",
   "-c",
   "-g",
   "-Os",
   "-w",
   "-std=gnu++11",
   "-ffunction-sections",
   "-fdata-sections",
   "-fno-threadsafe-statics",
   "-nostdlib",
   "--param",
   "max-inline-insns-single=500",
   "-fno-rtti",
   "-fno-exceptions",
   "-MMD",
   "-DF_CPU=48000000L",
   "-DARDUINO=10607",
   "-DARDUINO_SAMD_MKRWIFI1010",
   "-DARDUINO_ARCH_SAMD",
   "-DUSE_ARDUINO_MKR_PIN_LAYOUT",
   "-D__SAMD21G18A__",
   "-DUSB_VID=0x2341",
   "-DUSB_PID=0x8054",
   "-DUSBCON",
   "-DUSB_MANUFACTURER=\"Arduino LLC\"",
   "-DUSB_PRODUCT=\"Arduino MKR WiFi 1010\"",
   "-DUSE_BQ24195L_PMIC",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS\\4.5.0\\CMSIS\\Include\\",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\CMSIS-Atmel\\1.2.0\\CMSIS\\Device\\ATMEL\\",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino\\api\\deprecated-avr-comp",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\cores\\arduino",
   "-IC:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010",
   "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010\\variant.cpp",
   "-o",
   "c:\\source\\repos\\clangd-arduino\\build\\core\\variant.cpp.o"
  ],
  "file": "C:\\Users\\falve\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\samd\\1.8.11\\variants\\mkrwifi1010\\variant.cpp"
 }
]

I[12:11:46.274] clangd version 12.0.0 (https://github.com/kirillbobyrev/llvm-project 8a844a3a321ea56da21af4d48433ad493b1bcfae)
I[12:11:46.275] PID: 21548
I[12:11:46.275] Working directory: c:\source\repos\clangd-arduino
I[12:11:46.275] argv[0]: c:\Users\falve\AppData\Roaming\Code\User\globalStorage\llvm-vs-code-extensions.vscode-clangd\install\12.0.0\clangd_12.0.0\bin\clangd.exe
I[12:11:46.275] argv[1]: --log=verbose
I[12:11:46.275] argv[2]: --query-driver=C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-gcc.exe,C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-g++.exe
V[12:11:46.285] User config file is C:\Users\falve\AppData\Local\clangd\config.yaml
I[12:11:46.286] Starting LSP over stdin/stdout

...

I[12:11:46.685] Loaded compilation database from c:\source\repos\clangd-arduino\build/compile_commands.json
V[12:11:46.685] Broadcasting compilation database from c:\source\repos\clangd-arduino
I[12:11:46.689] Loaded compilation database from c:\source\repos\clangd-arduino\build\compile_commands.json
V[12:11:47.114] System include extraction: target extracted: "arm-none-eabi"
V[12:11:47.114] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1

V[12:11:47.114] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/arm-none-eabi

V[12:11:47.114] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/backward

V[12:11:47.114] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include

V[12:11:47.114] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include-fixed

V[12:11:47.114] System include extraction: adding  c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include

I[12:11:47.114] System includes extractor: successfully executed C:\Users\falve\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-g++.exe
	got includes: "c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1, c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/arm-none-eabi, c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include/c++/7.2.1/backward, c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include, c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/include-fixed, c:\users\falve\appdata\local\arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/include"
	got target: "arm-none-eabi"