clangd: Clangd fails to parse SYCL headers

Clangd (built from the current Intel SYCL LLVM branch, apologies if it’s behind) appears to be calling into Sema slightly differently to how the compiler does it normally; the instant I add -fsycl to the compilation flags and include the SYCL headers, clangd starts crashing in the guts of an AST traversal.

A one-line C++ file appears to trigger the problem (most of the time) if -fsycl is passed on the command line:

#include <CL/sycl.hpp>

Out of curiosity, is there any documentation around on how to start debugging these kinds of crashes? I’d be happy to dig into them more before reporting, but I’m not quite even managing to get a SourceLocation to turn into an actual file + line number yet.

Logs

I[15:17:09.828] clangd version 15.0.0 (https://github.com/intel/llvm.git a37ca8403341511c789f3e07768755a72befcd13)
I[15:17:09.828] Features: linux+debug
I[15:17:09.828] PID: 10777
I[15:17:09.828] Working directory: /localData/justinw/sycl
I[15:17:09.829] argv[0]: /localData/justinw/llvm-project/build/bin/clangd
I[15:17:09.829] argv[1]: --check=/localData/justinw/sycl/test.cpp
I[15:17:09.829] Entering check mode (no LSP server)
I[15:17:09.829] Testing on source file /localData/justinw/sycl/test.cpp
I[15:17:09.829] Loading compilation database...
I[15:17:09.866] Loaded compilation database from /localData/justinw/sycl/compile_commands.json
I[15:17:09.870] Compile command from CDB is: /localData/justinw/llvm-sycl/bin/clang++ --driver-mode=g++ -isystem /localData/justinw/llvm-sycl//include --gcc-toolchain=/d/sw/gcc/9.2.0/ --cuda-path=/d/sw/cuda/11.3.1/cuda-toolkit/ -std=c++17 -o /localData/justinw/sycl/test.o -c -fsycl --gcc-toolchain=/d/sw/gcc/9.2.0/ -resource-dir=/localData/justinw/llvm-project/build/lib/clang/15.0.0 -- /localData/justinw/sycl/test.cpp
I[15:17:09.870] Parsing command...
I[15:17:09.891] internal (cc1) args are: -cc1 -triple spir64-unknown-unknown -aux-triple x86_64-unknown-linux-gnu -fsycl-is-device -fdeclare-spirv-builtins -mllvm -sycl-opt -fenable-sycl-dae -Wno-sycl-strict -fsycl-int-header=/localData/000scratch/test-header-b4f8fa.h -fsycl-int-footer=/localData/000scratch/test-footer-0207df.h -sycl-std=2020 -fsycl-unique-prefix=0e1a0935590ef1ad -Wspir-compat -fsyntax-only -disable-free -clear-ast-before-backend -main-file-name test.cpp -mrelocation-model static -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -fno-verbose-asm -mconstructor-aliases -aux-target-cpu x86-64 -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -resource-dir /localData/justinw/llvm-project/build/lib/clang/15.0.0 -internal-isystem /localData/justinw/llvm-sycl/bin/../include/sycl -internal-isystem /localData/justinw/llvm-sycl/bin/../include -isystem /localData/justinw/llvm-sycl//include -I/d/sw/intel/latest/compilers_and_libraries_2020.0.166/linux/mkl/include -I/d/sw/openmpi/1.6.5/include -I/d/sw/dugio/latest/include -I/d/sw/fftw/3.2.2/include -c-isystem /intel_bug -internal-isystem /d/sw/gcc/9.2.0/lib/gcc/x86_64-pc-linux-gnu/9.2.0/../../../../include/c++/9.2.0 -internal-isystem /d/sw/gcc/9.2.0/lib/gcc/x86_64-pc-linux-gnu/9.2.0/../../../../include/c++/9.2.0/x86_64-pc-linux-gnu -internal-isystem /d/sw/gcc/9.2.0/lib/gcc/x86_64-pc-linux-gnu/9.2.0/../../../../include/c++/9.2.0/backward -internal-isystem /d/sw/gcc/9.2.0/lib/gcc/x86_64-pc-linux-gnu/9.2.0/../../../../include/c++/9.2.0 -internal-isystem /d/sw/gcc/9.2.0/lib/gcc/x86_64-pc-linux-gnu/9.2.0/../../../../include/c++/9.2.0/x86_64-pc-linux-gnu -internal-isystem /d/sw/gcc/9.2.0/lib/gcc/x86_64-pc-linux-gnu/9.2.0/../../../../include/c++/9.2.0/backward -internal-isystem /localData/justinw/llvm-project/build/lib/clang/15.0.0/include -internal-isystem /usr/local/include -internal-isystem /d/sw/gcc/9.2.0/lib/gcc/x86_64-pc-linux-gnu/9.2.0/../../../../x86_64-pc-linux-gnu/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -internal-isystem /localData/justinw/llvm-project/build/lib/clang/15.0.0/include -internal-isystem /usr/local/include -internal-isystem /d/sw/gcc/9.2.0/lib/gcc/x86_64-pc-linux-gnu/9.2.0/../../../../x86_64-pc-linux-gnu/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -std=c++17 -fdeprecated-macro -fdebug-compilation-dir=/localData/justinw/sycl -ferror-limit 19 -fmessage-length=230 -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -x c++ /localData/justinw/sycl/test.cpp
I[15:17:09.892] Building preamble...
I[15:17:34.123] Indexing headers...
I[15:17:39.842] Building AST...
clangd: /localData/justinw/llvm-project/clang/include/clang/AST/Type.h:691: const clang::ExtQualsTypeCommonBase* clang::QualType::getCommonPtr() const: Assertion `!isNull() && "Cannot retrieve a NULL type pointer"' failed.

Stack trace from GDB:

#0  0x00007ffff66e8337 in raise () from /lib64/libc.so.6
#1  0x00007ffff66e9a28 in abort () from /lib64/libc.so.6
#2  0x00007ffff66e1156 in __assert_fail_base () from /lib64/libc.so.6
#3  0x00007ffff66e1202 in __assert_fail () from /lib64/libc.so.6
#4  0x00000000004170c8 in clang::QualType::getCommonPtr (this=0x7fffffff31f8) at /localData/justinw/llvm-project/clang/include/clang/AST/Type.h:691
#5  0x000000000041853c in clang::QualType::getTypePtr (this=0x7fffffff31f8) at /localData/justinw/llvm-project/clang/include/clang/AST/Type.h:6503
#6  0x00000000004171ca in clang::QualType::operator-> (this=0x7fffffff31f8) at /localData/justinw/llvm-project/clang/include/clang/AST/Type.h:733
#7  0x0000000000cb6317 in clang::DecltypeType::DecltypeType (this=0xc16b3e0, E=0xc16b3b8, underlyingType=..., can=...) at /localData/justinw/llvm-project/clang/lib/AST/Type.cpp:3483
#8  0x0000000000cb6483 in clang::DependentDecltypeType::DependentDecltypeType (this=0xc16b3e0, Context=..., E=0xc16b3b8) at /localData/justinw/llvm-project/clang/lib/AST/Type.cpp:3497
#9  0x00000000008458fb in clang::ASTContext::getDecltypeType (this=0x98e5ce0, e=0xc16b3b8, UnderlyingType=...) at /localData/justinw/llvm-project/clang/lib/AST/ASTContext.cpp:5632
#10 0x0000000002b83792 in clang::serialization::AbstractTypeReader<clang::ASTRecordReader>::readDecltypeType (this=0x7fffffff3430) at tools/clang/include/clang/AST/AbstractTypeReader.inc:292
#11 0x0000000002b6bdb8 in clang::serialization::AbstractTypeReader<clang::ASTRecordReader>::read (this=0x7fffffff3430, kind=clang::Type::Decltype) at tools/clang/include/clang/AST/AbstractTypeReader.inc:45
#12 0x0000000002b3f9bf in clang::ASTReader::readTypeRecord (this=0xe3f2550, Index=64601) at /localData/justinw/llvm-project/clang/lib/Serialization/ASTReader.cpp:6448
#13 0x0000000002b42e75 in clang::ASTReader::GetType (this=0xe3f2550, ID=519208) at /localData/justinw/llvm-project/clang/lib/Serialization/ASTReader.cpp:7105
#14 0x0000000002b42faf in clang::ASTReader::getLocalType (this=0xe3f2550, F=..., LocalID=519208) at /localData/justinw/llvm-project/clang/lib/Serialization/ASTReader.cpp:7119
#15 0x0000000002b0ca5b in clang::ASTReader::readType (this=0xe3f2550, F=..., Record=..., Idx=@0x7fffffff3a98: 18) at /localData/justinw/llvm-project/clang/include/clang/Serialization/ASTReader.h:1817
#16 0x0000000002b0d08e in clang::ASTRecordReader::readType (this=0x7fffffff3a80) at /localData/justinw/llvm-project/clang/include/clang/Serialization/ASTRecordReader.h:177
#17 0x0000000002b41221 in clang::ASTRecordReader::readTypeSourceInfo (this=0x7fffffff3a80) at /localData/justinw/llvm-project/clang/lib/Serialization/ASTReader.cpp:6822
#18 0x0000000002bcd1e0 in clang::ASTDeclReader::readTypeSourceInfo (this=0x7fffffff3a30) at /localData/justinw/llvm-project/clang/lib/Serialization/ASTReaderDecl.cpp:123
#19 0x0000000002bce256 in clang::ASTDeclReader::VisitTypedefNameDecl (this=0x7fffffff3a30, TD=0xc16b2e0) at /localData/justinw/llvm-project/clang/lib/Serialization/ASTReaderDecl.cpp:679
#20 0x0000000002bce347 in clang::ASTDeclReader::VisitTypeAliasDecl (this=0x7fffffff3a30, TD=0xc16b2e0) at /localData/justinw/llvm-project/clang/lib/Serialization/ASTReaderDecl.cpp:699
#21 0x0000000002bfc6ac in clang::declvisitor::Base<std::add_pointer, clang::ASTDeclReader, void>::Visit (this=0x7fffffff3a30, D=0xc16b2e0) at tools/clang/include/clang/AST/DeclNodes.inc:329
#22 0x0000000002bcd802 in clang::ASTDeclReader::Visit (this=0x7fffffff3a30, D=0xc16b2e0) at /localData/justinw/llvm-project/clang/lib/Serialization/ASTReaderDecl.cpp:528
#23 0x0000000002bf8c1a in clang::ASTReader::ReadDeclRecord (this=0xe3f2550, ID=80954) at /localData/justinw/llvm-project/clang/lib/Serialization/ASTReaderDecl.cpp:3785
#24 0x0000000002b444fc in clang::ASTReader::GetDecl (this=0xe3f2550, ID=80954) at /localData/justinw/llvm-project/clang/lib/Serialization/ASTReader.cpp:7477
#25 0x0000000002b0ca9b in clang::ASTReader::GetLocalDecl (this=0xe3f2550, F=..., LocalID=80954) at /localData/justinw/llvm-project/clang/include/clang/Serialization/ASTReader.h:1850
#26 0x0000000002b44958 in <lambda(clang::ASTReader::ModuleFile*, clang::ASTReader::LexicalContents)>::operator()(clang::ASTReader::ModuleFile *, clang::ASTReader::LexicalContents) const (__closure=0x7fffffff3fc0, M=0x95872a0, 
    LexicalDecls=...) at /localData/justinw/llvm-project/clang/lib/Serialization/ASTReader.cpp:7557
#27 0x0000000002b44bda in clang::ASTReader::FindExternalLexicalDecls(clang::DeclContext const*, llvm::function_ref<bool (clang::Decl::Kind)>, llvm::SmallVectorImpl<clang::Decl*>&) (this=0xe3f2550, DC=0xc168af8, IsKindWeWant=..., 
    Decls=...) at /localData/justinw/llvm-project/clang/lib/Serialization/ASTReader.cpp:7571
#28 0x000000000097b4f2 in clang::ExternalASTSource::FindExternalLexicalDecls (this=0xe3f2568, DC=0xc168af8, Result=...) at /localData/justinw/llvm-project/clang/include/clang/AST/ExternalASTSource.h:185
#29 0x000000000098499f in clang::DeclContext::LoadLexicalDeclsFromExternalStorage (this=0xc168af8) at /localData/justinw/llvm-project/clang/lib/AST/DeclBase.cpp:1377
#30 0x0000000000984bc4 in clang::DeclContext::decls_begin (this=0xc168af8) at /localData/justinw/llvm-project/clang/lib/AST/DeclBase.cpp:1433
#31 0x0000000000418cf8 in clang::DeclContext::decls (this=0xc168af8) at /localData/justinw/llvm-project/clang/include/clang/AST/DeclBase.h:2127
#32 0x0000000001d14e99 in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseDeclContextHelper (this=0x7fffffff4b90, DC=0xc168af8) at /localData/justinw/llvm-project/clang/include/clang/AST/RecursiveASTVisitor.h:1416
#33 0x0000000001d0e510 in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseCXXRecordDecl (this=0x7fffffff4b90, D=0xc168ab8) at /localData/justinw/llvm-project/clang/include/clang/AST/RecursiveASTVisitor.h:1943
#34 0x0000000001d08469 in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseDecl (this=0x7fffffff4b90, D=0xc168ab8) at tools/clang/include/clang/AST/DeclNodes.inc:283
#35 0x0000000001d0d4b4 in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseClassTemplateDecl (this=0x7fffffff4b90, D=0xc168980) at /localData/justinw/llvm-project/clang/include/clang/AST/RecursiveASTVisitor.h:1821
#36 0x0000000001d08304 in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseDecl (this=0x7fffffff4b90, D=0xc168980) at tools/clang/include/clang/AST/DeclNodes.inc:227
#37 0x0000000001d14f2b in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseDeclContextHelper (this=0x7fffffff4b90, DC=0xcc348f8) at /localData/justinw/llvm-project/clang/include/clang/AST/RecursiveASTVisitor.h:1418
#38 0x0000000001d0b7d1 in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseNamespaceDecl (this=0x7fffffff4b90, D=0xcc348c8) at /localData/justinw/llvm-project/clang/include/clang/AST/RecursiveASTVisitor.h:1555
#39 0x0000000001d080a0 in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseDecl (this=0x7fffffff4b90, D=0xcc348c8) at tools/clang/include/clang/AST/DeclNodes.inc:131
#40 0x0000000001d14f2b in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseDeclContextHelper (this=0x7fffffff4b90, DC=0x9764158) at /localData/justinw/llvm-project/clang/include/clang/AST/RecursiveASTVisitor.h:1418
#41 0x0000000001d0b7d1 in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseNamespaceDecl (this=0x7fffffff4b90, D=0x9764128) at /localData/justinw/llvm-project/clang/include/clang/AST/RecursiveASTVisitor.h:1555
#42 0x0000000001d080a0 in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseDecl (this=0x7fffffff4b90, D=0x9764128) at tools/clang/include/clang/AST/DeclNodes.inc:131
#43 0x0000000001d14f2b in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseDeclContextHelper (this=0x7fffffff4b90, DC=0x97641e8) at /localData/justinw/llvm-project/clang/include/clang/AST/RecursiveASTVisitor.h:1418
#44 0x0000000001d0b7d1 in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseNamespaceDecl (this=0x7fffffff4b90, D=0x97641b8) at /localData/justinw/llvm-project/clang/include/clang/AST/RecursiveASTVisitor.h:1555
#45 0x0000000001d080a0 in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseDecl (this=0x7fffffff4b90, D=0x97641b8) at tools/clang/include/clang/AST/DeclNodes.inc:131
#46 0x0000000001d14f2b in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseDeclContextHelper (this=0x7fffffff4b90, DC=0xd9d23c0) at /localData/justinw/llvm-project/clang/include/clang/AST/RecursiveASTVisitor.h:1418
#47 0x0000000001d14277 in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseTranslationUnitDecl (this=0x7fffffff4b90, D=0xd9d2398) at /localData/justinw/llvm-project/clang/include/clang/AST/RecursiveASTVisitor.h:1519
#48 0x0000000001d08dc6 in clang::RecursiveASTVisitor<clang::CallGraph>::TraverseDecl (this=0x7fffffff4b90, D=0xd9d2398) at tools/clang/include/clang/AST/DeclNodes.inc:621
#49 0x0000000001d068a3 in clang::CallGraph::addToCallGraph (this=0x7fffffff4b90, D=0xd9d2398) at /localData/justinw/llvm-project/clang/include/clang/Analysis/CallGraph.h:70
#50 0x0000000003857391 in DeviceFunctionTracker::DeviceFunctionTracker (this=0x7fffffff4b90, S=...) at /localData/justinw/llvm-project/clang/lib/Sema/SemaSYCL.cpp:737
#51 0x0000000003863d98 in clang::Sema::MarkDevices (this=0xcfb8df0) at /localData/justinw/llvm-project/clang/lib/Sema/SemaSYCL.cpp:3941
#52 0x0000000002d56ada in clang::Sema::ActOnEndOfTranslationUnitFragment (this=0xcfb8df0, Kind=clang::Sema::Normal) at /localData/justinw/llvm-project/clang/lib/Sema/Sema.cpp:1137
#53 0x0000000002d56d97 in clang::Sema::ActOnEndOfTranslationUnit (this=0xcfb8df0) at /localData/justinw/llvm-project/clang/lib/Sema/Sema.cpp:1173
#54 0x0000000002a10a86 in clang::Parser::ParseTopLevelDecl (this=0x9fa9e20, Result=..., ImportState=@0x7fffffff5ba4: clang::Sema::ModuleImportState::FirstDecl)
    at /localData/justinw/llvm-project/clang/lib/Parse/Parser.cpp:701
#55 0x0000000002a1044b in clang::Parser::ParseFirstTopLevelDecl (this=0x9fa9e20, Result=..., ImportState=@0x7fffffff5ba4: clang::Sema::ModuleImportState::FirstDecl)
    at /localData/justinw/llvm-project/clang/lib/Parse/Parser.cpp:591
#56 0x0000000002a08129 in clang::ParseAST (S=..., PrintStats=false, SkipFunctionBodies=false) at /localData/justinw/llvm-project/clang/lib/Parse/ParseAST.cpp:161
#57 0x0000000002825a15 in clang::ASTFrontendAction::ExecuteAction (this=0x9b8f380) at /localData/justinw/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1083
#58 0x0000000002825345 in clang::FrontendAction::Execute (this=0x9b8f380) at /localData/justinw/llvm-project/clang/lib/Frontend/FrontendAction.cpp:976
#59 0x00000000015beb3a in clang::clangd::ParsedAST::build (Filename=..., Inputs=..., CI=std::unique_ptr<clang::CompilerInvocation> = {...}, CompilerInvocationDiags=..., Preamble=
    std::shared_ptr<const clang::clangd::PreambleData> (use count 2, weak count 0) = {...}) at /localData/justinw/llvm-project/clang-tools-extra/clangd/ParsedAST.cpp:552
#60 0x000000000071bba9 in clang::clangd::(anonymous namespace)::Checker::buildAST (this=0x7fffffff8f00) at /localData/justinw/llvm-project/clang-tools-extra/clangd/tool/Check.cpp:177
#61 0x000000000071cc71 in clang::clangd::check(llvm::StringRef, llvm::function_ref<bool (clang::clangd::Position const&)>, clang::clangd::ThreadsafeFS const&, clang::clangd::ClangdLSPServer::Options const&, bool) (File=..., 
    ShouldCheckLine=..., TFS=..., Opts=..., EnableCodeCompletion=false) at /localData/justinw/llvm-project/clang-tools-extra/clangd/tool/Check.cpp:285
#62 0x00000000006e6924 in main (argc=2, argv=0x7fffffffb148) at /localData/justinw/llvm-project/clang-tools-extra/clangd/tool/ClangdMain.cpp:980

System information

Output of clangd --version: clangd version 15.0.0 (https://github.com/intel/llvm.git a37ca8403341511c789f3e07768755a72befcd13)

Editor/LSP plugin: N/A in this case (tested via --check=), but normally VS Code

Operating system: CentOS 7.7

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 20 (1 by maintainers)

Commits related to this issue

Most upvoted comments

Nothing meaningful - just giving myself a crash course in LLVM, so named because I’m mostly just crashing LLVM over and over.

+2
jwake on May 5, 2022

Disclaimer: I’m totally unfamiliar with SYCL 😦 I’m pretty sure nobody’s tried this before. And we don’t really have the bandwidth to properly support all the language variants that are in-tree (OpenCL, OpenMP etc) let alone building clangd with a modified clang parser.

Happy to give some pointers, but a warning: It took a while to smooth out many bumps/crashes in clangd’s use of the clang parser, in a fork where this has been changed a lot and clangd hasn’t been tested I’d expect it to need some work.

Some ideas:

  • it’s probably first worth minimizing the input. i.e. what’s the minimum code you can reduce that header and its transitive includes to that will still crash.
  • this is crashing inside some SYCL-specific traversal (Sema::MarkDevices) that’s been added to Sema as part of each parse. Are you sure this needs to run at all in clangd?
  • it seems to be running a RecursiveASTVisitor to traverse the whole translation unit. This is at minimum going to be unacceptably slow for interactive use, as it has to deserialize the whole preamble (a PCH/module built from the transitive includes). For clangd’s own analyses we use RecursiveASTVisitor after calling ASTContext::setTraversalScope to limit the walk, but I suspect that hasn’t been called yet here.
  • the crash is happening in clang code, specifically AST deserialization code. Serialization/deserialization bugs could easily be lurking if SYCL involves changes to the AST and hasn’t been thoroughly battle-tested through use with PCHes or modules.
  • apart from serialization/PCH-related problems, I’d guess the next most significant difference is clangd’s (serialized) ASTs for headers don’t contain bodies of most functions for performance. This used to crash a lot when it was first added, and may be causing you problems.

I’m not quite even managing to get a SourceLocation to turn into an actual file + line number yet.

In general Loc.printToString(SM) should give you something useful, if it won’t something’s quite wrong. (Or you’re deeply buried in macros, where things are just inherently confusing)

+2
sam-mccall on Apr 7, 2022

As a casual viewer of this issue with the same problem,

Thanks for this fix!, @rafbiels

Works if I compile it myself and use it locally (does not appear to be in oneAPI 2024.0.2 but works with it).

Commands I used,

git clone https://github.com/intel/llvm.git
cd llvm/buildbot
python3 configure.py --ci-defaults --no-assertions -t Release
python3 compile.py -j 20 -t clang
python3 compile.py -j 20 -t sycl
python3 compile.py -j 20 -t clangd
cd ../build
sudo cp bin/clangd /usr/local/bin

Also I had to add this (but this may be a problem with my configuration):

set(CMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES ${CMAKE_CXX_IMPLICIT_INCLUDE_DIRECTORIES})
+1
mschilling0 on Feb 26, 2024
Read more comments on GitHub
← clangd: Clangd does not recognize basic headers (cpp file)
clangd: Don't run slow clang-tidy checks →