brave-browser: Repeating macOS linking errors

Some info after experiments on mac-imac3-ky.

1. The lld crash is pretty rare (around 1 in ~100 links) and only happens in lld Release/RelWithDebInfo configurations.
2. Rebuilding lld with ASan hides the crash, i.e. no sanitizer errors and no crash. Rebuilding lld in Debug configuration also hides the crash.
3. Rebuilding lld with UBSan revealed some unaligned memory access errors because of reinterpret_casts, but fixing them hasn’t solve the issue.
4. Rebuilding lld with TSan didn’t show any issues.

Properly symbolized crashes look like this:

ld64.lld(58975,0x70000f179000) malloc: *** error for object 0x6002a0484000: pointer being freed was not allocated
ld64.lld(58975,0x70000f179000) malloc: *** set a breakpoint in malloc_error_break to debug
0.	Running pass 'Function Pass Manager' on module 'Users/jenkins/jenkins/workspace/goodov-linking-errors-nightly/src/out/Release/obj/third_party/vulkan-deps/spirv-tools/src/libspvtools_opt.aamd_ext_to_khr.o6886248'.
1.	Running pass 'Debug Variable Analysis' on function '@_ZN8spvtools5utils14AppendToVectorERKNSt2Cr12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEPNS1_6vectorIjNS5_IjEEEE'
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  lld                      0x000000010d11fcee llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 46
1  lld                      0x000000010d11ee45 llvm::sys::RunSignalHandlers() + 85
2  lld                      0x000000010d120340 SignalHandler(int) + 272
3  libsystem_platform.dylib 0x00007ff809a3ae2d _sigtramp + 29
4  libsystem_platform.dylib 0x00000000000000e0 _sigtramp + 18446603370419213008
5  libsystem_c.dylib        0x00007ff809971d10 abort + 123
6  libsystem_malloc.dylib   0x00007ff80984f3e2 has_default_zone0 + 0
7  libsystem_malloc.dylib   0x00007ff8098525ed malloc_report + 151
8  lld                      0x000000010e0aafe8 llvm::LexicalScopes::~LexicalScopes() + 344
9  lld                      0x000000010e5b3183 (anonymous namespace)::LDVImpl::runOnMachineFunction(llvm::MachineFunction&, bool) + 15091
10 lld                      0x000000010e67205d llvm::MachineFunctionPass::runOnFunction(llvm::Function&) + 349
11 lld                      0x000000010f97be9f llvm::FPPassManager::runOnFunction(llvm::Function&) + 1055
12 lld                      0x000000010f982571 llvm::FPPassManager::runOnModule(llvm::Module&) + 65

0.	Running pass 'Function Pass Manager' on module 'Users/jenkins/jenkins/workspace/goodov-linking-errors-nightly/src/out/Release/obj/third_party/vulkan-deps/spirv-tools/src/libspvtools_opt.ainstrument_pass.o242484496'.
1.	Running pass 'Debug Variable Analysis' on function '@_ZN8spvtools3opt14InstrumentPass14GetVec4FloatIdEv'
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  lld                      0x000000011025316b llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 43
1  lld                      0x0000000110253534 SignalHandler(int) + 244
2  libsystem_platform.dylib 0x00007ff80d96be2d _sigtramp + 29
3  libsystem_malloc.dylib   0x00007ff80d7732a9 nanov2_size + 20
4  libsystem_c.dylib        0x00007ff80d8a2d10 abort + 123
5  libsystem_malloc.dylib   0x00007ff80d7803e2 has_default_zone0 + 0
6  libsystem_malloc.dylib   0x00007ff80d7835ed malloc_report + 151
7  lld                      0x000000010f692a95 llvm::LexicalScopes::~LexicalScopes() + 133
8  lld                      0x000000010f8a44eb (anonymous namespace)::LDVImpl::runOnMachineFunction(llvm::MachineFunction&, bool) + 2555
9  lld                      0x000000010f9255e4 llvm::MachineFunctionPass::runOnFunction(llvm::Function&) + 244

The crash happens on exit from LDVImpl::computeIntervals() in LexicalScopes::~LexicalScopes destructor: some unordered_map items fail to deallocate.

Chromium has hit a similar bug recently on Windows, but it doesn’t look like our case - they’ve got a stable reproducible assert().

I did debug the failed lld instance in lldb, but at the point of failure it’s absolutely unclear what exactly and at what point has failed. The code needs to be profiled carefully to pinpoint the exact place when things go downhill, it requires a lot of time.

Next steps

1. Report this issue to llvm, but I don’t expect a quick or easy fix for that. https://github.com/llvm/llvm-project/issues/58056
2. If no one objects, I will add a temporary fix to restart the linker inside linker_driver.py once in a case of failure when lto-enabled link is detected.

The process of building llvm with sanitizers

1. Change src/tools/clang/scripts/build.py to enable sanitizer:

@@ -852,7 +854,7 @@ def main():
     # PGO needs libclang_rt.profile but none of the other compiler-rt stuff.
     bootstrap_args.extend([
         '-D' + f
-        for f in compiler_rt_cmake_flags(sanitizers=False, profile=args.pgo)
+        for f in compiler_rt_cmake_flags(sanitizers=True, profile=args.pgo)
     ])
     if sys.platform == 'darwin':
       bootstrap_args.extend([
@@ -1197,6 +1205,7 @@ def main():
         cmake_args.append('-DRUNTIMES_' + triple + '_' + arg)
   cmake_args.append('-DLLVM_BUILTIN_TARGETS=' + all_triples)
   cmake_args.append('-DLLVM_RUNTIME_TARGETS=' + all_triples)
+  cmake_args.append('-DLLVM_USE_SANITIZER=Address')

   if os.path.exists(LLVM_BUILD_DIR):
     RmTree(LLVM_BUILD_DIR)

1. Run ./build.py --bootstrap --without-android --without-fuchsia

@wknapik https://downloadmoreram.com/ Let’s download and add some more RAM.