electron: SCCache race condition hampers *.debug file correctness

Preflight Checklist

• I have read the Contributing Guidelines for this project.
• I agree to follow the Code of Conduct that this project adheres to.
• I have searched the issue tracker for an issue that matches the one I want to file, without success.

Issue Details

• Electron Version:
  • Tested on v10.0.0-nightly.20200408
• Operating System:
  • Linux any
• Last Known Working Electron version:
  • None

Expected Behavior

Compile object output should be correct.

Actual Behavior

While testing for Linux reproducibility I ran into an issue where build-id and debug link production section of Linux x64 binaries where randomic at each build.

Checking at the relative debug file I noted that .strtab and .debug_str sections where of different size. Checking inside it was clear some symbol was messed up, expecially the mojom ones.

Since it was not clear why this was happening I run diffoscope on the out directory. This revealed that some .cc object output where going in wrong .o. . For example some A.cc was finishing in B.cc output B.o. So B.o was present twice, while A.o was never present in the final binary.

Since I did not know what was causing these files output to be messed up I runned multiple tests on filesystem/linker/sccache. I finally triaged the problem and it seems to be related to a newly discovered race condition in sccache on Linux platform.

This can be verified running a multiple builds with sccache disabled. In these case the output is not only deterministic, but it also have correct objects in correct locations.

Cc: @nornagon @ikkisoft @jkleinsc

To Reproduce

• Download https://github.com/electron/electron/blob/34a8e388fedc492d68ce00fcefd281871065f87c/script/reproducible/verify.sh
• create the directory /home/builduser/project
• run verify.sh script with ./verify.sh -b /home/builduser/project v10.0.0-nightly.20200408
• run diffoscope with --exclude-directory-metadata --text out.txt --html out.html the directories /home/builduser/project/src/out/Default{,2}

Some files will have wrong .o output.

Additional Information

This is the last issue we have on Linux platform to obtain reproducibility on all major platforms on electron (macOS, Windows, Linux). While this issue is affecting the reproducibility, incorrect sccache behaviour may affect strongly electron stability and hamper the ability use the crash reports the users are sending.