oneTBB: Clang thread sanitizer (TSan) reports many errors, most tests fail

Answering my own question: it seems the error go away if I set TBB_SANITIZE to thread, which causes __TBB_USE_ITT_NOTIFY to NOT be defined.

Hmm I’m still observing a lot of TSan issues running tests in parallel with Catch2 and OneTBB. I’ll try to provide a more concrete example in the upcoming weeks.

I would have thought threading correctness issues in a threading library would be high priority. 😃 In my experience, TSan makes these otherwise hard to debug and hard to reproduce issues much easier to fix, has that not been the case with TBB?

Definitely threading correctness issues is high priority; however, C++ races reported with Thread Sanitizer are not always correctness issues on real hardware. Back to 2006 when the first version of TBB was released, C++ did not know anything about threading, i.e. any threaded application had UB from pre-C++11 perspective. In such environment, TBB (and other threading solutions) was build based on experience, practice and knowledge about target architecture. The first supported platform was x86 with its TSO memory model. It is relatively strong model, i.e. any properly aligned int provides release-acquire semantics for any store-load pair. This basic assumption became fundamental thing inside TBB, especially in tbbmalloc. With new platforms (like IA-64, ARM and others) in pre-C++11 environment, the implementation became even more complicated but remained correct for each particular platform.

With C++11, we started moving from practice based approaches to C++11 memory model. With oneTBB, we replaced all tbb::atomic with std::atomic that allowed us using such tools like Thread Sanitizer. However, not all synchronizations were based on tbb::atomics, some of them are still based on primitives types (known to work correctly) but they produce C++ races that Thread Sanitizer detects. Much of them does not contribute into correctness issues; however, we pay attention to each reported race because several issues were found and fixed that might affect correctness.

I’m not sure what tbbmalloc is, is it some build option that can be disabled?

tbbmalloc is a scalable allocator that is supposed to improve TBB performance in heavy threaded applications. It is optional but I do not think that TBB cmake has an option to disable the build of tbbmalloc. There are two approaches to try tests without tbbmalloc:

• build only required tests, e.g. make conformance_parallel_for test_parallel_for
• build all but remove libtbbmalloc* files from build directory