math: lgamma is not thread safe
Description
We have moved away from using boost functions for most math specialty functions in commit 01bb6645815cd5546701c0575fcb4f38d4a41cca. This affected in particular the log gamma function. Instead we now use the std::lgamma functions which turn out to cause problems in threading contexts. The behaviour is generally undefined wrt to threading safety, see here.
On macOS Mojave the lgamma implementation of clang will introduce locking of a global mutex as can be seen by terrible scaling of multi-threaded benchmarks, see here.
This is a large problem for any threading based applications involving Stan (parallel chains or httpstan).
Example
See the discourse thread linked above for an example.
Expected Output
We should use a thread-safe version of the lgamma - and ideally one which does not lock a global mutex which drastically limits parallelisation performance under threading. At the worst we should distinguish between threading and non-threading.
Right now the behaviour of lgamma when using threads is implementation specific and generally undefined to my understanding.
Current Version:
v2.19.1
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 34 (34 by maintainers)
@bob-carpenter re: boost::math and thread safety: https://www.boost.org/doc/libs/1_70_0/libs/math/doc/html/math_toolkit/threads.html