math: lgamma is not thread safe

Description

We have moved away from using boost functions for most math specialty functions in commit 01bb6645815cd5546701c0575fcb4f38d4a41cca. This affected in particular the log gamma function. Instead we now use the std::lgamma functions which turn out to cause problems in threading contexts. The behaviour is generally undefined wrt to threading safety, see here.

On macOS Mojave the lgamma implementation of clang will introduce locking of a global mutex as can be seen by terrible scaling of multi-threaded benchmarks, see here.

This is a large problem for any threading based applications involving Stan (parallel chains or httpstan).

Example

See the discourse thread linked above for an example.

Expected Output

We should use a thread-safe version of the lgamma - and ideally one which does not lock a global mutex which drastically limits parallelisation performance under threading. At the worst we should distinguish between threading and non-threading.

Right now the behaviour of lgamma when using threads is implementation specific and generally undefined to my understanding.

Current Version:

v2.19.1

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 34 (34 by maintainers)

Most upvoted comments