bitcoin: bench: `linearizeoptimallyexample11` benchmark now running 4x slow than previously
According to https://corecheck.dev/benchmarks, the linearizeoptimallyexample11 benchmark 4x’d it’s execution time at some point in the last few days.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 20 (20 by maintainers)
Commits related to this issue
- Merge bitcoin/bitcoin#31191: build: Make G_FUZZING constexpr, require -DBUILD_FOR_FUZZING=ON to fuzz fafbf8acf419d5e2ca307e5804099361ca7471af Make G_FUZZING constexpr, require -DBUILD_FOR_FUZZING=ON ... — committed to bitcoin/bitcoin by ryanofsky 2 years ago
- doc: emphasize that `Assume` always evaluates We've been surprised multiple times by `Assume` doing heavy computations in `release`: * https://github.com/bitcoin/bitcoin/issues/31178#issuecomment-244... — committed to l0rinc/bitcoin by l0rinc 2 years ago
After testing this, it does seem like, in practice, the slowdown comes from checking the
g_fuzzingvariable, not from evaluating the assume condition. I don’t agree with (or don’t understand) Marco’s logic that this was neccesarily the case, since before #31093 if the compiler knew evaluating the assume condition didn’t have side effects and that result of the assume condition was not used, it would have no reason to evaluate it before #31093, but would need to evaluate it after #31093, so an optimization that would skip evaluating the assume condition wheng_fuzzingis false could result in a speedup.But in practice, the speedup that comes from doing this is relatively minor. When I run the LinearizeOptimallyExample11 benchmark on master, I get around 1.4 operations per second. With a change to skip evaluating the assume condition when
g_fuzzingis false, this increases to 1.5 operations per second. But if I revert #31093, it increases to 1.8 operation to second.Change I was tested with was:
diff
The slowdown appears to be introduced with https://github.com/bitcoin/bitcoin/commit/1c7ca6e64de9b47b2295c81cb0fedd432ffaf001
Not sure why yet.
In any case https://github.com/bitcoin/bitcoin/pull/31191 should fix the recently introduced
g_fuzzingregression. If there are suggestions for other (more fundamental) changes or alternative code patches, separate pull requests would probably be best going forward. I am happy to review any of them, if they exist.@maflcko Worth trying, but I have been using
Assumein many places assuming it’ll be optimized out entirely in production code. If we leave thisg_fuzzingin place, we will need to re-evaluate whether all thoseAssumes are still worth having if they have a runtime impact.Could we obviate in the DrahtBot’s response that the link doesn’t just provide a
test coverage report- since the regression was already visible there: https://corecheck.dev/bitcoin/bitcoin/pulls/31093@m3dwards Thanks for digging! I believe that explains it: since #31093,
Assume()calls are no longer optimized out in production builds.