bitcoin: bench: `linearizeoptimallyexample11` benchmark now running 4x slow than previously

According to https://corecheck.dev/benchmarks, the linearizeoptimallyexample11 benchmark 4x’d it’s execution time at some point in the last few days. Screenshot 2024-10-29 at 14 15 02

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 20 (20 by maintainers)

Commits related to this issue

Most upvoted comments

After testing this, it does seem like, in practice, the slowdown comes from checking the g_fuzzing variable, not from evaluating the assume condition. I don’t agree with (or don’t understand) Marco’s logic that this was neccesarily the case, since before #31093 if the compiler knew evaluating the assume condition didn’t have side effects and that result of the assume condition was not used, it would have no reason to evaluate it before #31093, but would need to evaluate it after #31093, so an optimization that would skip evaluating the assume condition when g_fuzzing is false could result in a speedup.

But in practice, the speedup that comes from doing this is relatively minor. When I run the LinearizeOptimallyExample11 benchmark on master, I get around 1.4 operations per second. With a change to skip evaluating the assume condition when g_fuzzing is false, this increases to 1.5 operations per second. But if I revert #31093, it increases to 1.8 operation to second.

Change I was tested with was:

diff

--- a/src/util/check.h
+++ b/src/util/check.h
@@ -41,21 +41,24 @@ T&& inline_check_non_fatal(LIFETIMEBOUND T&& val, const char* file, int line, co
 void assertion_fail(std::string_view file, int line, std::string_view func, std::string_view assertion);
 
 /** Helper for Assert()/Assume() */
-template <bool IS_ASSERT, typename T>
+template <typename T>
 constexpr T&& inline_assertion_check(LIFETIMEBOUND T&& val, [[maybe_unused]] const char* file, [[maybe_unused]] int line, [[maybe_unused]] const char* func, [[maybe_unused]] const char* assertion)
 {
-    if (IS_ASSERT || std::is_constant_evaluated() || g_fuzzing
-#ifdef ABORT_ON_FAILED_ASSUME
-        || true
-#endif
-    ) {
         if (!val) {
             assertion_fail(file, line, func, assertion);
         }
-    }
     return std::forward<T>(val);
 }
 
+constexpr bool AbortOnFailedAssume() {
+#ifdef ABORT_ON_FAILED_ASSUME
+   return true;
+#endif
+   if (std::is_constant_evaluated()) return true;
+   if (g_fuzzing) [[unlikely]] return true;
+   return false;
+}
+
 // All macros may use __func__ inside a lambda, so put them under nolint.
 // NOLINTBEGIN(bugprone-lambda-function-name)
 
@@ -76,7 +79,7 @@ constexpr T&& inline_assertion_check(LIFETIMEBOUND T&& val, [[maybe_unused]] con
     inline_check_non_fatal(condition, __FILE__, __LINE__, __func__, #condition)
 
 /** Identity function. Abort if the value compares equal to zero */
-#define Assert(val) inline_assertion_check<true>(val, __FILE__, __LINE__, __func__, #val)
+#define Assert(val) inline_assertion_check(val, __FILE__, __LINE__, __func__, #val)
 
 /**
  * Assume is the identity function.
@@ -88,7 +91,7 @@ constexpr T&& inline_assertion_check(LIFETIMEBOUND T&& val, [[maybe_unused]] con
  * - For non-fatal errors in interactive sessions (e.g. RPC or command line
  *   interfaces), CHECK_NONFATAL() might be more appropriate.
  */
-#define Assume(val) inline_assertion_check<false>(val, __FILE__, __LINE__, __func__, #val)
+#define Assume(val) inline_assertion_check(!AbortOnFailedAssume() || (val), __FILE__, __LINE__, __func__, #val)
 
 /**
  * NONFATAL_UNREACHABLE() is a macro that is used to mark unreachable code. It throws a NonFatalCheckError.

The slowdown appears to be introduced with https://github.com/bitcoin/bitcoin/commit/1c7ca6e64de9b47b2295c81cb0fedd432ffaf001

Not sure why yet.

In any case https://github.com/bitcoin/bitcoin/pull/31191 should fix the recently introduced g_fuzzing regression. If there are suggestions for other (more fundamental) changes or alternative code patches, separate pull requests would probably be best going forward. I am happy to review any of them, if they exist.

@maflcko Worth trying, but I have been using Assume in many places assuming it’ll be optimized out entirely in production code. If we leave this g_fuzzing in place, we will need to re-evaluate whether all those Assumes are still worth having if they have a runtime impact.

Could we obviate in the DrahtBot’s response that the link doesn’t just provide a test coverage report - since the regression was already visible there: https://corecheck.dev/bitcoin/bitcoin/pulls/31093

@m3dwards Thanks for digging! I believe that explains it: since #31093, Assume() calls are no longer optimized out in production builds.