bitcoin: build: x86 afl++ ASan build broken "error: inline assembly requires more registers than available"
Dockerfile to reproduce: https://gist.github.com/dergoegge/fc97743028f60719759b5498f5f022bf
Building with clang-19 and -fsantize=address fails with:
7.574 /bitcoin/src/secp256k1/src/scalar_4x64_impl.h:356:5: error: inline assembly requires more registers than available
7.574 356 | "movq 32(%%rsi), %%r11\n"
7.574 | ^
7.575 /bitcoin/src/secp256k1/src/scalar_4x64_impl.h:356:5: error: inline assembly requires more registers than available
7.575 /bitcoin/src/secp256k1/src/scalar_4x64_impl.h:356:5: error: inline assembly requires more registers than available
7.643 [ 24%] Building CXX object src/CMakeFiles/bitcoin_consensus.dir/uint256.cpp.o
7.722 3 errors generated.
7.738 gmake[3]: *** [src/secp256k1/src/CMakeFiles/secp256k1.dir/build.make:76: src/secp256k1/src/CMakeFiles/secp256k1.dir/secp256k1.c.o] Error 1
7.738 gmake[2]: *** [CMakeFiles/Makefile2:1020: src/secp256k1/src/CMakeFiles/secp256k1.dir/all] Error 2
Since this looks secp256k1 related I tried reproducing on a separate secp build but couldn’t. This could very well be a compiler/linker bug.
I am surprised that I’m the only one running into this, we don’t have a x86 ASan job anywhere? I think I first observed this sometime mid last year.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 25 (25 by maintainers)
Commits related to this issue
- docker: Temporarily don't use ASan See https://github.com/bitcoin/bitcoin/issues/31913 — committed to dergoegge/fuzzamoto by dergoegge 2 years ago
- Apply patch https://github.com/bitcoin/bitcoin/issues/31913#issuecomment-2843487857 — committed to davidgumberg/bitcoin by davidgumberg 2 years ago
- crypto: disable ASan for sha256_sse4 with Clang This also fails to compile when optimisations are being used, see: https://github.com/bitcoin/bitcoin/issues/31913. So disable ASan under any optimisat... — committed to fanquake/bitcoin by fanquake 2 years ago
- crypto: disable ASan for sha256_sse4 with Clang This also fails to compile when optimisations are being used, see: https://github.com/bitcoin/bitcoin/issues/31913. So disable ASan under any optimisat... — committed to fanquake/bitcoin by fanquake 2 years ago
- crypto: disable ASan for sha256_sse4 with Clang This can alsofail to compile when optimisations are being used, see: https://github.com/bitcoin/bitcoin/issues/31913. So disable just ASan for this fun... — committed to fanquake/bitcoin by fanquake 2 years ago
- Merge bitcoin/bitcoin#32437: crypto: disable ASan for sha256_sse4 with Clang 4e8ab5e00fa72016a7ec0e0505ca025d4e59e4d8 crypto: disable ASan for sha256_sse4 with Clang (fanquake) Pull request descript... — committed to bitcoin/bitcoin by fanquake 2 years ago
- crypto: disable ASan for sha256_sse4 with Clang This can alsofail to compile when optimisations are being used, see: https://github.com/bitcoin/bitcoin/issues/31913. So disable just ASan for this fun... — committed to fanquake/bitcoin by fanquake 2 years ago
- crypto: disable ASan for sha256_sse4 with Clang This can alsofail to compile when optimisations are being used, see: https://github.com/bitcoin/bitcoin/issues/31913. So disable just ASan for this fun... — committed to luke-jr/bitcoin by fanquake 2 years ago
Reproduced as follows:
Here is the configure line, and noteworthy output:
Full configure output
And, (modifying the
Dockerfileto make the cmake--build--verbose) here is the build command that fails:My guess is that
/AFLplusplus/afl-clang-fast++fails to handle the-fcf-protection=full -fcf-protection=noneflags properly.It failed with this build command:
Now that we’ve removed
ENABLE_HARDENINGthis is happening again. The suggested workaround of injecting flags manually viaAPPEND_*does not work.This is an issue for me with https://github.com/dergoegge/fuzzamoto and it’'ll also be a problem on oss-fuzz, when they bump their afl++ version.
So turns out newer versions of afl++ (i.e. newer than the one oss-fuzz is using) will insert
-fltowhen-fcf-protectionis detected. That results in-flto -fcf-protection -fsanitize=addressand for some reason that combination doesn’t work, not sure if it’s just not supported or there’s a bug but for us a simple workaround is to simply set-DENABLE_HARDENING=OFFwhen fuzzing with afl++ and ASan on x86. The PR that changed this on the afl++ side is: https://github.com/AFLplusplus/AFLplusplus/pull/1966.In conclusion, for a plain clang build on x86 with ASan use either
-O1or disable the secp256k1 custom assembly and for aafl-clang-*build with ASan use our-DENABLE_HARDENING=OFFcmake option (this will become relevant for our oss-fuzz build once they bump afl++).I don’t think there is anything for us left to do here, so closing.
I believe the explanation here is just that they’re incompatible. The asm code in libsecp256k1 requires many registers; an amount which is generally available in builds. But presumably, asan instrumentation reduces the set of general-purpose available register from use in the instrumentation itself, and as a result the code doesn’t compile anymore.