AFLplusplus: Android Frida-mode Exec Speed & Stability Regressions
Describe the bug Recent versions of afl++ have dramatically lower exec speed and stability in Android frida-mode.
In the provided example, no changes were made to the harness. The only difference between runs is using different versions of afl++. Frida version is locked at 16.0.13 between all tests shown here. Separately, I also tried various updated versions of Frida and that did not appear to make a difference.
I followed the Quarkslab frida-mode tutorial (https://blog.quarkslab.com/android-greybox-fuzzing-with-afl-frida-mode.html) that is suggested on the afl++ website here: https://aflplus.plus/docs/tutorials/. Harness and configuration here: https://github.com/quarkslab/android-fuzzing.
Performance using the open-source android-fuzzing wlinked harness and afl.js (supporting screenshots for reference):
| afl++ Version | Exec Speed | Stability |
|---|---|---|
| dev-branch (9cb3fe9) | ~100/s | ~75% |
| 4.31c (9cac7ce) | ~100/s | ~75% |
| 4.30c (4f53803) | ~100/s | ~75% |
| 4.21c (9f6b012) | ~30k/s | ~75% |
| 4.10c (775861e) | ~30k/s | ~75% |
| 4.09c (108fb0b) | ~26k/s | 100% |
To Reproduce
gh repo clone quarkslab/android-fuzzing
cd android-fuzzing
export NDK_PATH=<HERE>
adb push apk/qb.blogfuzz.apk /data/local/tmp/
adb push native/lib/libblogfuzz.so /data/local/tmp/
pushd jenv
cmake -DANDROID_PLATFORM=31 -DCMAKE_TOOLCHAIN_FILE=${NDK_PATH}/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a .
make
adb push libjenv.so /data/local/tmp/
popd
pushd wlinked_jni
mkdir -p build && cd build
cmake -DANDROID_PLATFORM=31 -DCMAKE_TOOLCHAIN_FILE=${NDK_PATH}/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a ..
make
adb push fuzz /data/local/tmp/
adb push ../afl.js /data/local/tmp/
popd
adb shell su -c "cd /data/local/tmp/ && mkdir -p /data/local/tmp/in && su -c rm -rf /data/local/tmp/out/* && su -c dd if=/dev/urandom of=/data/local/tmp/in/sample.bin bs=1 count=16 && /data/local/tmp/afl-fuzz -O -G 256 -i in -o out /data/local/tmp/fuzz"
Expected behavior Expect stability and performance to roughly stay the same or improve.
Screen output/Screenshots
Additional context Add any other context about the problem here.
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 21 (21 by maintainers)
Commits related to this issue
- Addressed frida-mode exec-speed regression on ARM64 for #2298 Co-authored-by: WorksButNotTested <@WorksButNotTested> — committed to etanner1/AFLplusplus by etanner1 2 years ago
- Merge pull request #2301 from etanner1/fix-issue-2298 Addressed frida-mode exec speed regression on ARM64 for #2298 — committed to AFLplusplus/AFLplusplus by vanhauser-thc 2 years ago
Might be worth making one PR for arm64, so we can land that right away and you can get on. Then another for the other architectures and I’ll try and get round to testing them next week.
I’ll take a look at the performance issue tomorrow. I’m not sure about the stability though.
The issue says that FRIDA was fixed at
16.0.13for all the tests? And I don’t think there have been any changes tofrida_modein AFL++ for some time either. I wonder if the issue is that the execution speed of the target is inherently dependent on the input and that each run (given how short they are in the screenshots) is being influenced by that?It might be useful to change the harness to run each iteration with a fixed input (and essentially ignore the input from the fuzzer) to get a more accurate comparison of performance?
That said, there is a marked difference between
4.21cand4.30c, so if the issue persists then that might be an area to look? Perhaps a bisect could help identify a regression if there is one?