AFLplusplus: Android Frida-mode Exec Speed & Stability Regressions

Describe the bug Recent versions of afl++ have dramatically lower exec speed and stability in Android frida-mode.

In the provided example, no changes were made to the harness. The only difference between runs is using different versions of afl++. Frida version is locked at 16.0.13 between all tests shown here. Separately, I also tried various updated versions of Frida and that did not appear to make a difference.

I followed the Quarkslab frida-mode tutorial (https://blog.quarkslab.com/android-greybox-fuzzing-with-afl-frida-mode.html) that is suggested on the afl++ website here: https://aflplus.plus/docs/tutorials/. Harness and configuration here: https://github.com/quarkslab/android-fuzzing.

Performance using the open-source android-fuzzing wlinked harness and afl.js (supporting screenshots for reference):

afl++ Version Exec Speed Stability
dev-branch (9cb3fe9) ~100/s ~75%
4.31c (9cac7ce) ~100/s ~75%
4.30c (4f53803) ~100/s ~75%
4.21c (9f6b012) ~30k/s ~75%
4.10c (775861e) ~30k/s ~75%
4.09c (108fb0b) ~26k/s 100%

To Reproduce

gh repo clone quarkslab/android-fuzzing
cd android-fuzzing
export NDK_PATH=<HERE>
 
adb push apk/qb.blogfuzz.apk /data/local/tmp/
adb push native/lib/libblogfuzz.so /data/local/tmp/
 
pushd jenv
cmake -DANDROID_PLATFORM=31 -DCMAKE_TOOLCHAIN_FILE=${NDK_PATH}/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a .
make
adb push libjenv.so /data/local/tmp/
popd
 
pushd wlinked_jni
mkdir -p build && cd build
cmake -DANDROID_PLATFORM=31 -DCMAKE_TOOLCHAIN_FILE=${NDK_PATH}/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a ..
make
adb push fuzz /data/local/tmp/
adb push ../afl.js /data/local/tmp/
popd
 
adb shell su -c "cd /data/local/tmp/ && mkdir -p /data/local/tmp/in && su -c rm -rf /data/local/tmp/out/* && su -c dd if=/dev/urandom of=/data/local/tmp/in/sample.bin bs=1 count=16 && /data/local/tmp/afl-fuzz -O -G 256 -i in -o out /data/local/tmp/fuzz"

Expected behavior Expect stability and performance to roughly stay the same or improve.

Screen output/Screenshots

Image

Image

Image

Image

Image

Image

Additional context Add any other context about the problem here.

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 21 (21 by maintainers)

Commits related to this issue

Most upvoted comments

Might be worth making one PR for arm64, so we can land that right away and you can get on. Then another for the other architectures and I’ll try and get round to testing them next week.

I’ll take a look at the performance issue tomorrow. I’m not sure about the stability though.

The issue says that FRIDA was fixed at 16.0.13 for all the tests? And I don’t think there have been any changes to frida_mode in AFL++ for some time either. I wonder if the issue is that the execution speed of the target is inherently dependent on the input and that each run (given how short they are in the screenshots) is being influenced by that?

It might be useful to change the harness to run each iteration with a fixed input (and essentially ignore the input from the fuzzer) to get a more accurate comparison of performance?

That said, there is a marked difference between 4.21c and 4.30c, so if the issue persists then that might be an area to look? Perhaps a bisect could help identify a regression if there is one?