bitcoin: Failure to run Fuzz tests when running with corpus

Is there an existing issue for this?

  • I have searched the existing issues

Current behaviour

When running the fuzz tests with fuzz corpus raises an error

FUZZ=process_message build_fuzz/bin/fuzz qa-assets/fuzz_corpora/process_message/                                               ─╯
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 64371175
INFO: Loaded 1 modules   (1252320 inline 8-bit counters): 1252320 [0x1061c8000, 0x1062f9be0), 
INFO: Loaded 1 PC tables (1252320 PCs): 1252320 [0x1062f9be0,0x1076159e0), 
=================================================================
==36574==ERROR: AddressSanitizer: container-overflow on address 0x60800002c268 at pc 0x000102074ef4 bp 0x00016ddd26e0 sp 0x00016ddd26d8
WRITE of size 8 at 0x60800002c268 thread T0
    #0 0x000102074ef0 in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::__init_copy_ctor_external(char const*, unsigned long)+0x1c4 (fuzz:arm64+0x100048ef0)
    #1 0x0001057b34f8 in fuzzer::ListFilesInDirRecursive(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, long*, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>*, bool)+0x26c (fuzz:arm64+0x1037874f8)
    #2 0x0001057b27c0 in fuzzer::GetSizedFilesFromDir(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::vector<fuzzer::SizedFile, std::__1::allocator<fuzzer::SizedFile>>*)+0x2c (fuzz:arm64+0x1037867c0)
    #3 0x0001057ae338 in fuzzer::ReadCorpora(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)+0x4c (fuzz:arm64+0x103782338)
    #4 0x0001057ae1a0 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long))+0x1dbc (fuzz:arm64+0x1037821a0)
    #5 0x0001057c1aa8 in main+0x24 (fuzz:arm64+0x103795aa8)
    #6 0x00018ce70270  (<unknown module>)
    #7 0xf3547ffffffffffc  (<unknown module>)

0x60800002c268 is located 72 bytes inside of 96-byte region [0x60800002c220,0x60800002c280)
allocated by thread T0 here:
    #0 0x0001094d92c4 in _Znwm+0x6c (libclang_rt.asan_osx_dynamic.dylib:arm64+0x612c4)
    #1 0x0001025f0a5c in std::__1::__split_buffer<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>&>::__split_buffer(unsigned long, unsigned long, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>&)+0xf0 (fuzz:arm64+0x1005c4a5c)
    #2 0x000102943a48 in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>* std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>::__push_back_slow_path<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)+0x244 (fuzz:arm64+0x100917a48)
    #3 0x0001057b3468 in fuzzer::ListFilesInDirRecursive(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, long*, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>*, bool)+0x1dc (fuzz:arm64+0x103787468)
    #4 0x0001057b27c0 in fuzzer::GetSizedFilesFromDir(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::vector<fuzzer::SizedFile, std::__1::allocator<fuzzer::SizedFile>>*)+0x2c (fuzz:arm64+0x1037867c0)
    #5 0x0001057ae338 in fuzzer::ReadCorpora(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)+0x4c (fuzz:arm64+0x103782338)
    #6 0x0001057ae1a0 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long))+0x1dbc (fuzz:arm64+0x1037821a0)
    #7 0x0001057c1aa8 in main+0x24 (fuzz:arm64+0x103795aa8)
    #8 0x00018ce70270  (<unknown module>)
    #9 0xf3547ffffffffffc  (<unknown module>)

HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_container_overflow=0.
If you suspect a false positive see also: https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow.
SUMMARY: AddressSanitizer: container-overflow (fuzz:arm64+0x100048ef0) in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::__init_copy_ctor_external(char const*, unsigned long)+0x1c4
Shadow bytes around the buggy address:
  0x60800002bf80: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fd fd
  0x60800002c000: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 fa
  0x60800002c080: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fd fd
  0x60800002c100: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 fa
  0x60800002c180: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fd fd
=>0x60800002c200: fa fa fa fa 00 00 00 00 00 00 00 00 00[fc]fc fc
  0x60800002c280: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 fa
  0x60800002c300: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00
  0x60800002c380: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 fa
  0x60800002c400: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x60800002c480: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==36574==ABORTING
[3]    36574 abort      FUZZ=process_message build_fuzz/bin/fuzz 

when running with without corpus , the fuzz test runs fine.

 FUZZ=process_message build_fuzz/bin/fuzz                                                                                       ─╯
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 85124989
INFO: Loaded 1 modules   (1252320 inline 8-bit counters): 1252320 [0x10672c000, 0x10685dbe0), 
INFO: Loaded 1 PC tables (1252320 PCs): 1252320 [0x10685dbe0,0x107b799e0), 
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: A corpus is not provided, starting from an empty corpus
#2      INITED cov: 2748 ft: 2747 corp: 1/1b exec/s: 0 rss: 193Mb
#6      NEW    cov: 2754 ft: 2848 corp: 2/2b lim: 4 exec/s: 0 rss: 193Mb L: 1/1 MS: 4 ChangeBinInt-ChangeBit-CopyPart-ChangeByte-
#8      NEW    cov: 2754 ft: 2851 corp: 3/4b lim: 4 exec/s: 0 rss: 194Mb L: 2/2 MS: 2 CopyPart-InsertByte-
#11     NEW    cov: 2757 ft: 2858 corp: 4/5b lim: 4 exec/s: 0 rss: 194Mb L: 1/2 MS: 3 ChangeBit-ChangeBinInt-ChangeBit-
#26     NEW    cov: 2757 ft: 2859 corp: 5/7b lim: 4 exec/s: 0 rss: 194Mb L: 2/2 MS: 5 CrossOver-ChangeBit-ChangeByte-CrossOver-CrossOver-
#27     NEW    cov: 2758 ft: 2878 corp: 6/8b lim: 4 exec/s: 0 rss: 194Mb L: 1/2 MS: 1 ChangeByte-
#53     NEW    cov: 2758 ft: 2879 corp: 7/12b lim: 4 exec/s: 0 rss: 195Mb L: 4/4 MS: 1 CopyPart-
#278    NEW    cov: 2759 ft: 2881 corp: 8/17b lim: 6 exec/s: 0 rss: 198Mb L: 5/5 MS: 5 ShuffleBytes-ChangeByte-EraseBytes-CrossOver-CrossOver-
#364    NEW    cov: 2759 ft: 2882 corp: 9/18b lim: 6 exec/s: 0 rss: 200Mb L: 1/5 MS: 1 ChangeByte-
#615    NEW    cov: 2759 ft: 2890 corp: 10/20b lim: 8 exec/s: 0 rss: 204Mb L: 2/5 MS: 1 InsertByte-
#657    NEW    cov: 2759 ft: 2892 corp: 11/27b lim: 8 exec/s: 0 rss: 204Mb L: 7/7 MS: 2 ChangeByte-CopyPart-
#692    NEW    cov: 2759 ft: 2893 corp: 12/34b lim: 8 exec/s: 0 rss: 205Mb L: 7/7 MS: 5 InsertRepeatedBytes-InsertByte-EraseBytes-ChangeBinInt-InsertRepeatedBytes-
#1013   NEW    cov: 2759 ft: 2895 corp: 13/45b lim: 11 exec/s: 0 rss: 210Mb L: 11/11 MS: 1 InsertRepeatedBytes-
#1341   NEW    cov: 2764 ft: 2901 corp: 14/58b lim: 14 exec/s: 0 rss: 215Mb L: 13/13 MS: 3 ChangeByte-InsertRepeatedBytes-InsertRepeatedBytes-
#1347   NEW    cov: 2765 ft: 2902 corp: 15/61b lim: 14 exec/s: 0 rss: 215Mb L: 3/13 MS: 1 CrossOver-
#1378   NEW    cov: 2765 ft: 2905 corp: 16/75b lim: 14 exec/s: 0 rss: 216Mb L: 14/14 MS: 1 InsertByte-
#1716   NEW    cov: 2765 ft: 2908 corp: 17/91b lim: 17 exec/s: 0 rss: 221Mb L: 16/16 MS: 3 CrossOver-InsertRepeatedBytes-InsertRepeatedBytes-
#1754   NEW    cov: 2765 ft: 2911 corp: 18/106b lim: 17 exec/s: 0 rss: 222Mb L: 15/16 MS: 3 InsertByte-ChangeBit-CopyPart-
#1982   NEW    cov: 2766 ft: 2912 corp: 19/110b lim: 17 exec/s: 0 rss: 226Mb L: 4/16 MS: 3 ChangeBit-EraseBytes-ChangeBit-
#2405   NEW    cov: 2767 ft: 2915 corp: 20/130b lim: 21 exec/s: 0 rss: 232Mb L: 20/20 MS: 3 InsertByte-InsertRepeatedBytes-InsertRepeatedBytes-
#2418   NEW    cov: 2769 ft: 2917 corp: 21/151b lim: 21 exec/s: 0 rss: 233Mb L: 21/21 MS: 3 InsertRepeatedBytes-InsertByte-CrossOver-
#2629   REDUCE cov: 2769 ft: 2917 corp: 21/150b lim: 21 exec/s: 0 rss: 236Mb L: 6/21 MS: 1 EraseBytes-
        NEW_FUNC[1/19]: 0x0001034cb5bc in CNetAddr::IsRFC1918() const+0x0 (fuzz:arm64+0x100f3b5bc)
        NEW_FUNC[2/19]: 0x0001034cbd94 in CNetAddr::IsRFC2544() const+0x0 (fuzz:arm64+0x100f3bd94)
#3051   NEW    cov: 2881 ft: 3227 corp: 22/173b lim: 25 exec/s: 0 rss: 244Mb L: 23/23 MS: 2 InsertRepeatedBytes-InsertRepeatedBytes-
#3071   REDUCE cov: 2881 ft: 3227 corp: 22/170b lim: 25 exec/s: 0 rss: 244Mb L: 3/23 MS: 5 ChangeBit-CrossOver-CMP-CrossOver-EraseBytes- DE: "\377\377\377\377"-
        NEW_FUNC[1/11]: 0x0001025a7a5c in std::__1::vector<unsigned char, std::__1::allocator<unsigned char>>::shrink_to_fit()+0x0 (fuzz:arm64+0x100017a5c)
        NEW_FUNC[2/11]: 0x0001025a8070 in std::__1::vector<unsigned char, 

Expected behaviour

To run the fuzz tests without any error and relevant log output.

Steps to reproduce

git clone https://github.com/bitcoin-core/qa-assets
cmake --preset=libfuzzer \
   -DCMAKE_C_COMPILER="$(brew --prefix llvm)/bin/clang" \
   -DCMAKE_CXX_COMPILER="$(brew --prefix llvm)/bin/clang++" \
   -DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld"

cmake --build build_fuzz -j$(sysctl -n hw.ncpu)
FUZZ=process_message build_fuzz/bin/fuzz qa-assets/fuzz_corpora/process_message/      

Relevant log output

NA

How did you obtain Bitcoin Core

Compiled from source

What version of Bitcoin Core are you using?

master @83a9e55ae1

Operating system and version

MacOS 15.3.1

Machine specifications

No response

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 19 (19 by maintainers)

Most upvoted comments

I could reproduce it following the same steps on MacOS 14.3

FUZZ=process_message build_fuzz/bin/fuzz qa-assets/fuzz_corpora/process_message/
fuzz(56832,0x1e31a5c40) malloc: nano zone abandoned due to inability to reserve vm space.
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 2192399851
INFO: Loaded 1 modules   (1252322 inline 8-bit counters): 1252322 [0x104c38000, 0x104d69be2),
INFO: Loaded 1 PC tables (1252322 PCs): 1252322 [0x104d69be8,0x106085a08),
=================================================================
==56832==ERROR: AddressSanitizer: container-overflow on address 0x60800002ca68 at pc 0x000100adcef4 bp 0x00016f36aa20 sp 0x00016f36aa18
WRITE of size 8 at 0x60800002ca68 thread T0
    #0 0x000100adcef0 in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::__init_copy_ctor_external(char const*, unsigned long)+0x1c4 (fuzz:arm64+0x100048ef0)
    #1 0x00010421b5bc in fuzzer::ListFilesInDirRecursive(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, long*, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>*, bool)+0x26c (fuzz:arm64+0x1037875bc)
    #2 0x00010421a884 in fuzzer::GetSizedFilesFromDir(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::vector<fuzzer::SizedFile, std::__1::allocator<fuzzer::SizedFile>>*)+0x2c (fuzz:arm64+0x103786884)
    #3 0x0001042163fc in fuzzer::ReadCorpora(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)+0x4c (fuzz:arm64+0x1037823fc)
    #4 0x000104216264 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long))+0x1dbc (fuzz:arm64+0x103782264)
    #5 0x000104229b6c in main+0x24 (fuzz:arm64+0x103795b6c)
    #6 0x00018c5590dc  (<unknown module>)
    #7 0x9f697ffffffffffc  (<unknown module>)

For reference: https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow#false-positives