cbor2: unexpected exceptions raised while parsing untrusted inputs using cbor2.loads

I’ve fixed the problems originally reported here. I believe that, to fix all the problems thoroughly, a rewrite would be needed, but I don’t have the bandwidth for that, and I have to draw the line somewhere in order to move on to other projects. I’ve released v5.6.0 which contains these fixes.

What difference would that make, and to whom?

Good question, I would suspect that downstream consumers of this library would want to know if they’re running a potentially insecure version (i.e. < v5.6.0). The easiest way to accomplish that would probably be issuing a security advisory, which would then automatically be picked up by tools like Dependabot and Snyk 👍

Great minds think alike! I was actually fuzzing cbor2 with Atheris very recently too. However, I focused on the C implementation and looked for memory corruption bugs. I did manage to find at least one, and there may be more. I would recommend setting up regular fuzz testing for this project. Here’s what I came up with.

My Dockerfile for reproduction (note some paths may have to change, like aarch64):

FROM debian:12-slim

RUN apt update && apt install -y \
    clang \
    git \
    python3-full \
    python3-pip \
    && rm -rf /var/lib/apt/lists/*

RUN python3 --version

RUN mkdir /app
WORKDIR /app

# Subject to change by upstream
# https://github.com/google/atheris/issues/36
ENV LIBFUZZER_LIB "/usr/lib/llvm-14/lib/clang/14.0.6/lib/linux/libclang_rt.fuzzer_no_main-aarch64.a"

ENV VIRTUAL_ENV "/opt/venv"
RUN python3 -m venv $VIRTUAL_ENV
ENV PATH "$VIRTUAL_ENV/bin:$PATH"

# https://github.com/google/atheris#building-from-source
RUN python3 -m pip install --no-binary atheris atheris
RUN git clone https://github.com/google/atheris.git
RUN python3 -m pip install atheris/

# https://github.com/google/atheris/blob/master/native_extension_fuzzing.md#step-1-compiling-your-extension
ENV CC "/usr/bin/clang"
ENV CFLAGS "-fsanitize=address,undefined,fuzzer-no-link"
ENV CXX "/usr/bin/clang++"
ENV CXXFLAGS "-fsanitize=address,undefined,fuzzer-no-link"
ENV LDSHARED "/usr/bin/clang -shared"

# https://github.com/agronholm/cbor2
ENV CBOR2_BUILD_C_EXTENSION "1"
RUN git clone https://github.com/agronholm/cbor2.git
RUN python3 -m pip install cbor2/

# Subject to change by upstream, but it's just a sanity check
RUN nm "$VIRTUAL_ENV/lib/python3.11/site-packages/_cbor2.cpython-311-aarch64-linux-gnu.so" \
    | grep asan \
    && echo "Found ASAN" \
    || echo "Missing ASAN"

# Allow Atheris to find fuzzer sanitizer shared libs
# https://github.com/google/atheris/blob/master/native_extension_fuzzing.md#option-a-sanitizerlibfuzzer-preloads
ENV LD_PRELOAD "$VIRTUAL_ENV/lib/python3.11/site-packages/asan_with_fuzzer.so"

# Skip memory allocation failures for now
ENV ASAN_OPTIONS "allocator_may_return_null=1"

COPY fuzz.py fuzz.py
ENTRYPOINT ["python", "fuzz.py"]

And my fuzz harness:

#!/usr/bin/python3

import sys
import atheris

with atheris.instrument_imports():
    # _cbor2 ensures the C library is imported
    from _cbor2 import loads


# Inspired by: https://github.com/google/oss-fuzz/blob/master/projects/ujson/ujson_fuzzer.py
def TestOneInput(data):
    try:
        loads(data)
    except Exception:
        # We're searching for memory corruption, not Python exceptions
        pass


def main():
    # Since everything interesting in this fuzzer is in native code, we can
    # disable Python coverage to improve performance and reduce coverage noise.
    atheris.Setup(sys.argv, TestOneInput, enable_python_coverage=False)
    atheris.Fuzz()


if __name__ == "__main__":
    main()

Build, then run the Docker image:

$ docker build -t cbor2-fuzz -f Dockerfile
$ docker run -v $(pwd):/tmp/output/ cbor2-fuzz -artifact_prefix=/tmp/output/

This then produces a crash like:

...
SUMMARY: AddressSanitizer: SEGV /usr/include/python3.11/object.h:537:9 in Py_DECREF
==1==ABORTING
MS: 2 CMP-CrossOver- DE: "\001\010\000\000"-; base unit: 1b850b729fc7234e8dcac9406224b06d235affb7
0xae,0xae,0xae,0xae,0xae,0xae,0xae,0xae,0xae,0x1,0x8,0xc2,0x98,0x43,0xd9,0x1,0x0,0xd8,0x24,0x9f,0x0,0x0,0xae,0xae,0xff,0xc2,0x6c,0xa7,0x99,
\256\256\256\256\256\256\256\256\256\001\010\302\230C\331\001\000\330$\237\000\000\256\256\377\302l\247\231
artifact_prefix='/tmp/output/'; Test unit written to /tmp/output/crash-95e879135b949a863283a11eaf98bb7b3b109783
Base64: rq6urq6urq6uAQjCmEPZAQDYJJ8AAK6u/8Jsp5k=

Which we can confirm like so:

$ echo -n "rq6urq6urq6uAQjCmEPZAQDYJJ8AAK6u/8Jsp5k=" | python -m cbor2.tool -d
Segmentation fault: 11

This appears at the following location:

==1==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0xffff96e7cc38 bp 0xfffffec31660 sp 0xfffffec31580 T0)
==1==The signal is caused by a READ memory access.
==1==Hint: address points to the zero page.
    #0 0xffff96e7cc38 in Py_DECREF /usr/include/python3.11/object.h:537:9
    #1 0xffff96e7cc38 in decode_definite_string /app/cbor2/source/decoder.c:653:9
    #2 0xffff96e7cc38 in decode_string /app/cbor2/source/decoder.c:718:15
    #3 0xffff96e79778 in decode /app/cbor2/source/decoder.c:1735:27
...

Which seems to be this code:

https://github.com/agronholm/cbor2/blob/850545ca33c1541de397ef2e6c6e1af221d4a0f8/source/decoder.c#L653

I’m not sure about exploitability here. Memory corruption in C code has more potential for exploitation than Python exceptions. I also did notice this big warning in the Py_DECREF docs. I’m not sure if that’s applicable in this situation, but again, it’s cause for concern.

The second largest concern is that MemoryError, as it has the potential for a DoS attack.

The most concerning error is that SystemError that says it returned NULL without setting an error. This looks like a bug in the C decoder implementation.