openssl: Wrong MAC calculation for certain plaintext lengths with AES128_CBC_SHA256 ciphersuites in TLS

When connecting to openssl server with a vanilla connection (i.e. no Encrypt then MAC, no Extended Master Secret) and TLS_*_AES_128_CBC_SHA256 ciphersuites, OpenSSL generates records with incorrect MAC, but only for certain sizes. The ones I’ve tested were between 1000-1024 bytes and 2000-2048 bytes (inclusive). Of those 2032-2047 and 1008-1023 (inclusive) generate incorrect MAC values.

Reproducer:

cat > make-files.py << EOF
for i in range(1000, 1025):
    with open("data-{0}.txt".format(i), "wb+") as f:
        f.write(b"A" * (i-1) + b"\n")
EOF
python3 make-files.py
openssl req -x509 -newkey rsa -keyout /tmp/localhost.key -out /tmp/localhost.crt -subj /CN=localhost -nodes -batch
openssl s_server -key /tmp/localhost.key -cert /tmp/localhost.crt -HTTP -cipher @SECLEVEL=0:AES128-SHA256

in another console:

git clone https://github.com/tomato42/tlsfuzzer
pushd tlsfuzzer
# won't be needed after https://github.com/tlsfuzzer/tlsfuzzer/pull/762 gets merged
git checkout length-testing
git clone https://github.com/tlsfuzzer/tlslite-ng .tlslite-ng
ln -s .tlslite-ng/tlslite tlslite
git clone https://github.com/tlsfuzzer/python-ecdsa .python-ecdsa
ln -s .python-ecdsa/src/ecdsa ecdsa
PYTHONPATH=. python3 scripts/test-lengths.py

(alternatively the server can be set up with either ECDHE-RSA-AES128-SHA256 or DHE-RSA-AES128-SHA256 ciphersuite and tlsfuzzer script executed with the -d option)

tlsfuzzer output:

...
length: 1008 ...
Error encountered while processing node <tlsfuzzer.expect.ExpectApplicationData object at 0x7f061e6d0cd0> (child: <tlsfuzzer.messages.AlertGenerator object at 0x7f061e6d0d00>) with last message being: None
Error while processing
Traceback (most recent call last):
  File "scripts/test-lengths.py", line 229, in main
    runner.run()
  File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/runner.py", line 196, in run
    header, parser = self.state.msg_sock.\
  File "/home/hkario/dev/tlsfuzzer/tlslite/messagesocket.py", line 102, in recvMessageBlocking
    for res in self.recvMessage():
  File "/home/hkario/dev/tlsfuzzer/tlslite/messagesocket.py", line 84, in recvMessage
    for ret in self.recvRecord():
  File "/home/hkario/dev/tlsfuzzer/tlslite/recordlayer.py", line 927, in recvRecord
    data = self._decryptThenMAC(header.type, data)
  File "/home/hkario/dev/tlsfuzzer/tlslite/recordlayer.py", line 702, in _decryptThenMAC
    raise TLSBadRecordMAC()
tlslite.errors.TLSBadRecordMAC

length: 1010 ...
Error encountered while processing node <tlsfuzzer.expect.ExpectApplicationData object at 0x7f061e6d5730> (child: <tlsfuzzer.messages.AlertGenerator object at 0x7f061e6d5760>) with last message being: None
Error while processing
Traceback (most recent call last):
  File "scripts/test-lengths.py", line 229, in main
    runner.run()
  File "/home/hkario/dev/tlsfuzzer/tlsfuzzer/runner.py", line 196, in run
    header, parser = self.state.msg_sock.\
  File "/home/hkario/dev/tlsfuzzer/tlslite/messagesocket.py", line 102, in recvMessageBlocking
    for res in self.recvMessage():
  File "/home/hkario/dev/tlsfuzzer/tlslite/messagesocket.py", line 84, in recvMessage
    for ret in self.recvRecord():
  File "/home/hkario/dev/tlsfuzzer/tlslite/recordlayer.py", line 927, in recvRecord
    data = self._decryptThenMAC(header.type, data)
  File "/home/hkario/dev/tlsfuzzer/tlslite/recordlayer.py", line 702, in _decryptThenMAC
    raise TLSBadRecordMAC()
tlslite.errors.TLSBadRecordMAC
...

====================
version: 1
====================
TOTAL: 27
SKIP: 0
PASS: 11
XFAIL: 0
FAIL: 16
XPASS: 0
====================
FAILED:
        'length: 1008'
        'length: 1009'
        'length: 1010'
        'length: 1011'
        'length: 1012'
        'length: 1013'
        'length: 1014'
        'length: 1015'
        'length: 1016'
        'length: 1017'
        'length: 1018'
        'length: 1019'
        'length: 1020'
        'length: 1021'
        'length: 1022'
        'length: 1023'

While debugging it, @beldmit and me suspect the assembly aesni_cbc_sha256_enc() method.

Update: I’ve tested all lengths between 1 and 8192, and the ones that fail are: 240-255, 496-511, 752-767, etc. every 256

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 1
Comments: 17 (16 by maintainers)

Commits related to this issue

Correct processing of AES-SHA stitched ciphers Fixes: #15706 — committed to beldmit/openssl by beldmit 3 years ago
Correct processing of AES-SHA stitched ciphers Fixes: #15706 Reviewed-by: Tomas Mraz <tomas@openssl.org> Reviewed-by: Shane Lontis <shane.lontis@oracle.com> Reviewed-by: Paul Dale <pauli@openssl.org... — committed to devnexen/openssl by beldmit 3 years ago
s_server: correctly handle 2^14 byte long records as the code uses BIO_gets, and it always null terminates the strings it reads, when it reads a record 2^14 byte long, it actually returns 2^14-1 byte... — committed to tomato42/openssl by tomato42 2 years ago
s_server: correctly handle 2^14 byte long records as the code uses BIO_gets, and it always null terminates the strings it reads, when it reads a record 2^14 byte long, it actually returns 2^14-1 byte... — committed to openssl/openssl by tomato42 2 years ago
s_server: correctly handle 2^14 byte long records as the code uses BIO_gets, and it always null terminates the strings it reads, when it reads a record 2^14 byte long, it actually returns 2^14-1 byte... — committed to tomato42/openssl by tomato42 2 years ago
s_server: correctly handle 2^14 byte long records as the code uses BIO_gets, and it always null terminates the strings it reads, when it reads a record 2^14 byte long, it actually returns 2^14-1 byte... — committed to openssl/openssl by tomato42 2 years ago

Most upvoted comments

Found it!!!

https://github.com/openssl/openssl/blob/f77208693ec3bda99618e6f76c0f8d279c0077bb/providers/implementations/ciphers/cipher_aes_cbc_hmac_sha256_hw.c#L736

should be p[aad_len - 2] = len >> 8;

beldmit on Jun 14, 2021