openssl: crash in ssl3_read_bytes/ tls_get_message_header

used version is 3.0.7-24 als provided in RHEL9. I get this crash from time to time:

(gdb) where
#0  0x00007f26bb0e14df in tls_get_message_header (mt=<synthetic pointer>, s=0x7f2684001940) at ssl/statem/statem_lib.c:1167
#1  read_state_machine (s=0x7f2684001940) at ssl/statem/statem.c:587
#2  state_machine (s=0x7f2684001940, server=<optimized out>) at ssl/statem/statem.c:442
#3  0x00007f26bb0d2969 in ssl3_read_bytes (s=<optimized out>, type=23, recvd_type=0x0, buf=0x7f25b0000d48 "", len=8192, peek=0, readbytes=0x7f25e69f39f0) at ssl/record/rec_layer_s3.c:1711
#4  0x00007f26bb0ab7fc in ssl3_read_internal (s=0x7f2684001940, buf=0x7f25b0000d48, len=8192, peek=0, readbytes=0x7f25e69f39f0) at ssl/s3_lib.c:4462
#5  0x00007f26bb0b2137 in SSL_read (s=<optimized out>, buf=<optimized out>, num=<optimized out>) at ssl/ssl_lib.c:1885

it looks like it does some kind of reset in ssl3_read_bytes() here:

   /*
     * Unexpected handshake message (ClientHello, NewSessionTicket (TLS1.3) or
     * protocol violation)
     */
    if ((s->rlayer.handshake_fragment_len >= 4)
            && !ossl_statem_get_in_handshake(s)) {
        int ined = (s->early_data_state == SSL_EARLY_DATA_READING);

        /* We found handshake data, so we're going back into init */
        ossl_statem_set_in_init(s, 1);

        i = s->handshake_func(s);

but it has been forgotten that init_buf is freed and reset to NULL (added around 2020).

About this issue

Original URL
State: closed
Created 4 months ago
Comments: 15 (6 by maintainers)

Most upvoted comments

While investigating further, I found even more such problems. Especially when enabling bidirectional mode there seems to be a blocking read which hangs around while a send is done. But now I wonder how this actually worked for 10 days with heavy load 😃 But I think it’s clear now, that this is not an openssl problem, but a usage problem.

mteske on Feb 23, 2024

yes it is multithreaded (the omniORB library). It’s supposed to be one thread per connection or locked properly, but of course it could be a bug there as well. Maybe it’s best if I put in some logging and try to recreate the crash and then either post it here or close the issue 😃

mteske on Feb 21, 2024