openssl: Undefined behavior around calls to generic functions like OPENSSL_LH_HASHFUNC
We started seeing a complaint from LLVM17 sanitizer via some application nominally using OpenSSL:
crypto/lhash/lhash.c:299:12: runtime error: call to function err_string_data_hash through pointer to incorrect function type 'unsigned long (*)(const void *)'
[...]/crypto/err/err.c:184: note: err_string_data_hash defined here
#0 0x7fa569e3a434 in getrn [...]/crypto/lhash/lhash.c:299:12
#1 0x7fa569e39a46 in OPENSSL_LH_insert [...]/crypto/lhash/lhash.c:119:10
#2 0x7fa569d866ee in err_load_strings [...]/crypto/err/err.c:280:15
[...]
Strangely there were not other reports, despite the fact that obviously this idiom is used for various LHASH and in STACK also.
Unsurprisingly this seems to be impacting many other C-based projects whose authors are not especially thrilled either:
https://github.com/systemd/systemd/issues/29972 https://github.com/python/cpython/issues/111178 https://github.com/php/php-src/commit/ca22505c8ca02950385f523e6056a9e129cf2289
There seems also to even be some conversation in the original LLVM thread on whether or not this is truly UB: https://reviews.llvm.org/D148827
About this issue
- Original URL
- State: open
- Created 3 years ago
- Comments: 35 (32 by maintainers)
Commits related to this issue
- Make hash functions take void pointers and do explicit casts ubsan on clang17 has started warning about the following undefined behavior: crypto/lhash/lhash.c:299:12: runtime error: call to function... — committed to nhorman/openssl by nhorman 3 years ago
- Make hash functions take void pointers and do explicit casts ubsan on clang17 has started warning about the following undefined behavior: crypto/lhash/lhash.c:299:12: runtime error: call to function... — committed to nhorman/openssl by nhorman 3 years ago
- Introduce hash thunking functions to do proper casting ubsan on clang17 has started warning about the following undefined behavior: crypto/lhash/lhash.c:299:12: runtime error: call to function err_s... — committed to nhorman/openssl by nhorman 3 years ago
- Introduce hash thunking functions to do proper casting ubsan on clang17 has started warning about the following undefined behavior: crypto/lhash/lhash.c:299:12: runtime error: call to function err_s... — committed to nhorman/openssl by nhorman 3 years ago
- Introduce hash thunking functions to do proper casting ubsan on clang17 has started warning about the following undefined behavior: crypto/lhash/lhash.c:299:12: runtime error: call to function err_s... — committed to nhorman/openssl by nhorman 3 years ago
- Introduce hash thunking functions to do proper casting ubsan on clang17 has started warning about the following undefined behavior: crypto/lhash/lhash.c:299:12: runtime error: call to function err_s... — committed to nhorman/openssl by nhorman 3 years ago
- Introduce hash thunking functions to do proper casting ubsan on clang17 has started warning about the following undefined behavior: crypto/lhash/lhash.c:299:12: runtime error: call to function err_s... — committed to nhorman/openssl by nhorman 3 years ago
- amend! Introduce hash thunking functions to do proper casting Introduce hash thunking functions to do proper casting ubsan on clang17 has started warning about the following undefined behavior: cry... — committed to nhorman/openssl by nhorman 3 years ago
- Introduce hash thunking functions to do proper casting ubsan on clang17 has started warning about the following undefined behavior: crypto/lhash/lhash.c:299:12: runtime error: call to function err_s... — committed to nhorman/openssl by nhorman 3 years ago
- amend! Introduce hash thunking functions to do proper casting Introduce hash thunking functions to do proper casting ubsan on clang17 has started warning about the following undefined behavior: cry... — committed to nhorman/openssl by nhorman 3 years ago
- Introduce hash thunking functions to do proper casting ubsan on clang17 has started warning about the following undefined behavior: crypto/lhash/lhash.c:299:12: runtime error: call to function err_s... — committed to openssl/openssl by nhorman 3 years ago
- sanitizer: suppress sanitizer issue in openssl ../openssl/providers/implementations/encode_decode/decode_der2key.c:316:5: runtime error: call to function EC_KEY_free through pointer to incorrect func... — committed to dotnwat/redpanda by dotnwat 2 years ago
- sanitizer: suppress sanitizer issue in openssl ../openssl/providers/implementations/encode_decode/decode_der2key.c:316:5: runtime error: call to function EC_KEY_free through pointer to incorrect func... — committed to dotnwat/redpanda by dotnwat 2 years ago
- Introduce thunking to EVP interface functions related to #22896. — committed to fwh-dc/oqs-openssl by fwh-dc 2 years ago
- Pass functions with correct signatures to the evp_generic_fetch_xxx family of methods. UBSan complains about functions being called with incorrect signatures. Relates to #22896. — committed to fwh-dc/oqs-openssl by fwh-dc 2 years ago
- Pass functions with correct signatures to the evp_generic_fetch_xxx family of methods. UBSan complains about functions being called with incorrect signatures. Relates to #22896. — committed to fwh-dc/oqs-openssl by fwh-dc 2 years ago
- Pass functions with correct signatures to the evp_generic_fetch_xxx family of methods. UBSan complains about functions being called with incorrect signatures. Relates to #22896. — committed to fwh-dc/oqs-openssl by fwh-dc 2 years ago
- fixup! Pass functions with correct signatures to the evp_generic_fetch_xxx family of methods. UBSan complains about functions being called with incorrect signatures. Relates to #22896. — committed to fwh-dc/oqs-openssl by fwh-dc 2 years ago
- fixup! fixup! Pass functions with correct signatures to the evp_generic_fetch_xxx family of methods. UBSan complains about functions being called with incorrect signatures. Relates to #22896. — committed to fwh-dc/oqs-openssl by fwh-dc 2 years ago
- fixup! fixup! fixup! Pass functions with correct signatures to the evp_generic_fetch_xxx family of methods. UBSan complains about functions being called with incorrect signatures. Relates to #22896. — committed to fwh-dc/oqs-openssl by fwh-dc 2 years ago
For historical context, that “weird dereference” was to deal with a compiler bug that crashed for various versions of Borland C on windows unless you used the explicit pointer dereference approach.
Keep in mind that lhash predates SSLeay - that code had been to a lot of unusual places long before it went into SSLeay.
Is this option to make all the callbacks be
void*-typed? I do not think this is viable. These callbacks are exposed publicly, and callers have relied on the callbacks being typed for a very, very long time. It would be unreasonable to pull the rug out from under the whole OpenSSL ecosystem and say that they’revoid*-typed now.That is, the C requirement to match the type signature when you call the function means that these two APIs are incompatible:
If you told the caller that the API is
sk_FOO_pop_free1, they will write the function with aFOO*parameter. If you then change it tosk_FOO_pop_free2, you have broken their code.For function pointers which are not exposed publicly, you can change the type signature all you like. But changing any public ones is an API break.
@nhorman Regarding https://github.com/openssl/openssl/issues/22896#issuecomment-1837250395, you need to keep reading a few sentences further.
n1256, section 6.3.2.3, paragraph 8 says:
(“Compatible” is defined in 6.2.7.)
n1256, section 6.5.2.2, paragraph 9 says:
Both paragraphs also appear in the C89 draft you linked to. See also https://github.com/openssl/openssl/issues/22896#issuecomment-1834579201 which paraphrased that rule.
@thesamesam , ok, I think I’m starting to get it now. CFI, as a run time validation tool for, among other things, indirect pointer calls, might modify some data in the function pointer for validation purposes, and if the type conversion isn’t made properly, CFI breaks. Even if the conversion is effectively a no-op for the purposes of our run time operation in isloation, other tools that rely on those conversions happening may break, and thats whats ubsan is warning about here.
Ok, thank you for the education. It would seem to me then, that the most reasonable solution (though its ugly), would be to modify DEFINE_LHASH_OF_EX such that the the definition of lh_##type##_new accept function pointers whos arguments are all void *, and perform the explicit cast to the proper type in the respective _hash function implementation.
Note: This is also applicable to the compare function that gets passed to lh_##type##_new
the paragraphs you refer to are not relevant as the ‘prototype’ in question refers to the prototype in the function pointer type, which is still mismatched with the real function type, and hence, the wrong conversions happen at call time
it’s not. see:
Note that you need >= Clang 17 for this.
At least your last question should be easy to answer if @jwalch could be so kind to change this line and rerun the test.
This isn’t about alignment and is a true positive. A function with type
void (T*)cannot be called through a function pointer of typevoid (*)(void *). You can do the cast, but when you call it, it needs to be cast back to the original type.I don’t think the proposed fix is (necessarily) right either. This is actually a pretty thorny consequence of how OpenSSL’s
STACK_OF(T)andLHASH_OF(T)types do type erasure. The question is whether functions likelh_TYPE_new,sk_TYPE_pop_free, andsk_TYPE_set_cmp_funcare expected to takevoid*- orT*-based function pointers. You have to pick one or the other.For
STACK_OF(T), both the docs and long-standing practice force us to pick theT*version. TheT*version is also more type-safe.LHASH_OF(T)is interesting because the docs actually imply avoid*pointer. But looking at the code, I think this may just be a doc bug. The helper macro does define type-specific function pointers.Now, how do you get it cast to the right type if you pick the
T*one and do type erasure? It’s pretty thorny but you basically need to pass in type-aware thunks. Here’s what I did for BoringSSL. It’s pretty gross, but fixes the UB.As you go to do that, you’ll quickly discover that the standard library
qsortandbsearchfunctions are actually practically impossible to use to implement OpenSSL’s API. (They’re missing avoid*userdata or context parameter to pass in a closure.)bsearch_randqsort_rfix this, but are a portability nightmare. We ultimately just reimplemented both in the library to avoid UB.