openssl: 3.0 FIPS provider is not compatible with tests in 3.0.5

I am not sure why we ended up testing the stable 3.0.X branch against master and NOT 3.0.0 source tarball against latest stable 3.0.X branch…

The consequence of this is that the end user can not run tests without failures if they run 3.0.0 source tarball against latest stable 3.0.X branch… I have verified that there is at least one breaking change (see comment below related to key mismatch) when using the OpenSSL 3.0 fips provider with OpenSSL 3.0.5.

The first failure I looked at happens because the tests were updated to match changes inside the provider… e.g:

Inside provider/exchange/kdf_exch.c the new code is

static int kdf_derive(void *vpkdfctx, unsigned char *secret, size_t *secretlen, size_t outlen)
{
    PROV_KDF_CTX *pkdfctx = (PROV_KDF_CTX *)vpkdfctx;
    size_t kdfsize;
    int ret;

    if (!ossl_prov_is_running())
        return 0;

    kdfsize = EVP_KDF_CTX_get_kdf_size(pkdfctx->kdfctx);

    if (secret == NULL) {
        *secretlen = kdfsize;
        return 1;
    }

    if (kdfsize != SIZE_MAX) {
        if (outlen < kdfsize) {
            ERR_raise(ERR_LIB_PROV, PROV_R_OUTPUT_BUFFER_TOO_SMALL);
            return 0;
        }
        outlen = kdfsize;
    }

    ret = EVP_KDF_derive(pkdfctx->kdfctx, secret, outlen, NULL);
    if (ret <= 0)
        return 0;

    *secretlen = outlen;
    return 1;
}

Inside the 3.0 fips provider the code is much simpler and does not do the

if (kdfsize != SIZE_MAX) check..

As evp_test was then updated to deliberately double the len this will fail with the 3.0 fips provider…

How do we fix this? What is the process that users should follow to deploy and test the OpenSSL 3.0 FIPS provider?

---

Test failures

04-test_encoder_decoder.t (Wstat: 256 Tests: 3 Failed: 1) Failed test: 3 Non-zero exit status: 1 25-test_verify.t (Wstat: 512 Tests: 163 Failed: 2) Failed tests: 115-116 Non-zero exit status: 2 30-test_evp.t (Wstat: 3072 Tests: 103 Failed: 12) Failed tests: 38, 48, 51-52, 54, 57-63 Non-zero exit status: 12 65-test_cmp_client.t (Wstat: 256 Tests: 3 Failed: 1) Failed test: 3 Non-zero exit status: 1 65-test_cmp_protect.t (Wstat: 256 Tests: 3 Failed: 1) Failed test: 3 Non-zero exit status: 1 80-test_cms.t (Wstat: 1792 Tests: 12 Failed: 7) Failed tests: 2-6, 9, 11 Non-zero exit status: 7 80-test_ssl_new.t (Wstat: 7168 Tests: 30 Failed: 28) Failed tests: 1-21, 23-28, 30 Non-zero exit status: 28 80-test_ssl_old.t (Wstat: 512 Tests: 11 Failed: 2) Failed tests: 7-8 Non-zero exit status: 2 90-test_sslapi.t (Wstat: 256 Tests: 2 Failed: 1) Failed test: 2 Non-zero exit status: 1 Files=243, Tests=3437, 474 wallclock secs ( 6.86 usr 0.60 sys + 375.70 cusr 41.42 csys = 424.58 CPU)

NOTE there are quite a few… keypair mismatch errors such as… X509_check_private_key:key values mismatch:crypto/x509/x509_cmp.c:405 happens quite