openssl: openssl-3.2.0-alpha1 fails tests when built out-of-source

In Gentoo, we build out-of-source for multilib as it makes life a lot easier (and it’s a bit quicker).

In 3.2.0-alpha1, tests seem to fail in this configuration:

cd /tmp
tar xvf openssl-3.2.0-alpha1.tar.xz
mkdir build && cd build
/tmp/openssl-3.2.0-alpha1/config && make -j$(nproc) && make test -j$(nproc) HARNESS_JOBS=“-j$(nproc)” VFP=1 -Onone

Test Summary Report                                                                                                                                                                                   [313/9610]
-------------------
01-test_sanity.t                      (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
01-test_symbol_presence.t             (Wstat: 512 (exited 2) Tests: 3 Failed: 2)
  Failed tests:  2-3
  Non-zero exit status: 2
01-test_test.t                        (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
02-test_errstr.t                      (Wstat: 34816 (exited 136) Tests: 137 Failed: 136)
  Failed tests:  1-136
  Non-zero exit status: 136
02-test_lhash.t                       (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
02-test_list.t                        (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
02-test_localetest.t                  (Wstat: 768 (exited 3) Tests: 3 Failed: 3)
  Failed tests:  1-3
  Non-zero exit status: 3
02-test_priority_queue.t              (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
02-test_stack.t                       (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
03-test_exdata.t                      (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
03-test_internal_ssl_cert_table.t     (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
03-test_params_api.t                  (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
03-test_property.t                    (Wstat: 256 (exited 1) Tests: 2 Failed: 1)
  Failed test:  2
  Non-zero exit status: 1
03-test_ui.t                          (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
04-test_asn1_decode.t                 (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
04-test_asn1_encode.t                 (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
04-test_asn1_string_table.t           (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
04-test_bio_callback.t                (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
04-test_bio_core.t                    (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
04-test_bio_dgram.t                   (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
04-test_bioprint.t                    (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
04-test_conf.t                        (Wstat: 512 (exited 2) Tests: 4 Failed: 2)
  Failed tests:  1, 3
  Non-zero exit status: 2
04-test_err.t                         (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
04-test_membio.t                      (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
04-test_params_conversion.t           (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
04-test_pem_read_depr.t               (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
04-test_pem_reading.t                 (Wstat: 6400 (exited 25) Tests: 55 Failed: 25)
  Failed tests:  1-3, 5, 7, 10-22, 25-26, 30, 47, 52, 54-55
  Non-zero exit status: 25
04-test_provider_default_search_path.t (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
Files=291, Tests=3580, 299 wallclock secs ( 3.95 usr  1.00 sys + 191.26 cusr 79.49 csys = 275.70 CPU)
Result: FAIL

$ perl configdata.pm --dump

Command line (with current working directory = .):

    /usr/bin/perl ../openssl-3.2.0-alpha1/Configure

Perl information:

    /usr/bin/perl
    5.38.0 for x86_64-linux

Enabled features:

    afalgeng
    apps
    argon2
    aria
    asm
    async
    autoalginit
    autoerrinit
    autoload-config
    bf
    blake2
    bulk
    cached-fetch
    camellia
    capieng
    cast
    chacha
    cmac
    cmp
    cms
    comp
    ct
    default-thread-pool
    deprecated
    des
    dgram
    dh
    docs
    dsa
    dso
    dtls
    dynamic-engine
    ec
    ec2m
    ecdh
    ecdsa
    ecx
    engine
    err
    filenames
    gost
    http
    idea
    legacy
    loadereng
    makedepend
    md4
    mdc2
    module
    multiblock
    nextprotoneg
    ocb
    ocsp
    padlockeng
    pic
    pinshared
    poly1305
    posix-io
    psk
    quic
    rc2
    rc4
    rdrand
    rfc3779
    rmd160
    scrypt
    secure-memory
    seed
    shared
    siphash
    siv
    sm2
    sm2-precomp
    sm3
    sm4
    sock
    srp
    srtp
    sse2
    ssl
    ssl-trace
    static-engine
    stdio
    tests
    thread-pool
    threads
    tls
    ts
    ui-console
    whirlpool
    tls1
    tls1-method
    tls1_1
    tls1_1-method
    tls1_2
    tls1_2-method
    tls1_3
    dtls1
    dtls1-method
    dtls1_2
    dtls1_2-method

Disabled features:

    acvp-tests          [cascade]        OPENSSL_NO_ACVP_TESTS
    asan                [default]        OPENSSL_NO_ASAN
    brotli              [default]        OPENSSL_NO_BROTLI
    brotli-dynamic      [default]        OPENSSL_NO_BROTLI_DYNAMIC
    buildtest-c++       [default]        
    winstore            [not-windows]    OPENSSL_NO_WINSTORE
    crypto-mdebug       [default]        OPENSSL_NO_CRYPTO_MDEBUG
    devcryptoeng        [default]        OPENSSL_NO_DEVCRYPTOENG
    ec_nistp_64_gcc_128 [default]        OPENSSL_NO_EC_NISTP_64_GCC_128
    egd                 [default]        OPENSSL_NO_EGD
    external-tests      [default]        OPENSSL_NO_EXTERNAL_TESTS
    fips                [default]        
    fips-securitychecks [cascade]        OPENSSL_NO_FIPS_SECURITYCHECKS
    fuzz-afl            [default]        OPENSSL_NO_FUZZ_AFL
    fuzz-libfuzzer      [default]        OPENSSL_NO_FUZZ_LIBFUZZER
    ktls                [default]        OPENSSL_NO_KTLS
    md2                 [default]        OPENSSL_NO_MD2 (skip crypto/md2)
    msan                [default]        OPENSSL_NO_MSAN
    rc5                 [default]        OPENSSL_NO_RC5 (skip crypto/rc5)
    sctp                [default]        OPENSSL_NO_SCTP
    tfo                 [default]        OPENSSL_NO_TFO
    trace               [default]        OPENSSL_NO_TRACE
    ubsan               [default]        OPENSSL_NO_UBSAN
    unit-test           [default]        OPENSSL_NO_UNIT_TEST
    uplink              [no uplink_arch] OPENSSL_NO_UPLINK
    weak-ssl-ciphers    [default]        OPENSSL_NO_WEAK_SSL_CIPHERS
    zlib                [default]        OPENSSL_NO_ZLIB
    zlib-dynamic        [default]        OPENSSL_NO_ZLIB_DYNAMIC
    zstd                [default]        OPENSSL_NO_ZSTD
    zstd-dynamic        [default]        OPENSSL_NO_ZSTD_DYNAMIC
    ssl3                [default]        OPENSSL_NO_SSL3
    ssl3-method         [default]        OPENSSL_NO_SSL3_METHOD

Config target attributes:

    AR => "ar",
    ARFLAGS => "qc",
    CC => "gcc",
    CFLAGS => "-Wall -O3",
    CXX => "g++",
    CXXFLAGS => "-Wall -O3",
    HASHBANGPERL => "/usr/bin/env perl",
    RANLIB => "ranlib",
    RC => "windres",
    asm_arch => "x86_64",
    bn_ops => "SIXTY_FOUR_BIT_LONG",
    build_file => "Makefile",
    build_scheme => [ "unified", "unix" ],
    cflags => "-pthread -m64",
    cppflags => "",
    cxxflags => "-std=c++11 -pthread -m64",
    defines => [ "OPENSSL_BUILDING_OPENSSL" ],
    disable => [  ],
    dso_ldflags => "-Wl,-z,defs",
    dso_scheme => "dlfcn",
    enable => [ "afalgeng" ],
    ex_libs => "-ldl -pthread",
    includes => [  ],
    lflags => "",
    lib_cflags => "",
    lib_cppflags => "-DOPENSSL_USE_NODELETE -DL_ENDIAN",
    lib_defines => [  ],
    module_cflags => "-fPIC",
    module_cxxflags => undef,
    module_ldflags => "-Wl,-znodelete -shared -Wl,-Bsymbolic",
    multilib => "64",
    perl_platform => "Unix",
    perlasm_scheme => "elf",
    shared_cflag => "-fPIC",
    shared_defflag => "-Wl,--version-script=",
    shared_defines => [  ],
    shared_ldflag => "-Wl,-znodelete -shared -Wl,-Bsymbolic",
    shared_rcflag => "",
    shared_sonameflag => "-Wl,-soname=",
    shared_target => "linux-shared",
    thread_defines => [  ],
    thread_scheme => "pthreads",
    unistd => "<unistd.h>",

Recorded environment:

    AR = 
    ARFLAGS = 
    AS = 
    ASFLAGS = 
    BUILDFILE = 
    CC = 
    CFLAGS = 
    CPP = 
    CPPDEFINES = 
    CPPFLAGS = 
    CPPINCLUDES = 
    CROSS_COMPILE = 
    CXX = 
    CXXFLAGS = 
    HASHBANGPERL = 
    LD = 
    LDFLAGS = 
    LDLIBS = 
    MT = 
    MTFLAGS = 
    OPENSSL_LOCAL_CONFIG_DIR = 
    PERL = 
    RANLIB = 
    RC = 
    RCFLAGS = 
    RM = 
    WINDRES = 
    __CNF_CFLAGS = 
    __CNF_CPPDEFINES = 
    __CNF_CPPFLAGS = 
    __CNF_CPPINCLUDES = 
    __CNF_CXXFLAGS = 
    __CNF_LDFLAGS = 
    __CNF_LDLIBS = 

Makevars:

    AR              = ar
    ARFLAGS         = qc
    ASFLAGS         = 
    CC              = gcc
    CFLAGS          = -Wall -O3
    CPPDEFINES      = 
    CPPFLAGS        = 
    CPPINCLUDES     = 
    CXX             = g++
    CXXFLAGS        = -Wall -O3
    HASHBANGPERL    = /usr/bin/env perl
    LDFLAGS         = 
    LDLIBS          = 
    PERL            = /usr/bin/perl
    RANLIB          = ranlib
    RC              = windres
    RCFLAGS         = 

NOTE: These variables only represent the configuration view.  The build file
template may have processed these variables further, please have a look at the
build file for more exact data:
    Makefile

build file:

    Makefile

build file templates:

    ../openssl-3.2.0-alpha1/Configurations/common0.tmpl
    ../openssl-3.2.0-alpha1/Configurations/unix-Makefile.tmpl

About this issue

Original URL
State: closed
Created 10 months ago
Comments: 39 (33 by maintainers)

Commits related to this issue

dev-libs/openssl: pass -j1 for install See https://github.com/openssl/openssl/issues/21999#issuecomment-1771150305 - upstream say parallelism isn't supported for the install targets. Bug: https://gi... — committed to gentoo/gentoo by thesamesam 8 months ago
Remove the source directory .num targets $(SRCDIR)/util/libcrypto.num and $(SRCDIR)/util/libssl.num were made their own targets to have 'make ordinals' reproduce them (run mknum.pl) only if needed. ... — committed to openssl/openssl by levitte 7 months ago
Remove the source directory .num targets $(SRCDIR)/util/libcrypto.num and $(SRCDIR)/util/libssl.num were made their own targets to have 'make ordinals' reproduce them (run mknum.pl) only if needed. ... — committed to wbeck10/openssl by levitte 7 months ago
Remove the source directory .num targets $(SRCDIR)/util/libcrypto.num and $(SRCDIR)/util/libssl.num were made their own targets to have 'make ordinals' reproduce them (run mknum.pl) only if needed. ... — committed to rsbeckerca/openssl by levitte 7 months ago

Most upvoted comments

+1. FWIW when developing I always find running ‘make test’ after changing a file to result in trying to run some test binaries before they are actually finished compiling, resulting in an error, which is why I always run make && make test. We should fix that.

hlandau on Nov 21, 2023

That’s an odd one, @hlandau.

This issue should really be closed, perhaps after raising a few topical ones from the details we’ve found here.

levitte on Nov 28, 2023

So far, it seems fine with -j1 just on the make test ... invocation. I must have either made a mistake when looking at the output before (thinking it was being slow and rebuilding tests unnecessarily) or I did -j1 for the whole lot when testing and confused the two. So, good enough for us. Thanks.

thesamesam on Nov 21, 2023

I’m not sure why it’s not a blocker, as it’s a regression for us and it’s meant we hadn’t been able to test any of it.

Running tests with -j1 seems to be a fair bit slower because actually compiling them takes a while but I guess it’s a workaround.

You can build the tests first with make -jn and then run make test -j1. Though I suspect make test -jn will also work here. Personally I tend to use make && make test.

hlandau on Nov 21, 2023

I’m going to play with this over the weekend and dig into it. I can still reproduce it in one environment consistently, others are more picky.

Please don’t worry about this bug until I’ve had a chance to give an update and hopefully some useful info over the next few days. Cheers!

thesamesam on Sep 7, 2023

../../test/exptest: error while loading shared libraries: ../../util/../util/../libcrypto.so.3: file too short

A possible issue here is running the tests while libcrypto.so.3 is still being written. That’s a bit odd since that should be done as part of make. Given the invocation you list above it’s hard to see how it could happen.

Is it possible libcrypto.so.3 is corrupted in some way? Would you be able to examine things in the build environment after the failure and see if it looks right? If you can get a shell in the build environment (and this isn’t in some fully automated build process) you could also see if running make test again works.

hlandau on Sep 7, 2023

Not able to reproduce this currently. That doesn’t mean this isn’t a real issue, but maybe I don’t have enough parallelism on this system to reproduce it.

The most likely cause is some kind of race condition around make test when used concurrently. Avoiding the use of HARNESS_JOBS is a likely workaround for now. There are certainly some slight bugs in concurrent use of make test.

Consider adding V=1 to make test to get verbose logging. This might give some hints.

hlandau on Sep 7, 2023