openssl: Test 04-test_encoder_decoder.t fails on NonStop x86

This particular test works fine on 3.0.0 as released and with updates along the 3.0.0 branch. Where it fails is in master:

04-test_encoder_decoder.t .......... 1/?
../../util/wrap.pl ../../test/endecode_test -rsa ../../test/certs/ee-key.pem -pss ../../test/certs/ca-pss-key.pem -context -config ../../test/default.cnf -provider default => 139

This is a reopen of what was #16950 focusing on the real issue rather than the red-herring subject of that issue. I’m trying to figure out what changed between the two branches in a significant way and would like help tracking this down. Running directly and debugging using endecode_test appears to report a 0 completion code when the wrap.pl reports 139.

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 128 (128 by maintainers)

Commits related to this issue

Statically link the legacy provider to endecode_test We already statically link libcrypto to endecode_test even in a "shared" build. This can cause problems on some platforms with tests that load the... — committed to mattcaswell/openssl by mattcaswell 3 years ago
Statically link the legacy provider to endecode_test We already statically link libcrypto to endecode_test even in a "shared" build. This can cause problems on some platforms with tests that load the... — committed to t8m/openssl by mattcaswell 3 years ago
Statically link the legacy provider to endecode_test We already statically link libcrypto to endecode_test even in a "shared" build. This can cause problems on some platforms with tests that load the... — committed to openssl/openssl by mattcaswell 3 years ago

Most upvoted comments

If I build normally and the delete the providers/legacy.so file then rerunning make shows me exactly what gets linked in as follows:

./config -d
make
rm providers/legacy.so
make

the final make shows me what goes into legacy.so. For me that is:

make depend && make _build_sw
make[1]: Entering directory '/home/matt/dev/openssl-write2'
make[1]: Leaving directory '/home/matt/dev/openssl-write2'
make[1]: Entering directory '/home/matt/dev/openssl-write2'
gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -O0 -g -DPEDANTIC -pedantic -Wno-long-long -DUNUSEDRESULT_DEBUG -Wall -Wmissing-declarations -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wswitch -Wsign-compare -Wshadow -Wformat -Wtype-limits -Wundef -Werror -Wmissing-prototypes -Wstrict-prototypes -L. -Wl,-z,defs -Wl,-znodelete -shared -Wl,-Bsymbolic   \
	-o providers/legacy.so -Wl,--version-script=providers/legacy.ld \
	providers/legacy-dso-legacyprov.o \
	providers/liblegacy.a providers/libcommon.a -lcrypto -ldl -pthread 
make[1]: Leaving directory '/home/matt/dev/openssl-write2'

So we can see it is made up of providers/legacy-dso-legacyprov.o, providers/liblegacy.a and providers/libcommon.a.

The .a files are made up of lots of .o files. by rebuilding them we can see what they contain. If I delete the liblegacy.a file via rm providers/liblegacy.a and then rebuild it with make providers/liblegacy.a I get:

rm -f providers/liblegacy.a
ar qc providers/liblegacy.a crypto/bn/asm/liblegacy-lib-x86_64-gcc.o crypto/bn/liblegacy-lib-rsaz-2k-avx512.o crypto/bn/liblegacy-lib-rsaz-3k-avx512.o crypto/bn/liblegacy-lib-rsaz-4k-avx512.o crypto/bn/liblegacy-lib-rsaz-avx2.o crypto/bn/liblegacy-lib-rsaz-x86_64.o crypto/bn/liblegacy-lib-rsaz_exp.o crypto/bn/liblegacy-lib-rsaz_exp_x2.o crypto/bn/liblegacy-lib-x86_64-gf2m.o crypto/bn/liblegacy-lib-x86_64-mont.o crypto/bn/liblegacy-lib-x86_64-mont5.o crypto/des/liblegacy-lib-des_enc.o crypto/des/liblegacy-lib-fcrypt_b.o crypto/liblegacy-lib-cpuid.o crypto/liblegacy-lib-ctype.o crypto/liblegacy-lib-x86_64cpuid.o crypto/md5/liblegacy-lib-md5-x86_64.o crypto/md5/liblegacy-lib-md5_dgst.o crypto/md5/liblegacy-lib-md5_one.o crypto/md5/liblegacy-lib-md5_sha1.o providers/common/liblegacy-lib-provider_util.o providers/implementations/ciphers/liblegacy-lib-cipher_blowfish.o providers/implementations/ciphers/liblegacy-lib-cipher_blowfish_hw.o providers/implementations/ciphers/liblegacy-lib-cipher_cast5.o providers/implementations/ciphers/liblegacy-lib-cipher_cast5_hw.o providers/implementations/ciphers/liblegacy-lib-cipher_des.o providers/implementations/ciphers/liblegacy-lib-cipher_des_hw.o providers/implementations/ciphers/liblegacy-lib-cipher_desx.o providers/implementations/ciphers/liblegacy-lib-cipher_desx_hw.o providers/implementations/ciphers/liblegacy-lib-cipher_idea.o providers/implementations/ciphers/liblegacy-lib-cipher_idea_hw.o providers/implementations/ciphers/liblegacy-lib-cipher_rc2.o providers/implementations/ciphers/liblegacy-lib-cipher_rc2_hw.o providers/implementations/ciphers/liblegacy-lib-cipher_rc4.o providers/implementations/ciphers/liblegacy-lib-cipher_rc4_hmac_md5.o providers/implementations/ciphers/liblegacy-lib-cipher_rc4_hmac_md5_hw.o providers/implementations/ciphers/liblegacy-lib-cipher_rc4_hw.o providers/implementations/ciphers/liblegacy-lib-cipher_seed.o providers/implementations/ciphers/liblegacy-lib-cipher_seed_hw.o providers/implementations/ciphers/liblegacy-lib-cipher_tdes_common.o providers/implementations/digests/liblegacy-lib-md4_prov.o providers/implementations/digests/liblegacy-lib-mdc2_prov.o providers/implementations/digests/liblegacy-lib-ripemd_prov.o providers/implementations/digests/liblegacy-lib-wp_prov.o providers/implementations/kdfs/liblegacy-lib-pbkdf1.o providers/implementations/kdfs/liblegacy-lib-pvkkdf.o providers/liblegacy-lib-prov_running.o
ranlib providers/liblegacy.a || echo Never mind.

Similarly for libcommon.a, delete it via rm providers/libcommon.a and rebuild it with make providers/libcommon.a:

rm -f providers/libcommon.a
ar qc providers/libcommon.a providers/common/der/libcommon-lib-der_digests_gen.o providers/common/der/libcommon-lib-der_dsa_gen.o providers/common/der/libcommon-lib-der_dsa_key.o providers/common/der/libcommon-lib-der_dsa_sig.o providers/common/der/libcommon-lib-der_ec_gen.o providers/common/der/libcommon-lib-der_ec_key.o providers/common/der/libcommon-lib-der_ec_sig.o providers/common/der/libcommon-lib-der_ecx_gen.o providers/common/der/libcommon-lib-der_ecx_key.o providers/common/der/libcommon-lib-der_rsa_gen.o providers/common/der/libcommon-lib-der_rsa_key.o providers/common/der/libcommon-lib-der_wrap_gen.o providers/common/libcommon-lib-provider_ctx.o providers/common/libcommon-lib-provider_err.o providers/implementations/ciphers/libcommon-lib-ciphercommon.o providers/implementations/ciphers/libcommon-lib-ciphercommon_block.o providers/implementations/ciphers/libcommon-lib-ciphercommon_ccm.o providers/implementations/ciphers/libcommon-lib-ciphercommon_ccm_hw.o providers/implementations/ciphers/libcommon-lib-ciphercommon_gcm.o providers/implementations/ciphers/libcommon-lib-ciphercommon_gcm_hw.o providers/implementations/ciphers/libcommon-lib-ciphercommon_hw.o providers/implementations/digests/libcommon-lib-digestcommon.o ssl/record/libcommon-lib-tls_pad.o
ranlib providers/libcommon.a || echo Never mind.

I don’t see init.o anywhere in that list for either liblegacy.a or libcommon.a. What about for you?

mattcaswell on Dec 17, 2021