oqs-provider: Bug when stopping provider on OSX

Describe the bug On a newly-prepared platform , I’m getting a ton of errors trying to run test-suite:

.  .  .
tests/test_speed.py:21: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

command = ['/Users/ur20980/src/liboqs/build/tests/speed_sig', 'SPHINCS+-SHAKE256-256s-robust', '-f']
working_dir = '.'
env = {'ANT_OPTS': '-Dhttp.proxyHost=llproxy.llan.ll.mit.edu -Dhttp.proxyPort=8080', 'AS': 'yasm -m amd64 -f macho64 -p gas -X gnu ', 'AS_INTEGRATED_ASSEMBLER': '1', 'BASH_SILENCE_DEPRECATION_WARNING': '1', ...}
expected_returncode = 0, input = None, ignore_returncode = False

    def run_subprocess(command, working_dir='.', env=None, expected_returncode=0, input=None, ignore_returncode=False):
        """
        Helper function to run a shell command and report success/failure
        depending on the exit status of the shell command.
        """
        env_ = os.environ.copy()
        if env is not None:
            env_.update(env)
        env = env_
    
        # Note we need to capture stdout/stderr from the subprocess,
        # then print it, which pytest will then capture and
        # buffer appropriately
        print(working_dir + " > " + " ".join(command))
    
        result = subprocess.run(
                command,
                input=input,
                stdout=subprocess.PIPE,
                stderr=subprocess.STDOUT,
                cwd=working_dir,
                env=env,
            )
    
        if not(ignore_returncode) and (result.returncode != expected_returncode):
            print(result.stdout.decode('utf-8'))
>           assert False, "Got unexpected return code {}".format(result.returncode)
E           AssertionError: Got unexpected return code -4

tests/helpers.py:42: AssertionError
-------------------------------------------- Captured stdout call --------------------------------------------
. > /Users/ur20980/src/liboqs/build/tests/speed_sig SPHINCS+-SHAKE256-256s-robust -f

__________________________________ test_sig[SPHINCS+-SHAKE256-256f-simple] ___________________________________
[gw11] darwin -- Python 3.11.3 /opt/local/bin/python3

sig_name = 'SPHINCS+-SHAKE256-256f-simple'

    @helpers.filtered_test
    @pytest.mark.parametrize('sig_name', helpers.available_sigs_by_name())
    def test_sig(sig_name):
        kats = helpers.get_kats("sig")
        if not(helpers.is_sig_enabled_by_name(sig_name)): pytest.skip('Not enabled')
>       helpers.run_subprocess( [helpers.path_to_executable('speed_sig'), sig_name, "-f"])

tests/test_speed.py:21: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

command = ['/Users/ur20980/src/liboqs/build/tests/speed_sig', 'SPHINCS+-SHAKE256-256f-simple', '-f']
working_dir = '.'
env = {'ANT_OPTS': '-Dhttp.proxyHost=llproxy.llan.ll.mit.edu -Dhttp.proxyPort=8080', 'AS': 'yasm -m amd64 -f macho64 -p gas -X gnu ', 'AS_INTEGRATED_ASSEMBLER': '1', 'BASH_SILENCE_DEPRECATION_WARNING': '1', ...}
expected_returncode = 0, input = None, ignore_returncode = False

    def run_subprocess(command, working_dir='.', env=None, expected_returncode=0, input=None, ignore_returncode=False):
        """
        Helper function to run a shell command and report success/failure
        depending on the exit status of the shell command.
        """
        env_ = os.environ.copy()
        if env is not None:
            env_.update(env)
        env = env_
    
        # Note we need to capture stdout/stderr from the subprocess,
        # then print it, which pytest will then capture and
        # buffer appropriately
        print(working_dir + " > " + " ".join(command))
    
        result = subprocess.run(
                command,
                input=input,
                stdout=subprocess.PIPE,
                stderr=subprocess.STDOUT,
                cwd=working_dir,
                env=env,
            )
    
        if not(ignore_returncode) and (result.returncode != expected_returncode):
            print(result.stdout.decode('utf-8'))
>           assert False, "Got unexpected return code {}".format(result.returncode)
E           AssertionError: Got unexpected return code -4

tests/helpers.py:42: AssertionError
-------------------------------------------- Captured stdout call --------------------------------------------
. > /Users/ur20980/src/liboqs/build/tests/speed_sig SPHINCS+-SHAKE256-256f-simple -f

__________________________________ test_sig[SPHINCS+-SHAKE256-192f-robust] ___________________________________
[gw4] darwin -- Python 3.11.3 /opt/local/bin/python3

sig_name = 'SPHINCS+-SHAKE256-192f-robust'

    @helpers.filtered_test
    @pytest.mark.parametrize('sig_name', helpers.available_sigs_by_name())
    def test_sig(sig_name):
        kats = helpers.get_kats("sig")
        if not(helpers.is_sig_enabled_by_name(sig_name)): pytest.skip('Not enabled')
>       helpers.run_subprocess( [helpers.path_to_executable('speed_sig'), sig_name, "-f"])

tests/test_speed.py:21: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

command = ['/Users/ur20980/src/liboqs/build/tests/speed_sig', 'SPHINCS+-SHAKE256-192f-robust', '-f']
working_dir = '.'
env = {'ANT_OPTS': '-Dhttp.proxyHost=llproxy.llan.ll.mit.edu -Dhttp.proxyPort=8080', 'AS': 'yasm -m amd64 -f macho64 -p gas -X gnu ', 'AS_INTEGRATED_ASSEMBLER': '1', 'BASH_SILENCE_DEPRECATION_WARNING': '1', ...}
expected_returncode = 0, input = None, ignore_returncode = False

    def run_subprocess(command, working_dir='.', env=None, expected_returncode=0, input=None, ignore_returncode=False):
        """
        Helper function to run a shell command and report success/failure
        depending on the exit status of the shell command.
        """
        env_ = os.environ.copy()
        if env is not None:
            env_.update(env)
        env = env_
    
        # Note we need to capture stdout/stderr from the subprocess,
        # then print it, which pytest will then capture and
        # buffer appropriately
        print(working_dir + " > " + " ".join(command))
    
        result = subprocess.run(
                command,
                input=input,
                stdout=subprocess.PIPE,
                stderr=subprocess.STDOUT,
                cwd=working_dir,
                env=env,
            )
    
        if not(ignore_returncode) and (result.returncode != expected_returncode):
            print(result.stdout.decode('utf-8'))
>           assert False, "Got unexpected return code {}".format(result.returncode)
E           AssertionError: Got unexpected return code -4

tests/helpers.py:42: AssertionError
-------------------------------------------- Captured stdout call --------------------------------------------
. > /Users/ur20980/src/liboqs/build/tests/speed_sig SPHINCS+-SHAKE256-192f-robust -f

__________________________________ test_sig[SPHINCS+-SHAKE256-256s-simple] ___________________________________
[gw2] darwin -- Python 3.11.3 /opt/local/bin/python3

sig_name = 'SPHINCS+-SHAKE256-256s-simple'

    @helpers.filtered_test
    @pytest.mark.parametrize('sig_name', helpers.available_sigs_by_name())
    def test_sig(sig_name):
        kats = helpers.get_kats("sig")
        if not(helpers.is_sig_enabled_by_name(sig_name)): pytest.skip('Not enabled')
>       helpers.run_subprocess( [helpers.path_to_executable('speed_sig'), sig_name, "-f"])

tests/test_speed.py:21: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

command = ['/Users/ur20980/src/liboqs/build/tests/speed_sig', 'SPHINCS+-SHAKE256-256s-simple', '-f']
working_dir = '.'
env = {'ANT_OPTS': '-Dhttp.proxyHost=llproxy.llan.ll.mit.edu -Dhttp.proxyPort=8080', 'AS': 'yasm -m amd64 -f macho64 -p gas -X gnu ', 'AS_INTEGRATED_ASSEMBLER': '1', 'BASH_SILENCE_DEPRECATION_WARNING': '1', ...}
expected_returncode = 0, input = None, ignore_returncode = False

    def run_subprocess(command, working_dir='.', env=None, expected_returncode=0, input=None, ignore_returncode=False):
        """
        Helper function to run a shell command and report success/failure
        depending on the exit status of the shell command.
        """
        env_ = os.environ.copy()
        if env is not None:
            env_.update(env)
        env = env_
    
        # Note we need to capture stdout/stderr from the subprocess,
        # then print it, which pytest will then capture and
        # buffer appropriately
        print(working_dir + " > " + " ".join(command))
    
        result = subprocess.run(
                command,
                input=input,
                stdout=subprocess.PIPE,
                stderr=subprocess.STDOUT,
                cwd=working_dir,
                env=env,
            )
    
        if not(ignore_returncode) and (result.returncode != expected_returncode):
            print(result.stdout.decode('utf-8'))
>           assert False, "Got unexpected return code {}".format(result.returncode)
E           AssertionError: Got unexpected return code -4

tests/helpers.py:42: AssertionError
-------------------------------------------- Captured stdout call --------------------------------------------
. > /Users/ur20980/src/liboqs/build/tests/speed_sig SPHINCS+-SHAKE256-256s-simple -f

___________________________________ test_sig[SPHINCS+-SHA256-128f-robust] ____________________________________
[gw9] darwin -- Python 3.11.3 /opt/local/bin/python3

sig_name = 'SPHINCS+-SHA256-128f-robust'

    @helpers.filtered_test
    @pytest.mark.parametrize('sig_name', helpers.available_sigs_by_name())
    def test_sig(sig_name):
        kats = helpers.get_kats("sig")
        if not(helpers.is_sig_enabled_by_name(sig_name)): pytest.skip('Not enabled')
>       output = helpers.run_subprocess(
            [helpers.path_to_executable('kat_sig'), sig_name],
        )

tests/test_kat.py:29: 
.  .  .

To Reproduce Steps to reproduce the behavior:

Clone, build optimized version, as per README.
Go to build subdirectory.
ninja run_tests
See error

Expected behavior Tests passing or being skipped, test-scripts not crashing.

Screenshots If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

OS: MacOS Ventura 13.3.1 (a)
OpenSSL version: 3.2.0-dev
Compiler version used: Apple Clang
Build variables used:

OPT="-DOQS_DIST_BUILD=ON -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Release"
env RUN_TESTS_PARALLEL_UPPER=FALSE cmake -GNinja .. -DOQS_USE_OPENSSL=ON ${OPT} -DCMAKE_INSTALL_PREFIX="${HOME}/openssl-3"

liboqs version: current main

Additional context I hope you can point me at where/what in Python environment seems missing - maybe an older version of some module, or such…?

About this issue

Original URL
State: closed
Created a year ago
Comments: 18 (17 by maintainers)

Most upvoted comments

As far as I can see, current master runs and passes tests on my MacOS Ventura, both Intel and M2 platforms. Using OpenSSL-3.1.1 and 3.2.0-dev.

Thanks for these tests on what is now a release: Glad to hear that!

baentsch on Jun 10, 2023

As far as I can see, current master runs and passes tests on my MacOS Ventura, both Intel and M2 platforms. Using OpenSSL-3.1.1 and 3.2.0-dev.

Thus, let me close this for now - might re-open if hit with the same problem again. 😉 Thanks!

mouse07410 on Jun 9, 2023

In any case this doesn’t currently look like an oqs-provider issue. More likely it is an OpenSSL problem. Perhaps open an issue there?

Opened an OpenSSL issue for this. Also, it affects both Apple Silicon and x86_64 platforms.

mouse07410 on May 22, 2023

Correct workaround is -DOQS_USE_OPENSSL=OFF

Thanks for the test/reconfirmation. We’ll look into this.

baentsch on May 11, 2023