pytest: Tests run when called by name but seg fault when part of a collection

Hi,

I’m running the python3-pytest package of Ubuntu 18.04, v3.3.2-2, with cov-2.5.1 from a package and faulthandler-1.5.0 from pip. (I’m not using a virtual environment.) The application I am testing makes extensive use of gevent. This has worked well until now but today, when refactoring my tests to move the startup/teardown code to a module, I’m getting seg faults.

I’ve tried to isolate the issue, and it seems to be something around pytest teardown after the last test in a collection, ie

• running one test by name works
• running all the tests in a file seg faults (after successful completion of all tests)
• running one test by mark also seg faults (after successful completion of that test).

ie

$ pytest-3 test_app_cancel_upload.py -m crash_when_marked -v
===================================================================== test session starts ======================================================================
platform linux -- Python 3.6.5, pytest-3.3.2, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python3
cachedir: ../../.cache
rootdir: /home/mark/dbl-uploader-clients/dbl_dot_local, inifile: pytest.ini
plugins: faulthandler-1.5.0, cov-2.5.1
collected 5 items                                                                                                                                              

test_app_cancel_upload.py::test_cancel_upload_jobs PASSED                                                                                                [100%]

====================================================================== 4 tests deselected ======================================================================
=========================================================== 1 passed, 4 deselected in 10.47 seconds ============================================================
Fatal Python error: Segmentation fault

Current thread 0x00007fe73c7eb740 (most recent call first):
Erreur de segmentation (core dumped)
$ pytest-3 test_app_cancel_upload.py::test_cancel_upload_jobs -v
===================================================================== test session starts ======================================================================
platform linux -- Python 3.6.5, pytest-3.3.2, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python3
cachedir: ../../.cache
rootdir: /home/mark/dbl-uploader-clients/dbl_dot_local, inifile: pytest.ini
plugins: faulthandler-1.5.0, cov-2.5.1
collected 1 item                                                                                                                                               

test_app_cancel_upload.py::test_cancel_upload_jobs PASSED                                                                                                [100%]

=================================================================== 1 passed in 8.83 seconds ===================================================================
$ 

The test above looks like this:

@pytest.mark.crash_when_marked
@pytest.mark.cancelUploadJobs
def test_cancel_upload_jobs(temp_config_path):
    """
    Test cancelUploadJobs for host bundle
    """
    config_ob = Config(prepend_real(temp_config_path))
    du_app = DownUpApp(config_ob, verbose=True)
    # Find an entry to download
    entry_id, entry_revision = first_text_entry_ids(du_app.dbl_endpoints)
    # task to download entry
    tasks_xml = """<tasks>
    <downloadMetadata>
        <entry>{0}</entry>
        <revision>{1}</revision>
        <label>downloadedBundle</label>
        <tasks>
            <createUploadJob/>
        </tasks>
    </downloadMetadata>
</tasks>""".format(entry_id, entry_revision)
    tasks_dom = etree.fromstring(tasks_xml)
    first_bundle_id = du_app.storer.new_bundle("text", tasks_dom)
    du_app.tick_while_active(60)
    jobs = du_app.dbl_endpoints.entry_upload_jobs(entry_id)[3]
    assert len(jobs) > 0
    second_bundle_id = du_app.storer.label(first_bundle_id, "downloadedBundle")
    tasks_xml = """<tasks>
    <cancelUploadJobs/>
</tasks>"""
    du_app.storer.add_tasks_xml(second_bundle_id, etree.fromstring(tasks_xml))
    du_app.tick_while_active(60)
    assert not du_app.is_active()
    jobs = du_app.dbl_endpoints.entry_upload_jobs(entry_id)[3]
    assert len(jobs) == 0

and two of the fixtures look like this:

@pytest.fixture()
def temp_bundle_dir():
    """
    Set up a temporary bundle dir with matching settings file, delete it after use.
    """
    temp_dir = tempfile.mkdtemp()
    if platform.system() == "Windows":
        temp_dir = re.sub(
                "\\\\",
                "/",
                temp_dir
            )
    with open(prepend_test_data_dir("valid_settings.xml.template"), "r") as t_in:
        template = t_in.read()
    template = re.sub("%%TMP_DIR%%", temp_dir, template)
    temp_bundle_dir_path = os.path.join(temp_dir, "bundles")
    os.mkdir(temp_bundle_dir_path)
    temp_session_bundle_dir_path = os.path.join(temp_dir, "sessions")
    os.mkdir(temp_session_bundle_dir_path)
    print('built {0}'.format(temp_dir))
    yield temp_dir
    print('cleanup {0}'.format(temp_dir))
    shutil.rmtree(temp_dir)

@pytest.fixture
def temp_config_path(temp_bundle_dir):
    temp_dir = temp_bundle_dir
    with open(prepend_test_data_dir("valid_settings.xml.template"), "r") as t_in:
        template = t_in.read()
    template = re.sub("%%TMP_DIR%%", temp_dir, template)
    temp_config_path = os.path.join(temp_dir, "valid_settings.xml")
    with open(temp_config_path, "w") as t_out:
        t_out.write(template)
    print('config path: {0}'.format(temp_config_path))
    return temp_config_path

This looks to me like a bug in pytest, since the test behaviour varies according to how it is invoked. But I’m quite willing to believe that there’s also something evil in my code - I just have no clue how to find it.

All suggestions welcome.