spyder: `test_no_empty_file_items` not working on some CI setups

Issue Report Checklist

  • Searched the issues page for similar reports
  • Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
  • Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
  • Could not reproduce inside jupyter qtconsole (if console-related)
  • Tried basic troubleshooting (if a bug/error)
    • Restarted Spyder
    • Reset preferences with spyder --reset
    • Reinstalled the latest version of Anaconda
    • Tried the other applicable steps from the Troubleshooting Guide
  • Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

This is a bit weird. The test spyder/plugins/findinfiles/widgets/tests/test_widgets.py::test_no_empty_file_items has started failing on the Debian CI system (which uses lxc containers). Some of these failures can be seen in the test logs at https://ci.debian.net/packages/s/spyder/unstable/amd64/ Here’s a relevant part of one of them:

=================================== FAILURES ===================================
___________________________ test_no_empty_file_items ___________________________

findinfiles = <spyder.plugins.findinfiles.widgets.main_widget.FindInFilesWidget object at 0x7f15887e4280>
qtbot = <pytestqt.qtbot.QtBot object at 0x7f1588f8beb0>

    @flaky(max_runs=5)
    @pytest.mark.skipif(not running_in_ci(), reason="Only works on CIs")
    @pytest.mark.skipif(os.name == 'nt', reason="Fails on Windows")
    def test_no_empty_file_items(findinfiles, qtbot):
        """
        Test that a search that hits the max number of results doesn't generate
        empty file items.
    
        This is a regression test for issue spyder-ide/spyder#16256
        """
        findinfiles.set_search_text("spam")
        findinfiles.set_directory(osp.join(LOCATION, "data"))
        findinfiles.set_conf('max_results', 6)
    
        with qtbot.waitSignal(findinfiles.sig_max_results_reached):
            findinfiles.find()
    
        # Assert the results are the expected ones to reproduce the bug this test
        # tries to catch. In other words, this is here to prevent future changes to
        # our test files that would render this test useless.
        results = {
            'spam.py': [(2, 7), (5, 1), (7, 12)],
            'spam.txt': [(1, 0), (1, 5), (3, 22)]
        }
>       assert process_search_results(findinfiles.result_browser.data) == results
E       AssertionError: assert {'spam.cpp': [(2, 9), (6, 15), (8, 2)], 'spam.py': [(2, 7), (5, 1), (7, 12)]} == {'spam.py': [(2, 7), (5, 1), (7, 12)], 'spam.txt': [(1, 0), (1, 5), (3, 22)]}
E         Common items:
E         {'spam.py': [(2, 7), (5, 1), (7, 12)]}
E         Left contains 1 more item:
E         {'spam.cpp': [(2, 9), (6, 15), (8, 2)]}
E         Right contains 1 more item:
E         {'spam.txt': [(1, 0), (1, 5), (3, 22)]}
E         Full diff:
E         - {'spam.py': [(2, 7), (5, 1), (7, 12)], 'spam.txt': [(1, 0), (1, 5), (3, 22)]}
E         + {'spam.cpp': [(2, 9), (6, 15), (8, 2)], 'spam.py': [(2, 7), (5, 1), (7, 12)]}

/tmp/autopkgtest-lxc.3rwfusl7/downtmp/build.Dr5/src/spyder/plugins/findinfiles/widgets/tests/test_widgets.py:340: AssertionError
=============================== warnings summary ===============================

But on my local setup, all works as expected, running what is supposedly an identical lxc container. The only difference I can think of is in the locale (I’m in en_GB.UTF-8, while I’m guessing the ci.debian.net ones are en_US.UTF-8), but trying to run an en_US.UTF-8 container locally failed to reproduce the problem. It seems that what has happened here is that in some setups, this find process is looking at the files in a different order from in other setups. But it is still returning exactly 6 finds. I don’t know whether the order of file searching is deterministic across locales/setups, but I’m guessing that may be the source of this test failure.

Versions

  • Spyder version: 5.3.3
  • Python version: 3.10.6
  • Qt version:
  • PyQt version:
  • Operating System name/version: Debian/Linux testing

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (15 by maintainers)

Most upvoted comments

I think for now we should try to add the missing cases to the test. As you @dalthviz, I’m worried that trying to sort the search results could impact negatively the performance of this plugin.

Although it’s probably worth noting that if sticking with the current list-all-possibilities scheme, the assert can be simplified to:

    assert process_search_results(findinfiles.result_browser.data) in results

Thanks for submitting the PR adding the extra cases for the result! Looks a little bit ugly (adding all the possible cases) as you said but maybe it could be better than the sort usage (since thinking about it maybe with a large directory it could have an effect over perfomance).

What do you think @ccordoba12 @juliangilbey ? Could it be better to go for the moment with just adding the missing cases to the test or trying to sort the results on the os.walk loop?

Here’s possibly where @dalthviz fix could be implemented, but I’m wary of messing with it myself https://github.com/spyder-ide/spyder/blob/master/spyder/plugins/findinfiles/widgets/search_thread.py#L123

Here’s the result of process_search_results(findinfiles.result_browser.data) when I run it locally

{'spam.txt': [(1, 0), (1, 5), (3, 22)], 'spam.cpp': [(2, 9), (6, 15), (8, 2)]}

*edit: wrong sort order

OK now I kinda get what’s going on. The issue must be in findinfiles.find(), because it’s returning a non-deterministic result of files. For me, it’s finding spam.txt first, and then spam.cpp

a SIMPLE solution would be to just have that as another option for results (I’m creating PR now), but it’s kinda ugly

Ah, OK; I’ll submit a new PR (but it will be a few days before I have a chance).