scipy: Failures for new `pytest-fail-slow` check in Windows CI jobs
Issue to keep track of spurious failures after gh-20672 was merged yesterday.
This one showed up on gh-17924:
______________ TestDifferentialEvolutionSolver.test_strategy_fn _______________
[gw0] win32 -- Python 3.12.3 C:\hostedtoolcache\windows\Python\3.12.3\x64\python.exe
Test passed but took too long to run: Duration 1.9977539000001343s > 1.0s
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 25 (25 by maintainers)
Commits related to this issue
- CI: limit pytest-fail-slow usage to a single CI job Avoids spurious failures on PRs, which happen frequently. Closes gh-20806 — committed to rgommers/scipy by rgommers 2 years ago
- CI: limit pytest-fail-slow usage to a single CI job Avoids spurious failures on PRs, which happen frequently. Closes gh-20806 — committed to rgommers/scipy by rgommers 2 years ago
- CI: limit pytest-fail-slow usage to a single CI job Avoids spurious failures on PRs, which happen frequently. Closes gh-20806 — committed to rgommers/scipy by rgommers 2 years ago
No, it would have failed, but CI wasn’t run on gh-20757 after the relevant fail-slow PR (gh-20672) merged.
Noted again.
But we still haven’t seen real noise yet. Already some good has come out of it: the fail slow marker revealed that COBYQA was uneccessarily slow. Here, it only came up because the two PRs were happening in parallel. To lessen the burden of applying the mark to the test, I’ll go ahead and submit a PR and ping you.
I’d be fine removing
pytest-fail-slowit if we really do have trouble with it, though.