scipy: Failures for new `pytest-fail-slow` check in Windows CI jobs

Issue to keep track of spurious failures after gh-20672 was merged yesterday.

This one showed up on gh-17924:

 ______________ TestDifferentialEvolutionSolver.test_strategy_fn _______________
[gw0] win32 -- Python 3.12.3 C:\hostedtoolcache\windows\Python\3.12.3\x64\python.exe
Test passed but took too long to run: Duration 1.9977539000001343s > 1.0s

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 25 (25 by maintainers)

Commits related to this issue

Most upvoted comments

This extra test didn’t fail the runtime check in the PR so any fails will be sporadic.

No, it would have failed, but CI wasn’t run on gh-20757 after the relevant fail-slow PR (gh-20672) merged.

I think the trial policy to have tests run under a second is going create noise from sporadic fails that’ll take extra work to quieten.

Noted again.

But we still haven’t seen real noise yet. Already some good has come out of it: the fail slow marker revealed that COBYQA was uneccessarily slow. Here, it only came up because the two PRs were happening in parallel. To lessen the burden of applying the mark to the test, I’ll go ahead and submit a PR and ping you.

I’d be fine removing pytest-fail-slow it if we really do have trouble with it, though.