scipy: BUG: filter out unhelpful & noisy numpy warnings in implementation, not in tests

This was surfaced by #19804, causing me to take a look where those warnings were coming from. I was surprised that - in the example below at least - the warnings are duplicated between scipy & numpy (not caught by the new pytest 8 because the messages are identical).

Looking closer, we’re filtering out a lot of warnings in tests that should IMO really be filtered out in the implementations, rather than the tests.

As an example:

>>> from scipy import stats
>>> stats.skew([])
[...]/scipy/stats/_stats_py.py:1175: RuntimeWarning: Mean of empty slice.
  mean = a.mean(axis, keepdims=True)
[...]/numpy/core/_methods.py:121: RuntimeWarning: invalid value encountered in divide
  ret = um.true_divide(
[...]/numpy/core/fromnumeric.py:3504: RuntimeWarning: Mean of empty slice.
  return _methods._mean(a, axis=axis, dtype=dtype,
[...]/numpy/core/_methods.py:129: RuntimeWarning: invalid value encountered in scalar divide
  ret = ret.dtype.type(ret / rcount)
nan

4 warnings here are at least 3 too many, and we need to filter at least the ones from numpy; though IMO even “Mean of empty slice” is not helpful, and we could perhaps just return np.nan without a warning.

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Comments: 21 (21 by maintainers)

Commits related to this issue

Most upvoted comments

Are these warnings that were filtered/suppressed in the test (and that’s why they didn’t error)? (Surely there are other uses of warning filters and np.testing.suppress_warnings besides these?)

To be honest I don’t know. I just noted the output from pytest and copied it here. I’d be as surprised as you though if pytest shows warnings here that had explicitly been captured.

One thing to maybe check is your pytest version - v8 got much stricter if a function raises several warnings (rather than just taking the first one)

…it becomes a problem specifically for 'omit' and even more so in an n-D case where only one slice becomes too small?

Yes, exactly. We didn’t think it made sense for a single short slice to cause the whole operation to error out.

The 1-D behavior with raising errors is quite reasonable (probably even preferred)

In some contexts, yes. Then again, there’s also an argument to be made for consistency within SciPy and between SciPy and NumPy. np.mean and np.var could have errored out, but for better or for worse, they set an example of returning nan with empty input. That’s an appropriate result (if one is going to be returned) because the statistic is undefined. Many important SciPy functions (the moment functions, the t-tests, wilcoxon) followed suit. So it would make some sense for the rest to do the same, even in the 1D case, rather than the user having to guess.

Cases in which there are enough obvservations to calculate a numerical value of a statistic but not enough for the traditional, asymptotic p-value approximation to be accurate are bit more gray. But in those cases, I’d say we should just give a permutation or Monte Carlo p-value[^1]. At this point, it’s very easy and quite fast to do so (e.g. see method of pearsonr).

[^1]: The downsides are that the Monte Carlo p-values are stochastic and the permutation p-values, while they can be be deterministic and exact for small samples, technically change the interpretation of the test… There are solutions, but this is already getting too long : )

Having fiddled a little bit with special test suite recently, it is like a boeing 747. Wonderful to look at go, but almost impossible to do surgery on it or modernize. I still don’t know what it does in certain places and hoping the code is correct enough. So not sure what the right way to go about this since there are also warnings coming up from the internals of the tests.