scipy: BUG: stats: `test_axis_nan_policy_axis_is_None[mixed-*-xp*]` fails on mips64el

Describe your issue.

In scipy 1.51.1, stats test test_axis_nan_policy_axis_is_None[mixed-*-xp*] in test_axis_nan_policy.py is failing on mips64el architecture. That’s test_axis_nan_policy_axis_is_None[mixed-propagate-xp_mean_1samp-args61-kwds61-1-1-False-<lambda>] test_axis_nan_policy_axis_is_None[mixed-propagate-xp_mean_2samp-args62-kwds62-2-1-True-<lambda>] test_axis_nan_policy_axis_is_None[mixed-propagate-xp_var-args63-kwds63-1-1-False-<lambda>] test_axis_nan_policy_axis_is_None[mixed-omit-xp_var-args63-kwds63-1-1-False-<lambda>]

Test logs can be found at https://buildd.debian.org/status/fetch.php?pkg=scipy&arch=mips64el&ver=1.15.1-1exp1&stamp=1737272238&raw=0

I’m not sure how big a deal it is. The debian build could just skip these 4 tests of course.

Reproducing Code Example

# on mips64el
python3 -c "import scipy.stats; scipy.stats.test(extra_argv=['-k', 'test_axis_nan_policy_axis_is_None[mixed-propagate or test_axis_nan_policy_axis_is_None[mixed-omit', '-v'])"

Error message

_ test_axis_nan_policy_axis_is_None[mixed-propagate-xp_mean_1samp-args61-kwds61-1-1-False-<lambda>] _

hypotest = <function xp_mean_1samp at 0xffdba5cfe0>, args = (), kwds = {}
n_samples = 1, n_outputs = 1, paired = False
unpacker = <function <lambda> at 0xffdba5ede0>, nan_policy = 'propagate'
data_generator = 'mixed'

    @pytest.mark.parametrize(("hypotest", "args", "kwds", "n_samples", "n_outputs",
                              "paired", "unpacker"), axis_nan_policy_cases)
    @pytest.mark.parametrize(("nan_policy"), ("propagate", "omit", "raise"))
    @pytest.mark.parametrize(("data_generator"),
                             ("all_nans", "all_finite", "mixed", "empty"))
    def test_axis_nan_policy_axis_is_None(hypotest, args, kwds, n_samples,
                                          n_outputs, paired, unpacker, nan_policy,
                                          data_generator):
        # check for correct behavior when `axis=None`
        if not unpacker:
            def unpacker(res):
                return res
    
        rng = np.random.default_rng(0)
    
        if data_generator == "empty":
            data = [rng.random((2, 0)) for i in range(n_samples)]
        else:
            data = [rng.random((2, 20)) for i in range(n_samples)]
    
        if data_generator == "mixed":
            masks = [rng.random((2, 20)) > 0.9 for i in range(n_samples)]
            for sample, mask in zip(data, masks):
                sample[mask] = np.nan
        elif data_generator == "all_nans":
            data = [sample * np.nan for sample in data]
    
        data_raveled = [sample.ravel() for sample in data]
    
        if nan_policy == 'raise' and data_generator not in {"all_finite", "empty"}:
            message = 'The input contains nan values'
    
            # check for correct behavior whether or not data is 1d to begin with
            with pytest.raises(ValueError, match=message):
                hypotest(*data, axis=None, nan_policy=nan_policy,
                         *args, **kwds)
            with pytest.raises(ValueError, match=message):
                hypotest(*data_raveled, axis=None, nan_policy=nan_policy,
                         *args, **kwds)
    
            return
    
        # behavior of reference implementation with 1d input, public function with 1d
        # input, and public function with Nd input and `axis=None` should be consistent.
        # This means:
        # - If the reference version raises an error or emits a warning, it's because
        #   the sample is too small, so check that the public function emits an
        #   appropriate "too small" warning
        # - Any results returned by the three versions should be the same.
        with warnings.catch_warnings():  # treat warnings as errors
            warnings.simplefilter("error")
    
            ea_str, eb_str, ec_str = None, None, None
            try:
                res1da = nan_policy_1d(hypotest, data_raveled, unpacker, *args,
                                       n_outputs=n_outputs, nan_policy=nan_policy,
                                       paired=paired, _no_deco=True, **kwds)
            except (RuntimeWarning, ValueError, ZeroDivisionError) as ea:
                res1da = None
                ea_str = str(ea)
    
            try:
>               res1db = hypotest(*data_raveled, *args, nan_policy=nan_policy, **kwds)

args       = ()
data       = [array([[0.63696169, 0.26978671, 0.04097352, 0.01652764, 0.81327024,
        0.91275558, 0.60663578, 0.72949656, 0.543...        nan, 0.13509651,        nan, 0.52535432,
        0.31024188, 0.48583536,        nan,        nan, 0.3577952 ]])]
data_generator = 'mixed'
data_raveled = [array([0.63696169, 0.26978671, 0.04097352, 0.01652764, 0.81327024,
       0.91275558, 0.60663578, 0.72949656, 0.54362...3,        nan, 0.13509651,        nan, 0.52535432,
       0.31024188, 0.48583536,        nan,        nan, 0.3577952 ])]
ea_str     = None
eb_str     = None
ec_str     = None
hypotest   = <function xp_mean_1samp at 0xffdba5cfe0>
kwds       = {}
mask       = array([[False, False, False, False, False, False, False, False, False,
        False, False, False, False, False, Fals...se, False,  True, False,
        False, False,  True, False,  True, False, False, False,  True,
         True, False]])
masks      = [array([[False, False, False, False, False, False, False, False, False,
        False, False, False, False, False, Fal...e, False,  True, False,
        False, False,  True, False,  True, False, False, False,  True,
         True, False]])]
n_outputs  = 1
n_samples  = 1
nan_policy = 'propagate'
paired     = False
res1da     = array([nan])
rng        = Generator(PCG64) at 0xFF7F53DFC0
sample     = array([[0.63696169, 0.26978671, 0.04097352, 0.01652764, 0.81327024,
        0.91275558, 0.60663578, 0.72949656, 0.5436...,        nan, 0.13509651,        nan, 0.52535432,
        0.31024188, 0.48583536,        nan,        nan, 0.3577952 ]])
unpacker   = <function <lambda> at 0xffdba5ede0>

scipy/stats/tests/test_axis_nan_policy.py:532: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
scipy/stats/tests/test_axis_nan_policy.py:45: in xp_mean_1samp
    return stats._stats_py._xp_mean(*args, **kwargs)
        args       = (array([0.63696169, 0.26978671, 0.04097352, 0.01652764, 0.81327024,
       0.91275558, 0.60663578, 0.72949656, 0.54362...,        nan, 0.13509651,        nan, 0.52535432,
       0.31024188, 0.48583536,        nan,        nan, 0.3577952 ]),)
        kwargs     = {'nan_policy': 'propagate'}
scipy/stats/_stats_py.py:10894: in _xp_mean
    return xp.mean(x, axis=axis, keepdims=keepdims)
        _          = 'propagate'
        axis       = None
        contains_nan = True
        dtype      = None
        keepdims   = False
        message    = 'After omitting NaNs, one or more sample arguments is too small; all returned values will be NaN. See documentation for sample size requirements.'
        nan_policy = 'propagate'
        weights    = None
        x          = array([0.63696169, 0.26978671, 0.04097352, 0.01652764, 0.81327024,
       0.91275558, 0.60663578, 0.72949656, 0.543624...73,        nan, 0.13509651,        nan, 0.52535432,
       0.31024188, 0.48583536,        nan,        nan, 0.3577952 ])
        xp         = <module 'scipy._lib.array_api_compat.numpy' from '/build/reproducible-path/scipy-1.15.1/.pybuild/cpython3_3.12_scipy/build/scipy/_lib/array_api_compat/numpy/__init__.py'>
/usr/lib/python3/dist-packages/numpy/core/fromnumeric.py:3504: in mean
    return _methods._mean(a, axis=axis, dtype=dtype,
        a          = array([0.63696169, 0.26978671, 0.04097352, 0.01652764, 0.81327024,
       0.91275558, 0.60663578, 0.72949656, 0.543624...73,        nan, 0.13509651,        nan, 0.52535432,
       0.31024188, 0.48583536,        nan,        nan, 0.3577952 ])
        axis       = None
        dtype      = None
        keepdims   = False
        kwargs     = {'keepdims': False}
        out        = None
        where      = <no value>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

a = array([0.63696169, 0.26978671, 0.04097352, 0.01652764, 0.81327024,
       0.91275558, 0.60663578, 0.72949656, 0.543624...73,        nan, 0.13509651,        nan, 0.52535432,
       0.31024188, 0.48583536,        nan,        nan, 0.3577952 ])
axis = None, dtype = None, out = None, keepdims = False

    def _mean(a, axis=None, dtype=None, out=None, keepdims=False, *, where=True):
        arr = asanyarray(a)
    
        is_float16_result = False
    
        rcount = _count_reduce_items(arr, axis, keepdims=keepdims, where=where)
        if rcount == 0 if where is True else umr_any(rcount == 0, axis=None):
            warnings.warn("Mean of empty slice.", RuntimeWarning, stacklevel=2)
    
        # Cast bool, unsigned int, and int to float64 by default
        if dtype is None:
            if issubclass(arr.dtype.type, (nt.integer, nt.bool_)):
                dtype = mu.dtype('f8')
            elif issubclass(arr.dtype.type, nt.float16):
                dtype = mu.dtype('f4')
                is_float16_result = True
    
>       ret = umr_sum(arr, axis, dtype, out, keepdims, where=where)
E       RuntimeWarning: invalid value encountered in reduce

a          = array([0.63696169, 0.26978671, 0.04097352, 0.01652764, 0.81327024,
       0.91275558, 0.60663578, 0.72949656, 0.543624...73,        nan, 0.13509651,        nan, 0.52535432,
       0.31024188, 0.48583536,        nan,        nan, 0.3577952 ])
arr        = array([0.63696169, 0.26978671, 0.04097352, 0.01652764, 0.81327024,
       0.91275558, 0.60663578, 0.72949656, 0.543624...73,        nan, 0.13509651,        nan, 0.52535432,
       0.31024188, 0.48583536,        nan,        nan, 0.3577952 ])
axis       = None
dtype      = None
is_float16_result = False
keepdims   = False
out        = None
rcount     = 40
where      = True

/usr/lib/python3/dist-packages/numpy/core/_methods.py:118: RuntimeWarning

SciPy/NumPy/Python version and system information

#
# info taken from amd64.  mips64el versions will be similar
#
1.15.1 1.26.4 sys.version_info(major=3, minor=13, micro=1, releaselevel='final', serial=0)
Build Dependencies:
  blas:
    detection method: pkgconfig
    found: true
    include directory: /usr/include/x86_64-linux-gnu
    lib directory: /usr/lib/x86_64-linux-gnu
    name: blas
    openblas configuration: unknown
    pc file directory: /usr/lib/x86_64-linux-gnu/pkgconfig
    version: 3.12.0
  lapack:
    detection method: pkgconfig
    found: true
    include directory: /usr/include/x86_64-linux-gnu
    lib directory: /usr/lib/x86_64-linux-gnu
    name: lapack
    openblas configuration: unknown
    pc file directory: /usr/lib/x86_64-linux-gnu/pkgconfig
    version: 3.12.0
  pybind11:
    detection method: pkgconfig
    include directory: /include
    name: pybind11
    version: 2.13.6
Compilers:
  c:
    args: -g, -O2, -Werror=implicit-function-declaration, -ffile-prefix-map=/build/scipy-1.15.1=.,
      -fstack-protector-strong, -fstack-clash-protection, -Wformat, -Werror=format-security,
      -fcf-protection, -Wdate-time, -D_FORTIFY_SOURCE=2
    commands: cc
    linker: ld.bfd
    linker args: -Wl,-z,relro, -g, -O2, -Werror=implicit-function-declaration, -ffile-prefix-map=/build/scipy-1.15.1=.,
      -fstack-protector-strong, -fstack-clash-protection, -Wformat, -Werror=format-security,
      -fcf-protection, -Wdate-time, -D_FORTIFY_SOURCE=2
    name: gcc
    version: 14.2.0
  c++:
    args: -g, -O2, -ffile-prefix-map=/build/scipy-1.15.1=., -fstack-protector-strong,
      -fstack-clash-protection, -Wformat, -Werror=format-security, -fcf-protection,
      -Wdate-time, -D_FORTIFY_SOURCE=2
    commands: c++
    linker: ld.bfd
    linker args: -Wl,-z,relro, -g, -O2, -ffile-prefix-map=/build/scipy-1.15.1=., -fstack-protector-strong,
      -fstack-clash-protection, -Wformat, -Werror=format-security, -fcf-protection,
      -Wdate-time, -D_FORTIFY_SOURCE=2
    name: gcc
    version: 14.2.0
  cython:
    commands: cython
    linker: cython
    name: cython
    version: 3.0.11
  fortran:
    args: -fPIC
    commands: gfortran
    linker: ld.bfd
    linker args: -Wl,-z,relro, -fPIC
    name: gcc
    version: 14.2.0
  pythran:
    include directory: ../../../usr/lib/python3/dist-packages/pythran
    version: 0.17.0
Python Information:
  path: /usr/bin/python3.13
  version: '3.13'

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 16 (16 by maintainers)

Commits related to this issue

Most upvoted comments

Yes, worth adding to the build as its own unit test. That’s what I meant by

If not, and if you have a development environment, can you make it a unit test and run it just like other SciPy tests?

I want to see if it is just due to mean emitting a warning when there are NaNs in the input or if there is something more complicated happening in test_axis_nam_policy.

If you’re able to trigger the problem with this little unit test, please also see if it can be further simplified and still trigger the proble, e.g.

Does axis need to be None? Does it happen for any array with NaNs in it? How about with a regular NumPy import? With np.asarray instead of asanyarray?