scipy: BUG: stats.nct.pdf inconsistent behavior when compared to MATLAB and R

Describe your issue.

Hello,

When computing stats.nctpdf with large t and nc values, the function returns ‘inf’. This behavior is inconsistent with both MATLAB and R. This “bug” appeared with version scipy 1.10 (behavior was consistent before). I’m not sure whether it’s truly a bug or whether MATLAB and R are both wrong in not returning inf.

The following R code (equivalent to the Python code) returns 0.0043403 instead of ‘inf’ (R version 4.3.1):

t <- 8.45316107575123
N <- 25
df <- N-1
safe_int <- .9999
safe_range <- t / sqrt(N) + c(-1, 1) * qt(1 - (1 - safe_int) / 2, df) / sqrt(N)

dt(t, df, ncp=safe_range[[2]]*sqrt(N))

Same value is returned in MATLAB (R2023b):

t = 8.45316107575123;
N = 25;
df = N-1;
safe_int = .9999;
safe_range = t / sqrt(N) + [-1, 1] * tinv(1 - (1 - safe_int) / 2, df) / sqrt(N);

nctpdf(t, df, safe_range(2)*sqrt(N))

Reproducing Code Example

import numpy as np
from scipy import stats

t = 8.45316107575123
N = 25
df = N-1
safe_int = .9999
safe_range = t / np.sqrt(N) + np.array([-1, 1]) * stats.t.ppf(1 - (1 - safe_int) / 2, df=df) / np.sqrt(N)

# SciPy 1.9.3 returns 0.0043403; SciPy 1.10 and above return Inf
stats.nct.pdf(t, df, nc=safe_range[1]*np.sqrt(N))

Error message

N/A

SciPy/NumPy/Python version and system information

1.11.3 1.26.0 sys.version_info(major=3, minor=10, micro=12, releaselevel='final', serial=0)
Build Dependencies:
  blas:
    detection method: pkgconfig
    found: true
    include directory: C:/Users/constama/miniforge3/Library/include
    lib directory: C:/Users/constama/miniforge3/Library/lib
    name: blas
    openblas configuration: unknown
    pc file directory: C:\bld\scipy-split_1696467770591\_h_env\Library\lib\pkgconfig
    version: 3.9.0
  lapack:
    detection method: pkgconfig
    found: true
    include directory: C:/Users/constama/miniforge3/Library/include
    lib directory: C:/Users/constama/miniforge3/Library/lib
    name: lapack
    openblas configuration: unknown
    pc file directory: C:\bld\scipy-split_1696467770591\_h_env\Library\lib\pkgconfig
    version: 3.9.0
  pybind11:
    detection method: pkgconfig
    include directory: C:/Users/constama/miniforge3/Library/include
    name: pybind11
    version: 2.11.1
Compilers:
  c:
    commands: clang-cl
    linker: lld-link
    name: clang-cl
    version: 17.0.0
  c++:
    commands: clang-cl
    linker: lld-link
    name: clang-cl
    version: 17.0.0
  cython:
    commands: cython
    linker: cython
    name: cython
    version: 0.29.36
  fortran:
    commands: flang-new
    linker: lld-link
    name: flang
    version: 17.0.0
  pythran:
    include directory: ..\..\_h_env\lib\site-packages\pythran
    version: 0.14.0
Machine Information:
  build:
    cpu: x86_64
    endian: little
    family: x86_64
    system: windows
  cross-compiled: false
  host:
    cpu: x86_64
    endian: little
    family: x86_64
    system: windows
Python Information:
  path: C:\bld\scipy-split_1696467770591\_h_env\python.exe
  version: '3.10'

About this issue

  • Original URL
  • State: open
  • Created 9 months ago
  • Comments: 25 (14 by maintainers)

Commits related to this issue

Most upvoted comments

This is our bad: it’s a resurfacing of https://github.com/boostorg/math/issues/184 as a result of accepting a PR we shouldn’t have 😦 The issue is now better tested and fixed again in 1.85 which will be out shortly (I hope). Apologies all round.

Update: that was tested against develop, there was an old root finding bug which resurfaced in 1.84 causing excessive iterations to be used, I need to double check that…