scipy: BUG: stats.nct.pdf inconsistent behavior when compared to MATLAB and R
Describe your issue.
Hello,
When computing stats.nctpdf with large t and nc values, the function returns ‘inf’. This behavior is inconsistent with both MATLAB and R. This “bug” appeared with version scipy 1.10 (behavior was consistent before). I’m not sure whether it’s truly a bug or whether MATLAB and R are both wrong in not returning inf.
The following R code (equivalent to the Python code) returns 0.0043403 instead of ‘inf’ (R version 4.3.1):
t <- 8.45316107575123
N <- 25
df <- N-1
safe_int <- .9999
safe_range <- t / sqrt(N) + c(-1, 1) * qt(1 - (1 - safe_int) / 2, df) / sqrt(N)
dt(t, df, ncp=safe_range[[2]]*sqrt(N))
Same value is returned in MATLAB (R2023b):
t = 8.45316107575123;
N = 25;
df = N-1;
safe_int = .9999;
safe_range = t / sqrt(N) + [-1, 1] * tinv(1 - (1 - safe_int) / 2, df) / sqrt(N);
nctpdf(t, df, safe_range(2)*sqrt(N))
Reproducing Code Example
import numpy as np
from scipy import stats
t = 8.45316107575123
N = 25
df = N-1
safe_int = .9999
safe_range = t / np.sqrt(N) + np.array([-1, 1]) * stats.t.ppf(1 - (1 - safe_int) / 2, df=df) / np.sqrt(N)
# SciPy 1.9.3 returns 0.0043403; SciPy 1.10 and above return Inf
stats.nct.pdf(t, df, nc=safe_range[1]*np.sqrt(N))
Error message
N/A
SciPy/NumPy/Python version and system information
1.11.3 1.26.0 sys.version_info(major=3, minor=10, micro=12, releaselevel='final', serial=0)
Build Dependencies:
blas:
detection method: pkgconfig
found: true
include directory: C:/Users/constama/miniforge3/Library/include
lib directory: C:/Users/constama/miniforge3/Library/lib
name: blas
openblas configuration: unknown
pc file directory: C:\bld\scipy-split_1696467770591\_h_env\Library\lib\pkgconfig
version: 3.9.0
lapack:
detection method: pkgconfig
found: true
include directory: C:/Users/constama/miniforge3/Library/include
lib directory: C:/Users/constama/miniforge3/Library/lib
name: lapack
openblas configuration: unknown
pc file directory: C:\bld\scipy-split_1696467770591\_h_env\Library\lib\pkgconfig
version: 3.9.0
pybind11:
detection method: pkgconfig
include directory: C:/Users/constama/miniforge3/Library/include
name: pybind11
version: 2.11.1
Compilers:
c:
commands: clang-cl
linker: lld-link
name: clang-cl
version: 17.0.0
c++:
commands: clang-cl
linker: lld-link
name: clang-cl
version: 17.0.0
cython:
commands: cython
linker: cython
name: cython
version: 0.29.36
fortran:
commands: flang-new
linker: lld-link
name: flang
version: 17.0.0
pythran:
include directory: ..\..\_h_env\lib\site-packages\pythran
version: 0.14.0
Machine Information:
build:
cpu: x86_64
endian: little
family: x86_64
system: windows
cross-compiled: false
host:
cpu: x86_64
endian: little
family: x86_64
system: windows
Python Information:
path: C:\bld\scipy-split_1696467770591\_h_env\python.exe
version: '3.10'
About this issue
- Original URL
- State: open
- Created 9 months ago
- Comments: 25 (14 by maintainers)
This is our bad: it’s a resurfacing of https://github.com/boostorg/math/issues/184 as a result of accepting a PR we shouldn’t have 😦 The issue is now better tested and fixed again in 1.85 which will be out shortly (I hope). Apologies all round.
Update: that was tested against develop, there was an old root finding bug which resurfaced in 1.84 causing excessive iterations to be used, I need to double check that…