scipy: BUG: Erratic results from RectBivariateSpline when smoothing large arrays
Describe your issue.
According to the documentation, RectBivariateSpline can be used for smoothing by setting s to a non zero value. This works on small grids, but on larger grids, using any s value other than zero produces a spline that evaluates to NaN everywhere. To be clear, this occurs on data that do not contain any NaN. The problem occurs without any warning or error message. The problem occurs irregularly (running the code sometimes work, but most of the time it does not).
The problem occurs on grid with dtype int as well as float.
The problem occurs on Windows 10 and mac OS.
Since the problem does not alway occur, I suspect it may be due to a failure to allocate memory. If enough memory is available, everything works fine. However, it the memory allocation fails, RectBivariateSpline returns a nonsensical result without informing the user about the memory error.
Reproducing Code Example
import numpy as np
from scipy import interpolate
s = 1 # any value greater than zero causes the problem
nx, ny = 1000, 1700 # Set to larger values to reproduce the problem more consistently
img = np.random.randint(0, 100, size=(nx, ny))
x = np.arange(nx)
y = np.arange(ny)
spl = interpolate.RectBivariateSpline(x, y, img, s=s)
print(np.isnan(spl(x, y)).any())
Error message
No error message
SciPy/NumPy/Python version information
1.9.3 1.23.5 sys.version_info(major=3, minor=10, micro=8, releaselevel=‘final’, serial=0)
1.10.0 1.24.1 sys.version_info(major=3, minor=10, micro=8, releaselevel=‘final’, serial=0)
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 20 (19 by maintainers)
Makes sort of sense but that’s not the final answer still. The ValueError is reasonable: s is too small. However it’s a bit worrying that s=1 gives some answer and s=2 does not. Normally I’d expect that it errors our of s is too small, and returns some result for larger values of s. This behavior seems to hint that your fix still papers over the underlying issue.
The patch also looks a tad strange with wildly different code flow for
nx == nmaxxandnx == nxe. Given thatnmaxxandnxeare most likely equal to each other, it’s very much not clear what is the effect of the control flow change.I still think it’s worth figuring out what actually triggers the problem.
There was no recent changes in or around RectBivariateSpline. git blame concurs: https://github.com/scipy/scipy/blame/v1.14.1/scipy/interpolate/_fitpack2.py#L1526-L1611
This is in the plan already:)
FFLAGSwill work. Meson only supports bog-standard env vars and prefer command line switches for everything else. And it comes with docs even:) https://mesonbuild.com/Reference-tables.html#compiler-and-linker-flag-environment-variablesNote though that env vars only works from a clean build, because they affect the configure stage (i.e.,
meson setup) and not a partial rebuild. So you can eithergit clean -xdfand set the env vars you want, or alternatively you can edit the relevantmeson.buildentry and addfortran_args: -fbounds-checktemporarily.Note that
-gis never needed, that’s already the default indev.pyand in general is controlled via setting the build type: https://mesonbuild.com/Builtin-options.html#details-for-buildtype.