scipy: ENH: `interpolate.make_splrep`: fails for ill-conditioned problems

Describe your issue.

make_splrep( x, y, w=weights, k=order, s=smoothness, )

with:

x = 20.00000 153.81000 175.57000 202.47000 237.11000 253.61000 258.56000 273.40000 284.54000 293.61000 298.56000 301.86000 305.57000 307.22000 308.45000 310.10000 310.10000 310.50000

y = 53.00000 49.50000 48.60000 46.80000 43.20000 40.32000 39.60000 36.00000 32.40000 28.80000 25.20000 21.60000 18.00000 14.40000 10.80000 7.20000 3.60000 0.00000

w = 1.38723 1.38723 1.38723 1.38723 1.38723 1.38723 1.38723 1.38723 1.38723 1.38723 1.38723 1.38723 1.38723 1.38723 1.38723 1.38723 1.38723 1.38723

k = 2

s = 12.0

returns a RuntimeError ‘Internal error. Please report it to SciPy developers’.

Note that in this case x includes two duplicated values.

Running this without specifying the weights, produces a valid bspline object.

Also note that this same example run with the old splrep() and the same arguments, does not fail and produces a valid (t,c,k) for a bspline object.

Reproducing Code Example

from scipy.interpolate import make_splrep

x = [20.00000,
153.81000,
175.57000,
202.47000,
237.11000,
253.61000,
258.56000,
273.40000,
284.54000,
293.61000,
298.56000,
301.86000,
305.57000,
307.22000,
308.45000,
310.10000,
310.10000,
310.50000]

y = [53.00000,
49.50000,
48.60000,
46.80000,
43.20000,
40.32000,
39.60000,
36.00000,
32.40000,
28.80000,
25.20000,
21.60000,
18.00000,
14.40000,
10.80000,
7.20000,
3.60000,
0.00000]

w = [1.38723,
1.38723,
1.38723,
1.38723,
1.38723,
1.38723,
1.38723,
1.38723,
1.38723,
1.38723,
1.38723,
1.38723,
1.38723,
1.38723,
1.38723,
1.38723,
1.38723,
1.38723]

k = 2

s = 12.0

make_splrep(
    x,
    y,
    w=weights,
    k=order,
    s=smoothness,
)

Error message

networkmodelconnector.exceptions.BSplineFitException: Internal error. Please report it to SciPy developers.

SciPy/NumPy/Python version and system information

1.15.2 1.26.4 sys.version_info(major=3, minor=11, micro=11, releaselevel='final', serial=0)
Build Dependencies:
  blas:
    detection method: extraframeworks
    found: true
    include directory: unknown
    lib directory: unknown
    name: Accelerate
    openblas configuration: unknown
    pc file directory: unknown
    version: unknown
  lapack:
    detection method: extraframeworks
    found: true
    include directory: unknown
    lib directory: unknown
    name: Accelerate
    openblas configuration: unknown
    pc file directory: unknown
    version: unknown
  pybind11:
    detection method: config-tool
    include directory: unknown
    name: pybind11
    version: 2.13.6
Compilers:
  c:
    commands: cc
    linker: ld64
    name: clang
    version: 15.0.0
  c++:
    commands: c++
    linker: ld64
    name: clang
    version: 15.0.0
  cython:
    commands: cython
    linker: cython
    name: cython
    version: 3.0.12
  fortran:
    commands: gfortran
    linker: ld64
    name: gcc
    version: 13.3.0
  pythran:
    include directory: ../../../../../../private/var/folders/2s/h6hvv9ps03xgz_krkkstvq_r0000gn/T/pip-build-env-p7gx8hhw/overlay/lib/python3.11/site-packages/pythran
    version: 0.17.0
Machine Information:
  build:
    cpu: aarch64
    endian: little
    family: aarch64
    system: darwin
  cross-compiled: false
  host:
    cpu: aarch64
    endian: little
    family: aarch64
    system: darwin
Python Information:
  path: /private/var/folders/2s/h6hvv9ps03xgz_krkkstvq_r0000gn/T/cibw-run-4oaw7jjs/cp311-macosx_arm64/build/venv/bin/python
  version: '3.11'

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 16 (13 by maintainers)

Most upvoted comments

Several things seem to be going on here.

First, this OP statement does not seem to be correct:

this same example run with the old splrep() and the same arguments, does not fail and produces a valid (t,c,k) for a bspline object.

Using x, y and w from the OP,

In [5]: tck = splrep(x, y, w=w, k=2, s=12); BSpline(*tck)(x)
Out[5]: 
array([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
       nan, nan, nan, nan, nan])

In [6]: tck = splrep(x, y, k=2, s=12); BSpline(*tck)(x)
Out[6]: 
array([52.91315852, 50.14097155, 48.59291111, 46.25490503, 42.55321297,
       40.51648816, 39.7754699 , 36.31304728, 32.49105859, 28.60476531,
       25.27998696, 21.93667047, 17.1015815 , 14.58516652, 11.64192351,
        4.60491033,  4.60491033,  1.10886194])

So the weighted problem is ill-defined. The unweighted problem, however splrep does produce a non-nan result. Moreover, for the unweighted problem, make_splrep is equivalent to splrep:

In [18]: np.max(abs(make_splrep(x, y, k=2, s=12)(x) - BSpline(*splrep(x, y, k=2, s=12))(x)))
Out[18]: np.float64(2.842170943040401e-14)

Could you please confirm or clarify @jamesuber ? It’d be helpful to see a case where splrep produces a non-nan result but make_splrep fails.

Second, decreasing the value of s below some threshold , splrep starts emitting a warning (+returns nans):

In [44]: BSpline(*splrep(x, y, k=2, s=9.6))(x) - y
/home/br/repos/scipy/scipy/build-install/lib/python3.12/site-packages/scipy/interpolate/_fitpack_py.py:304: RuntimeWarning: The maximal number of iterations (20) allowed for finding smoothing
spline with fp=s has been reached. Probable cause: s too small.
(abs(fp-s)/s>0.001)
  res = _impl.splrep(x, y, w, xb, xe, k, task, s, t, full_output, per, quiet)
Out[44]: 
array([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
       nan, nan, nan, nan, nan])

At the same threshold, make_splrep fails with an “Internal SciPy error”. This is bad and should be caught earlier. Checking for a zero pivot and raising an informative error sounds like a good solution @czgdp1807