scipy: BUG: warning from `optimize.least_squares` for astropy with OpenBLAS 0.3.26-dev
Describe your issue.
When running astropy’s test suite locally I see one test behaving differently depending on
which version of scipy I have installed.
Namely, I get a warning with scipy 1.13.0, which doesn’t appear with 1.12.0.
Specifically, I see this problem with the following wheel scipy-1.13.0-cp312-cp312-macosx_12_0_arm64.whl
I’ve investigated the problem but most of what I tried to gather information failed. In particular
- I wasn’t able to reproduce without astropy (or to be more transparent, I haven’t had the time to remove astropy from my reproducer yet)
- I wasn’t able to reproduce with local builds of scipy (I can build it, but the warning never shows up), so I couldn’t bisect it, and I’m suspecting the issue might be in the wheel building process (?)
That said, I’ll try to be as thorough as possible in my report with what I did find. For completeness, here’s the script I used for building scipy locally
set -euxo pipefail
export PKG_CONFIG_PATH="/opt/homebrew/opt/openblas/lib/pkgconfig"
pushd _scipy
git submodule update --init
rm -fr dist
python -m build --wheel
popd
python -m pip install ./_scipy/dist/*.whl
where _scipy is my local clone of the repo.
I tried checking in the main branch (currently 53a9e10241814b70c0975e099588bdef09006694) as well as the v1.13.0 tag, both of which behave fine when built locally (no warning is emitted).
Here’s my env’s info with the local build
Env info (scipy 1.13.0 built locally)
1.13.0 1.26.4 sys.version_info(major=3, minor=12, micro=2, releaselevel='final', serial=0)
Build Dependencies:
blas:
detection method: pkgconfig
found: true
include directory: /opt/homebrew/Cellar/openblas/0.3.27/include
lib directory: /opt/homebrew/Cellar/openblas/0.3.27/lib
name: openblas
openblas configuration: USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP=1 VORTEX MAX_THREADS=56
pc file directory: /opt/homebrew/opt/openblas/lib/pkgconfig
version: 0.3.27
lapack:
detection method: pkgconfig
found: true
include directory: /opt/homebrew/Cellar/openblas/0.3.27/include
lib directory: /opt/homebrew/Cellar/openblas/0.3.27/lib
name: openblas
openblas configuration: USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP=1 VORTEX MAX_THREADS=56
pc file directory: /opt/homebrew/opt/openblas/lib/pkgconfig
version: 0.3.27
pybind11:
detection method: config-tool
include directory: unknown
name: pybind11
version: 2.12.0
Compilers:
c:
commands: cc
linker: ld64
name: clang
version: 15.0.0
c++:
commands: c++
linker: ld64
name: clang
version: 15.0.0
cython:
commands: cython
linker: cython
name: cython
version: 3.0.10
fortran:
commands: gfortran
linker: ld64
name: gcc
version: 13.2.0
pythran:
include directory: ../../../../../../../private/var/folders/47/t9k4zdz96vzd4v1rxyv07nvh0000gn/T/build-env-wzebhths/lib/python3.12/site-packages/pythran
version: 0.15.0
Machine Information:
build:
cpu: aarch64
endian: little
family: aarch64
system: darwin
cross-compiled: false
host:
cpu: aarch64
endian: little
family: aarch64
system: darwin
Python Information:
path: /private/var/folders/47/t9k4zdz96vzd4v1rxyv07nvh0000gn/T/build-env-wzebhths/bin/python
version: '3.12'
Reproducing Code Example
from astropy.modeling.models import Gaussian1D
from scipy.optimize import least_squares
import numpy as np
def objective_function(fps, *args):
"Function to minimize."
model, x, y = args
fit_param_indices = list(range(len(model.param_names)))
parameters = model.parameters
fit_param_indices = set(fit_param_indices)
offset = 0
param_metrics = model._param_metrics
for name in model.param_names:
slice_ = param_metrics[name]["slice"]
values = fps[offset : offset + 1]
parameters[slice_] = values
offset += 1
# Update model parameters before calling ``tied`` constraints.
model._array_to_parameters()
return np.ravel(model(x) - y)
def dfunc(params, model, x, y):
fit_deriv = np.array(model.fit_deriv(x, *params))
return np.array([np.ravel(_) for _ in np.array(1.0) * fit_deriv]).T
inf = float("inf")
least_squares(
fun=objective_function,
x0=np.array([1.0e-03, 3.0e00, 2.0e-01]),
args=(
Gaussian1D(amplitude=0.001, mean=3.0, stddev=0.2),
np.linspace(-500, 500, 2000, endpoint=True),
np.ones(2000),
),
jac=dfunc,
bounds=((-inf, -inf, 1.1754943508222875e-38), (inf, inf, inf)),
)
Error message
/Users/clm/.pyenv/versions/3.12.2/envs/astropy.dev/lib/python3.12/site-packages/scipy/optimize/_lsq/common.py:398: RuntimeWarning: invalid value encountered in cast
return min_step, np.equal(steps, min_step) * np.sign(s).astype(int)
SciPy/NumPy/Python version and system information
1.13.0 1.26.4 sys.version_info(major=3, minor=12, micro=2, releaselevel='final', serial=0)
Build Dependencies:
blas:
detection method: pkgconfig
found: true
include directory: /opt/arm64-builds/include
lib directory: /opt/arm64-builds/lib
name: openblas
openblas configuration: USE_64BITINT=0 DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= SANDYBRIDGE MAX_THREADS=64
pc file directory: /opt/arm64-builds/lib/pkgconfig
version: 0.3.26.dev
lapack:
detection method: pkgconfig
found: true
include directory: /opt/arm64-builds/include
lib directory: /opt/arm64-builds/lib
name: openblas
openblas configuration: USE_64BITINT=0 DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= SANDYBRIDGE MAX_THREADS=64
pc file directory: /opt/arm64-builds/lib/pkgconfig
version: 0.3.26.dev
pybind11:
detection method: config-tool
include directory: unknown
name: pybind11
version: 2.12.0
Compilers:
c:
commands: cc
linker: ld64
name: clang
version: 15.0.0
c++:
commands: c++
linker: ld64
name: clang
version: 15.0.0
cython:
commands: cython
linker: cython
name: cython
version: 3.0.10
fortran:
commands: gfortran
linker: ld64
name: gcc
version: 13.2.0
pythran:
include directory: ../../../../../../private/var/folders/l_/53d72k7j4zl9kj8fzncsd8hw0000gn/T/pip-build-env-mabxfhvc/overlay/lib/python3.12/site-packages/pythran
version: 0.15.0
Machine Information:
build:
cpu: aarch64
endian: little
family: aarch64
system: darwin
cross-compiled: false
host:
cpu: aarch64
endian: little
family: aarch64
system: darwin
Python Information:
path: /private/var/folders/l_/53d72k7j4zl9kj8fzncsd8hw0000gn/T/cibw-run-ehybylsk/cp312-macosx_arm64/build/venv/bin/python
version: '3.12'
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 32 (20 by maintainers)
See #20569
FYI, today I’m trying to reduce the reproducer down as much as I can. I just reached the point where it no longer depends on astropy but I still have ~1200 lines of vendored code. I’ll melt it down some more but at least now it’s isolated.