scipy: BUG: error for arrays on non-default device

Describe your issue.

scipy.special.logsumexp raise ValueError when called on an array with non-default device. This was seen in https://github.com/scikit-learn/scikit-learn/pull/30777.

Reproducing Code Example

from scipy.special import logsumexp
import array_api_strict
arr = array_api_strict.asarray([1, 2, 3], device=array_api_strict.Device('device1'))
logsumexp(arr)

Error message

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[4], line 1
----> 1 logsumexp(arr)

File ~/micromamba/envs/scikit-learn-dev/lib/python3.13/site-packages/scipy/special/_logsumexp.py:118, in logsumexp(a, axis, b, keepdims, return_sign)
    116 if xp_size(a) != 0:
    117     with np.errstate(divide='ignore', invalid='ignore'):  # log of zero is OK
--> 118         out, sgn = _logsumexp(a, b, axis=axis, return_sign=return_sign, xp=xp)
    119 else:
    120     shape = np.asarray(a.shape)  # NumPy is convenient for shape manipulation

File ~/micromamba/envs/scikit-learn-dev/lib/python3.13/site-packages/scipy/special/_logsumexp.py:215, in _logsumexp(a, b, axis, return_sign, xp)
    209 m = (xp.sum(i_max_dt, axis=axis, keepdims=True, dtype=a.dtype) if b is None
    210      else xp.sum(b * i_max_dt, axis=axis, keepdims=True, dtype=a.dtype))
    212 # Arithmetic between infinities will introduce NaNs.
    213 # `+ a_max` at the end naturally corrects for removing them here.
    214 # TODO hack device=a_max.device
--> 215 shift = xp.where(xp.isfinite(a_max), a_max, xp.asarray(0, dtype=a_max.dtype)) # , device=a_max.device))
    217 # Shift, exponentiate, scale, and sum
    218 exp = b * xp.exp(a - shift) if b is not None else xp.exp(a - shift)

File ~/micromamba/envs/scikit-learn-dev/lib/python3.13/site-packages/array_api_strict/_searching_functions.py:124, in where(condition, x1, x2)
    121     raise TypeError("`condition` must be have a boolean data type")
    123 if len({a.device for a in (condition, x1, x2)}) > 1:
--> 124     raise ValueError("Inputs to `where` must all use the same device")
    126 x1, x2 = Array._normalize_two_args(x1, x2)
    127 return Array._new(np.where(condition._array, x1._array, x2._array), device=x1.device)

ValueError: Inputs to `where` must all use the same device

SciPy/NumPy/Python version and system information

1.15.2 2.2.1 sys.version_info(major=3, minor=13, micro=0, releaselevel='final', serial=0)
Build Dependencies:
  blas:
    detection method: pkgconfig
    found: true
    include directory: /home/lesteve/micromamba/envs/scikit-learn-dev/include
    lib directory: /home/lesteve/micromamba/envs/scikit-learn-dev/lib
    name: blas
    openblas configuration: unknown
    pc file directory: /home/lesteve/micromamba/envs/scikit-learn-dev/lib/pkgconfig
    version: 3.9.0
  lapack:
    detection method: pkgconfig
    found: true
    include directory: /home/lesteve/micromamba/envs/scikit-learn-dev/include
    lib directory: /home/lesteve/micromamba/envs/scikit-learn-dev/lib
    name: lapack
    openblas configuration: unknown
    pc file directory: /home/lesteve/micromamba/envs/scikit-learn-dev/lib/pkgconfig
    version: 3.9.0
  pybind11:
    detection method: pkgconfig
    include directory: /home/lesteve/micromamba/envs/scikit-learn-dev/include
    name: pybind11
    version: 2.13.6
Compilers:
  c:
    args: -march=nocona, -mtune=haswell, -ftree-vectorize, -fPIC, -fstack-protector-strong,
      -fno-plt, -O2, -ffunction-sections, -pipe, -isystem, /home/lesteve/micromamba/envs/scikit-learn-dev/include,
      -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/scipy-split_1739790645292/work=/usr/local/src/conda/scipy-split-1.15.2,
      -fdebug-prefix-map=/home/lesteve/micromamba/envs/scikit-learn-dev=/usr/local/src/conda-prefix,
      -DNDEBUG, -D_FORTIFY_SOURCE=2, -O2, -isystem, /home/lesteve/micromamba/envs/scikit-learn-dev/include
    commands: /home/conda/feedstock_root/build_artifacts/scipy-split_1739790645292/_build_env/bin/x86_64-conda-linux-gnu-cc
    linker: ld.bfd
    linker args: -Wl,-O2, -Wl,--sort-common, -Wl,--as-needed, -Wl,-z,relro, -Wl,-z,now,
      -Wl,--disable-new-dtags, -Wl,--gc-sections, -Wl,--allow-shlib-undefined, -Wl,-rpath,/home/lesteve/micromamba/envs/scikit-learn-dev/lib,
      -Wl,-rpath-link,/home/lesteve/micromamba/envs/scikit-learn-dev/lib, -L/home/lesteve/micromamba/envs/scikit-learn-dev/lib,
      -march=nocona, -mtune=haswell, -ftree-vectorize, -fPIC, -fstack-protector-strong,
      -fno-plt, -O2, -ffunction-sections, -pipe, -isystem, /home/lesteve/micromamba/envs/scikit-learn-dev/include,
      -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/scipy-split_1739790645292/work=/usr/local/src/conda/scipy-split-1.15.2,
      -fdebug-prefix-map=/home/lesteve/micromamba/envs/scikit-learn-dev=/usr/local/src/conda-prefix,
      -DNDEBUG, -D_FORTIFY_SOURCE=2, -O2, -isystem, /home/lesteve/micromamba/envs/scikit-learn-dev/include
    name: gcc
    version: 13.3.0
  c++:
    args: -fvisibility-inlines-hidden, -fmessage-length=0, -march=nocona, -mtune=haswell,
      -ftree-vectorize, -fPIC, -fstack-protector-strong, -fno-plt, -O2, -ffunction-sections,
      -pipe, -isystem, /home/lesteve/micromamba/envs/scikit-learn-dev/include, -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/scipy-split_1739790645292/work=/usr/local/src/conda/scipy-split-1.15.2,
      -fdebug-prefix-map=/home/lesteve/micromamba/envs/scikit-learn-dev=/usr/local/src/conda-prefix,
      -DNDEBUG, -D_FORTIFY_SOURCE=2, -O2, -isystem, /home/lesteve/micromamba/envs/scikit-learn-dev/include
    commands: /home/conda/feedstock_root/build_artifacts/scipy-split_1739790645292/_build_env/bin/x86_64-conda-linux-gnu-c++
    linker: ld.bfd
    linker args: -Wl,-O2, -Wl,--sort-common, -Wl,--as-needed, -Wl,-z,relro, -Wl,-z,now,
      -Wl,--disable-new-dtags, -Wl,--gc-sections, -Wl,--allow-shlib-undefined, -Wl,-rpath,/home/lesteve/micromamba/envs/scikit-learn-dev/lib,
      -Wl,-rpath-link,/home/lesteve/micromamba/envs/scikit-learn-dev/lib, -L/home/lesteve/micromamba/envs/scikit-learn-dev/lib,
      -fvisibility-inlines-hidden, -fmessage-length=0, -march=nocona, -mtune=haswell,
      -ftree-vectorize, -fPIC, -fstack-protector-strong, -fno-plt, -O2, -ffunction-sections,
      -pipe, -isystem, /home/lesteve/micromamba/envs/scikit-learn-dev/include, -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/scipy-split_1739790645292/work=/usr/local/src/conda/scipy-split-1.15.2,
      -fdebug-prefix-map=/home/lesteve/micromamba/envs/scikit-learn-dev=/usr/local/src/conda-prefix,
      -DNDEBUG, -D_FORTIFY_SOURCE=2, -O2, -isystem, /home/lesteve/micromamba/envs/scikit-learn-dev/include
    name: gcc
    version: 13.3.0
  cython:
    commands: cython
    linker: cython
    name: cython
    version: 3.0.12
  fortran:
    args: -march=nocona, -mtune=haswell, -ftree-vectorize, -fPIC, -fstack-protector-strong,
      -fno-plt, -O2, -ffunction-sections, -pipe, -isystem, /home/lesteve/micromamba/envs/scikit-learn-dev/include,
      -I/home/conda/feedstock_root/build_artifacts/scipy-split_1739790645292/_build_env/include,
      -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/scipy-split_1739790645292/work=/usr/local/src/conda/scipy-split-1.15.2,
      -fdebug-prefix-map=/home/lesteve/micromamba/envs/scikit-learn-dev=/usr/local/src/conda-prefix
    commands: /home/conda/feedstock_root/build_artifacts/scipy-split_1739790645292/_build_env/bin/x86_64-conda-linux-gnu-gfortran
    linker: ld.bfd
    linker args: -Wl,-O2, -Wl,--sort-common, -Wl,--as-needed, -Wl,-z,relro, -Wl,-z,now,
      -Wl,--disable-new-dtags, -Wl,--gc-sections, -Wl,--allow-shlib-undefined, -Wl,-rpath,/home/lesteve/micromamba/envs/scikit-learn-dev/lib,
      -Wl,-rpath-link,/home/lesteve/micromamba/envs/scikit-learn-dev/lib, -L/home/lesteve/micromamba/envs/scikit-learn-dev/lib,
      -march=nocona, -mtune=haswell, -ftree-vectorize, -fPIC, -fstack-protector-strong,
      -fno-plt, -O2, -ffunction-sections, -pipe, -isystem, /home/lesteve/micromamba/envs/scikit-learn-dev/include,
      -I/home/conda/feedstock_root/build_artifacts/scipy-split_1739790645292/_build_env/include,
      -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/scipy-split_1739790645292/work=/usr/local/src/conda/scipy-split-1.15.2,
      -fdebug-prefix-map=/home/lesteve/micromamba/envs/scikit-learn-dev=/usr/local/src/conda-prefix
    name: gcc
    version: 13.3.0
  pythran:
    include directory: ../../_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.13/site-packages/pythran
    version: 0.17.0
Machine Information:
  build:
    cpu: x86_64
    endian: little
    family: x86_64
    system: linux
  cross-compiled: false
  host:
    cpu: x86_64
    endian: little
    family: x86_64
    system: linux
Python Information:
  path: /home/lesteve/micromamba/envs/scikit-learn-dev/bin/python
  version: '3.13'

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 16 (16 by maintainers)

Most upvoted comments

I’d keep it simple and use the first argument.

  • IF the backend actually has heuristics that auto-resolve device mismatches,
  • AND the user is dissatisfied with the default output,
  • THEN they can just explicitly coerce all inputs to the same device.
  • OR they can open a ticket upstream stating why in their opinion the heuristics are bad (e.g. because they chose to transfer the larger array to the device of the smaller array instead of the other way around).

Again, nothing that scipy should dabble in.

I will do a cleanup of passing arrays to xp.where and such. These functions accept Python scalars now, so this sort of problem (which is probably very common) will be alleviated.

That said, I suppose we need to remember that when creating new arrays inside functions, we need to match device in addition to dtype.

Not sure how we’re going to test this automatically. I imagine our GPU job only tests with GPU as the default device.