scipy: BUG/DOC: sparse: ND COO unexpected behaviour 1.15.0rc1

Describe your issue.

csc_array accepts dense array-likes since 1.15.0rc1, and accepts other csc_array instances with ndim > 2, but raises a ValueError if passed a coo_array with ndim > 2.

Reproducing Code Example

>>> import numpy as np
>>> from scipy.sparse import coo_array, csc_array
>>> csc_array(np.zeros((2, 3, 4)))
<Compressed Sparse Column sparse array of dtype 'float64'
	with 0 stored elements and shape (2, 3, 4)>
>>> coo_array(np.zeros((2, 3, 4)))
<COOrdinate sparse array of dtype 'float64'
	with 0 stored elements and shape (2, 3, 4)>
>>> csc_array(_)
Traceback (most recent call last):
  File "<python-input-10>", line 1, in <module>
    csc_array(_)
    ~~~~~~~~~^^^
  File "/home/joren/.pyenv/versions/3.13.1/lib/python3.13/site-packages/scipy/sparse/_compressed.py", line 36, in __init__
    arg1 = arg1.asformat(self.format)
  File "/home/joren/.pyenv/versions/3.13.1/lib/python3.13/site-packages/scipy/sparse/_base.py", line 456, in asformat
    return convert_method(copy=copy)
  File "/home/joren/.pyenv/versions/3.13.1/lib/python3.13/site-packages/scipy/sparse/_coo.py", line 339, in tocsc
    raise ValueError(f'Cannot convert. CSC format must be 2D. Got {self.ndim}D')
ValueError: Cannot convert. CSC format must be 2D. Got 3D

Error message

Traceback (most recent call last):
  File "<python-input-10>", line 1, in <module>
    csc_array(_)
    ~~~~~~~~~^^^
  File "/home/joren/.pyenv/versions/3.13.1/lib/python3.13/site-packages/scipy/sparse/_compressed.py", line 36, in __init__
    arg1 = arg1.asformat(self.format)
  File "/home/joren/.pyenv/versions/3.13.1/lib/python3.13/site-packages/scipy/sparse/_base.py", line 456, in asformat
    return convert_method(copy=copy)
  File "/home/joren/.pyenv/versions/3.13.1/lib/python3.13/site-packages/scipy/sparse/_coo.py", line 339, in tocsc
    raise ValueError(f'Cannot convert. CSC format must be 2D. Got {self.ndim}D')
ValueError: Cannot convert. CSC format must be 2D. Got 3D

SciPy/NumPy/Python version and system information

1.15.0rc1 2.2.0 sys.version_info(major=3, minor=13, micro=1, releaselevel='final', serial=0)
Build Dependencies:
  blas:
    detection method: pkgconfig
    found: true
    include directory: /opt/_internal/cpython-3.13.0/lib/python3.13/site-packages/scipy_openblas32/include
    lib directory: /opt/_internal/cpython-3.13.0/lib/python3.13/site-packages/scipy_openblas32/lib
    name: scipy-openblas
    openblas configuration: OpenBLAS 0.3.28 DYNAMIC_ARCH NO_AFFINITY Haswell MAX_THREADS=64
    pc file directory: /project
    version: 0.3.28
  lapack:
    detection method: pkgconfig
    found: true
    include directory: /opt/_internal/cpython-3.13.0/lib/python3.13/site-packages/scipy_openblas32/include
    lib directory: /opt/_internal/cpython-3.13.0/lib/python3.13/site-packages/scipy_openblas32/lib
    name: scipy-openblas
    openblas configuration: OpenBLAS 0.3.28 DYNAMIC_ARCH NO_AFFINITY Haswell MAX_THREADS=64
    pc file directory: /project
    version: 0.3.28
  pybind11:
    detection method: config-tool
    include directory: unknown
    name: pybind11
    version: 2.13.6
Compilers:
  c:
    commands: cc
    linker: ld.bfd
    name: gcc
    version: 10.2.1
  c++:
    commands: c++
    linker: ld.bfd
    name: gcc
    version: 10.2.1
  cython:
    commands: cython
    linker: cython
    name: cython
    version: 3.0.11
  fortran:
    commands: gfortran
    linker: ld.bfd
    name: gcc
    version: 10.2.1
  pythran:
    include directory: ../../tmp/pip-build-env-fa6gfmf0/overlay/lib/python3.13/site-packages/pythran
    version: 0.17.0
Machine Information:
  build:
    cpu: x86_64
    endian: little
    family: x86_64
    system: linux
  cross-compiled: false
  host:
    cpu: x86_64
    endian: little
    family: x86_64
    system: linux
Python Information:
  path: /opt/python/cp313-cp313/bin/python
  version: '3.13'

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 22 (22 by maintainers)

Most upvoted comments

The correct annotation for the axes input to tensordot should match the annotation for numpy tensordot. It might also show up better in the array api standard documentation. So I think it is Union[int, Tuple[Sequence[int], Sequence[int]]]

But I prefer the description in the docs to the annotation (cuz I don’t know enough about annotations to read them).

axes : integer_like or length 2 array_like
        * integer_like
          If an int N, sum over the last N axes of `a` and the first N axes
          of `b` in order. The sizes of the corresponding axes must match.
        * (2,) array_like
          Or, a list of two axes sequences to be summed over, first sequence
          applying to `a`, second to `b`. Both elements of array_like must be the same length. 

So, axes=(1,2) is expected to raise, while axes=((1,2), (2,0)) is allowed (if the integers are valid for the ndims).

I think we need to fix the CSC (and CSR) constructors to give a reasonable message for nD dense input. And we need docs for tensordot.

Thanks @jorenham ! It is really helpful for someone to work through each feature of each new function. And annotations require that. But we really should have caught the missing doc_string. :{ Anyway – Thanks very much!

Only COO format supports ndim>2. And tensordot should definitely have documentation. And the axis handling should either be better or raise an exception that says it is not fully developed yet. I’ll take a look.

I just reread the top post and noticed the csc_array(np.zeros((2, 3, 4))) line, somehow I missed that - apologies for the noise!

Thanks for digging into this!

  • CSC should not support ndim > 2, so that’s a bug to be fixed.
  • tensordot definitely needs documentation, and better axis handling.

@dschult do you have time for a quick fix PR?

Yep, tensordot just looks like a lack of documentation. For the other issues, is it just a case of being more explicit in error messages / the release notes about the (limited) extent of the >2D support so far?

Apart from the lack of documentation, the last error here looks kinda buggy to me:

>>> a.tensordot(a, axes=1)
<COOrdinate sparse array of dtype 'float64'
	with 3 stored elements and shape (3, 3)>
>>> a.tensordot(a, axes=2)
np.float64(3.0)
>>> a.tensordot(a, axes=(1,))
Traceback (most recent call last):
  File "<python-input-25>", line 1, in <module>
    a.tensordot(a, axes=(1,))
    ~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/home/joren/.pyenv/versions/3.13.1/lib/python3.13/site-packages/scipy/sparse/_coo.py", line 958, in tensordot
    axes_self, axes_other = _process_axes(self.ndim, other.ndim, axes)
                            ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/joren/.pyenv/versions/3.13.1/lib/python3.13/site-packages/scipy/sparse/_coo.py", line 1206, in _process_axes
    raise ValueError("axes must be a tuple/list of length 2")
ValueError: axes must be a tuple/list of length 2
>>> a.tensordot(a, axes=(1,2))
Traceback (most recent call last):
  File "<python-input-26>", line 1, in <module>
    a.tensordot(a, axes=(1,2))
    ~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/home/joren/.pyenv/versions/3.13.1/lib/python3.13/site-packages/scipy/sparse/_coo.py", line 958, in tensordot
    axes_self, axes_other = _process_axes(self.ndim, other.ndim, axes)
                            ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/joren/.pyenv/versions/3.13.1/lib/python3.13/site-packages/scipy/sparse/_coo.py", line 1208, in _process_axes
    if len(axes_a) != len(axes_b):
       ~~~^^^^^^^^
TypeError: object of type 'int' has no len()