scipy: BUG: sparse: `np.dot` gives element-wise multiplication for sparse arrays
Describe your issue.
This was reported over in networkx/networkx#6792.
The docstring for np.dot states:
dot(a, b, out=None)
Dot product of two arrays. Specifically,
…
- If both
aandbare 2-D arrays, it is matrix multiplication, but using :func:matmulora @ bis preferred.
So users would expect np.dot(A, A) where A is a sparse array to be equivalent to A @ A. However, np.dot(A, A) is instead equivalent to A * A, i.e. element-wise multiplication.
I suspect this is a holdover from sparse matrices, where the * operator meant matrix multiplication.
Reproducing Code Example
>>> import scipy as sp
>>> A = sp.sparse.csr_array([[0, 0, 1], [4, 0, 6], [7, 0, 0]])
>>> (A @ A).toarray() # matmul, as expected
array([[ 7, 0, 0],
[42, 0, 4],
[ 0, 0, 7]])
>>> (A * A).toarray() # element-wise
array([[ 0, 0, 1],
[16, 0, 36],
[49, 0, 0]])
>>> np.dot(A, A).toarray() # Expect matmul, get el-wise
array([[ 0, 0, 1],
[16, 0, 36],
[49, 0, 0]])
# np.dot gives matmul with sparse matrices as expected
>>> M = sp.sparse.csr_matrix(A)
>>> np.dot(M, M).toarray()
array([[ 7, 0, 0],
[42, 0, 4],
[ 0, 0, 7]])
Error message
N/A
SciPy/NumPy/Python version and system information
1.11.1 1.25.0 sys.version_info(major=3, minor=11, micro=3, releaselevel='final', serial=0)
Build Dependencies:
blas:
detection method: pkgconfig
found: true
include directory: /usr/local/include
lib directory: /usr/local/lib
name: openblas
openblas configuration: USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= HASWELL MAX_THREADS=2
pc file directory: /usr/local/lib/pkgconfig
version: 0.3.21.dev
lapack:
detection method: pkgconfig
found: true
include directory: /usr/local/include
lib directory: /usr/local/lib
name: openblas
openblas configuration: USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= HASWELL MAX_THREADS=2
pc file directory: /usr/local/lib/pkgconfig
version: 0.3.21.dev
pybind11:
detection method: config-tool
include directory: unknown
name: pybind11
version: 2.10.4
Compilers:
c:
commands: cc
linker: ld.bfd
name: gcc
version: 10.2.1
c++:
commands: c++
linker: ld.bfd
name: gcc
version: 10.2.1
cython:
commands: cython
linker: cython
name: cython
version: 0.29.35
fortran:
commands: gfortran
linker: ld.bfd
name: gcc
version: 10.2.1
pythran:
include directory: ../../tmp/pip-build-env-lnljzw4o/overlay/lib/python3.11/site-packages/pythran
version: 0.13.1
Machine Information:
build:
cpu: x86_64
endian: little
family: x86_64
system: linux
cross-compiled: false
host:
cpu: x86_64
endian: little
family: x86_64
system: linux
Python Information:
path: /opt/python/cp311-cp311/bin/python
version: '3.11'
About this issue
- Original URL
- State: open
- Created 3 years ago
- Comments: 15 (12 by maintainers)
I think this is expected, unfortunately.
np.dotassumes the inputs are numpy arrays, and scipy sparse matrices are not numpy arrays. So when you callnp.dot(coo_array, coo_array), numpy treats it as a dot product between two scalars object arrays of typecoo_array. A dot product between two scalars is the same as multiplying those scalars, so this effectively resolves tocoo_array * coo_array.If you want the matrix product between two sparse arrays, you should not use numpy functions, but rather the matrix product operator, i.e.
A @ B.It’s unfortunate that things work this way, but I’m not sure it’s fixable without changing some of the design decisions NumPy made early in the life of the project.
I think the first blocker is that we need to find/replace the places where scipy.sparse depends on numpy treating it as a python object. Once that is cleared up we can turn to getting it to work with the dunder methods.
Though it is possible I am missing the forest due to all the trees. 😃
I do still intend to look into this more – but there are many things in front of it in my queue at the moment. In general the
np.*functions do not take sparse as inputs.You should use
sparseA.dot(B)instead ofnp.dot(sparseA, B).So, instead of writing your own
np.dotoverload, use the sparse functions/methods.It sounds like you would be welcome to look into following the approach of https://github.com/scipy/scipy/issues/18938#issuecomment-2513540357 @JerryChen97 ! Unless @dschult gave this a try after writing that?
Sounds like the right approach - I don’t think there’s a better way than that.