scikit-learn: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices

It’s not a regression, and not a blocker for 0.20.1 (please feel free to untag it), but it still fairly annoying as it means we get PendingDeprecationWarning when using the latest scipy and it’s better to fix that as soon as possible.

In my case, within my code, changing from “.todense()” to “.toarray()” for sparse matrices removed the warnings. As well as “.sum()” over the some axis on sparse matrix are converted to np.matrix. Here may lie the specific problem. This last one forced me to convert sparse matrices toarray or only by adding use a redefinition with the following function (not fully implemented):

import numpy as np
def sum_rows(A):
    """ Sumation along the rows (add all elements of the columns for each row independently).
        Similar to A.sum(axis=1) but return csr_matrix. Used to avoid pytest deprecation warning.
        A must be csr_matrix to be 'efficient'.
    """
    # about A
    NRows_A, NCols_A = A.shape
    indxrow_A, indxcol_A = A.nonzero()
    # about Aux
    NCols_aux = 1
    NRows_aux = NCols_A
    # drop duplicates of indxcol_A to set ones on Aux (to not multiply > 1)
    row_indx_aux = np.unique(indxcol_A)
    col_indx_aux = [0] * len(row_indx_aux)
    data = [1] * len(indxcol_A)
    Aux = sps__csr_matrix( (data, (row_indx_aux, col_indx_aux)), (NRows_aux, NCols_aux) )
    return A * Aux

If you avoid them, you avoid the warnings with pytest. Hope my toc helps someone.

This was fixed for scikit-learn in https://github.com/scikit-learn/scikit-learn/pull/13076 however keeping this issue open as this is still an issue for all projects that depend on scikit-learn (when running their test suites), unless they manually filter out the warning. I’m hoping someone comes up with some solution to avoid these manual changes.

In the project where I face the problem, I use pytest and I’ve added:

filterwarnings=
    ignore:the matrix subclass is not the recommended way*:PendingDeprecationWarning

under the [pytest] section of pytest.ini.

As mentioned in https://github.com/scipy/scipy/issues/9734#issuecomment-459528915 this is sub-optimal but at least now the tests are green 😃

Opened https://github.com/scipy/scipy/issues/9734

Yes, the next scipy release is expected in a month or so https://mail.python.org/pipermail/scipy-dev/2018-October/023132.html so it shouldn’t be too long.

There is no need to filter warnings triggered by scipy itself since those will already be filtered by the next scipy version: scipy/scipy#8887.

Just to remove the warnings, you can just squash with warnings module

That would not remove the warnings per se, that would just filter them out and we don’t want that.

Like @rth said, when some mathematical computation is being done on a sparse matrix, we need to convert it first to an array in order to avoid the warning.

To be fair the ‘easy’ tag is a bit misleading because when I looked at it, it wasn’t obvious at all where to make the matrix-to-array conversions. The conversions can be done in various places and it takes a good knowledge about the repo to identify where exactly it makes sense to convert an object, and where it makes sense to keep its original type.

Ok, I’ll pick it up unless @grassknoted updates soon.

Yes, thank you @grassknoted