scikit-learn: Array API tests fail on main

Describe the bug

I ran the Array API tests on main and got 10 failing tests. (Last week, with an older main and everything else the same, I had 4 failing tests.)

array_api_compat==1.7.1

I only ran the cpu tests.

Steps/Code to Reproduce

pytest sklearn/utils/tests/test_array_api.py

Expected Results

all tests pass

Actual Results

FAILED sklearn/utils/tests/test_array_api.py::test_get_namespace_ndarray_with_dispatch - AssertionError
FAILED sklearn/utils/tests/test_array_api.py::test_average[None-None-False-21-numpy-None-None] - AssertionError
FAILED sklearn/utils/tests/test_array_api.py::test_average_raises_with_invalid_parameters[0-weights1-TypeError-1D weights expected-numpy-None-None] - ValueError: Shape of weights must be consistent with shape of a along specified axis.
FAILED sklearn/utils/tests/test_array_api.py::test_average_raises_with_invalid_parameters[0-weights2-ValueError-Length of weights-numpy-None-None] - AssertionError: Regex pattern did not match.
FAILED sklearn/utils/tests/test_array_api.py::test_count_nonzero[None-None-csr_matrix-numpy-None-None] - AssertionError
FAILED sklearn/utils/tests/test_array_api.py::test_count_nonzero[None-None-csr_array-numpy-None-None] - AssertionError
FAILED sklearn/utils/tests/test_array_api.py::test_count_nonzero[int-None-csr_matrix-numpy-None-None] - AssertionError
FAILED sklearn/utils/tests/test_array_api.py::test_count_nonzero[int-None-csr_array-numpy-None-None] - AssertionError
FAILED sklearn/utils/tests/test_array_api.py::test_count_nonzero[float-None-csr_matrix-numpy-None-None] - AssertionError
FAILED sklearn/utils/tests/test_array_api.py::test_count_nonzero[float-None-csr_array-numpy-None-None] - AssertionError

Versions

System:
    python: 3.12.2 (main, Apr 18 2024, 11:14:27) [GCC 13.2.1 20230801]
executable: /home/stefanie/.pyenv/versions/3.12.2/envs/scikit-learn_dev/bin/python
   machine: Linux-6.9.5-arch1-1-x86_64-with-glibc2.39

Python dependencies:
      sklearn: 1.6.dev0
          pip: 24.0
   setuptools: 69.5.1
        numpy: 2.1.0.dev0
        scipy: 1.13.0
       Cython: 3.0.10
       pandas: 2.2.2
   matplotlib: 3.8.4
       joblib: 1.4.0
threadpoolctl: 3.4.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 14
         prefix: libopenblas
       filepath: /home/stefanie/.pyenv/versions/3.12.2/envs/scikit-learn_dev/lib/python3.12/site-packages/scipy.libs/libopenblasp-r0-24bff013.3.26.dev.so
        version: 0.3.26.dev
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 14
         prefix: libgomp
       filepath: /usr/lib/libgomp.so.1.0.0
        version: None

Also just tested with numpy==2.0.0, and the same failures.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 22 (22 by maintainers)

Most upvoted comments

About lock-files, I think we should merge https://github.com/scikit-learn/scikit-learn/pull/29388 first.

Merged. I will update my PR with main.

Was there was a good reason to have pytorch=1.13? If not, then I would try first to remove the PyTorch pin …

Running locally pytest sklearn/utils/tests/test_array_api.py with pytorch-cpu 2.3 and numpy 2.0, it does not seem to create more failures than in the top message.

Waiting for reactions on the above issue before trying to fix things. We could remove the check that the thing returned from xp.sum is on the same device as the input. But it isn’t clear to me if what we see is a bug in Numpy or us being too strict in our check.

For completeness, one of the potential reason this was not noticed before is that lock-file in the CI for the array API tests do use 1.26.4 (I think because we require pytorch=1.13 that wants numpy<2):

❯ rg numpy build_tools/azure/pylatest_conda_forge_mkl_linux-64_conda.lock 
209:https://conda.anaconda.org/conda-forge/linux-64/numpy-1.26.4-py311h64a7726_0.conda#a502d7aad449a1206efb366d6a12c52d

@lesteve Might be because of some updates in the new version of numpy. Will have to check this.