deepsensor: Potential bug: Incompatibility with NumPy 2.0 leads to AttributeError
Description
The recent upgrade to NumPy 2.0.0 introduces a breaking change due to the removal of np.sctypes. This update has resulted in deepsensor raising an AttributeError upon import when paired with the new version of NumPy. This indicates that the current deepsensor codebase is not compatible with NumPy 2.0.0’s updated API, causing it to break. Indeed, the documentation for NumPy 2.0.0 indicates that np.sctypes has indeed been removed from the namespace:
https://numpy.org/devdocs/numpy_2_0_migration_guide.html#changes-to-namespaces
Here is the full traceback received after trying to import deepsensor.torch:
Traceback (most recent call last):
File "~/DeepSensor/NumPy-issue/deepsensor_numpy_test.py", line 1, in <module>
import deepsensor.torch
File "~/deepsensor_numpy_test/lib/python3.10/site-packages/deepsensor/__init__.py", line 20, in <module>
from .data.processor import DataProcessor
File "~/deepsensor_numpy_test/lib/python3.10/site-packages/deepsensor/data/__init__.py", line 2, in <module>
from .loader import TaskLoader
File "~/deepsensor_numpy_test/lib/python3.10/site-packages/deepsensor/data/loader.py", line 1, in <module>
from deepsensor.data.task import Task, flatten_X
File "~/deepsensor_numpy_test/lib/python3.10/site-packages/deepsensor/data/task.py", line 5, in <module>
import lab as B
File "~/deepsensor_numpy_test/lib/python3.10/site-packages/lab/__init__.py", line 9, in <module>
from .generic import *
File "~/deepsensor_numpy_test/lib/python3.10/site-packages/lab/generic.py", line 11, in <module>
from .types import (
File "~/deepsensor_numpy_test/lib/python3.10/site-packages/lab/types.py", line 99, in <module>
Int = Union[tuple([int, Dimension] + np.sctypes["int"] + np.sctypes["uint"])]
File "~/deepsensor_numpy_test/lib/python3.10/site-packages/numpy/__init__.py", line 397, in __getattr__
raise AttributeError(
AttributeError: `np.sctypes` was removed in the NumPy 2.0 release. Access dtypes explicitly instead.. Did you mean: 'dtypes'?
Sorry to be the bearer of bad news @wesselb, but the error seems to come from the lab package when it tries to access a numpy function. This breaking change, which has also broken lab, could be reported as a separate issue here:
https://github.com/wesselb/lab/issues
This error will affect users who pip install deepsensor using the repository in its current form. One straightforward solution would be pinning the NumPy version in the requirements/requirements*.txt files to the most recent compatible version (i.e. numpy<=1.26.4 or numpy<2).
Reproduction steps
1. Create a new virtual environment for testing (e.g. `deepsensor_numpy_test`, as seen above)
2. pip install `deepsensor` 0.3.6 and its dependencies (including NumPy 2.0.0)
3. Try to import deepsensor code (e.g. `import deepsensor.torch`)
4. Get the Traceback shown above
Version
I am using python 3.10.9. Here is a truncated package list from pip list for completeness:
dask 2024.6.2
deepsensor 0.3.6
matplotlib 3.9.0
netCDF4 1.7.1
NeuralProcesses 0.2.6
numpy 2.0.0
pandas 2.2.2
pip 24.1
scikit-learn 1.5.0
scipy 1.13.1
seaborn 0.13.2
torch 2.3.0
xarray 2024.6.0
The breaking change appears to affect the lab package, which is not pip installed. I found the version number in my virtual environment here: ~/deepsensor_numpy_test/lib/python3.10/site-packages/lab/_version.py, indicating that I’m using the latest version of lab==1.6.5.
OS
Linux
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 17 (1 by maintainers)
I can confirm that the problem seems to be fixed!
My test:
deepsensor-test-env)pip install deepsensor torchimport deepsensor.torchAnd the code successfully imports! That’s the extent of my testing at the moment. There is a warning that is raised, but it looks unrelated:
@wesselb and @davidwilby, thanks so much for working on this! I’m glad to see this working again. : )
I’ve opened a PR on
fdmto hopefully go towards fixing this issue.@DaniJonesOcean, I’ve digged into this, and I think
backends==1.6.6should be compatible with NumPy 2.0. There is one incompability in a dependencyfdm. You can locally hotfix this as follows:After this, the following snippet should run correctly:
I will release a fix for
fdmsoon. For the moment, you can use the above hotfix.@DaniJonesOcean, that’s great to hear! Thanks again, @davidwilby. 😃 Since the problem seems to be fixed, I think the issue can be closed, so I’ll go ahead and do that. 😃
@DaniJonesOcean Could you try reinstalling in a fresh environment? This looks like something has gone wrong. Line 99, which gives the error, isn’t what it should be (I double checked the package on PyPI).
Thank you! 😃 Once a little bit of time frees up for me, I should be able to start digging into this too.