pandas: ValueError: cannot set WRITEABLE flag to True of this array

When using numpy=1.16.0 I get this error, when I downgrade numpy=1.15.4 problem is gone

Upgrading to pytables 3.5.1 fixes the problem for me also with numpy 1.16.2

For me pip install numpy==1.15.4 also resolves this issue.

@TomAugspurger this issue comes up on Google on top (at least for me it did) and it seems that the remedy is simple, so maybe it’s enough to just wait for an upstream fix. Really great that you would ask 😃

pip3 install numpy==1.15.4 also solved for me…

However, when I downgraded numpy i kept getting this error ImportError: No module named 'numpy.core._multiarray_umath'. Finally figured out it was happening because I stored the .h5 file with numpy 1.16 installed and it wouldn’t reopen with the downgraded numpy…

@gfyoung Sure here is the Versions:

'dependencies': 
{'pandas': '0.23.4', 'pytest': '3.4.0', 'pip': '18.1', 'setuptools': '40.6.3', 'Cython': '0.29.3', 'numpy': '1.16.0', 'scipy': '1.2.0', 'pyarrow': None, 'xarray': None, 'IPython': '6.5.0', 'sphinx': None, 'patsy': '0.5.0', 'dateutil': '2.7.5', 'pytz': '2018.7', 'blosc': None, 'bottleneck': '1.2.1', 'tables': '3.4.4', 'numexpr': '2.6.8', 'feather': None, 'matplotlib': '3.0.2', 'openpyxl': '2.5.12', 'xlrd': '1.1.0', 'xlwt': '1.3.0', 'xlsxwriter': '0.7.3', 'lxml': '4.1.1', 'bs4': '4.4.1', 'html5lib': '1.0b8', 'sqlalchemy': '1.2.15', 'pymysql': '0.9.2', 'psycopg2': '2.7.6.1 (dt dec pq3 ext lo64)', 'jinja2': '2.10', 's3fs': None, 'fastparquet': None, 'pandas_gbq': None, 'pandas_datareader': '0.7.0'}

unfortunately i can not share the file but i think the issues indeed comes from numpy as @vvvlc said here is an other issue on their git:

https://github.com/nipy/nibabel/issues/697

PS: just downgraded to numpy=1.15.4 and indeed it resolves the issue

Is there anything pandas can do in the meantime? Or just wait for the next pytables release?

On Mon, Jan 21, 2019 at 5:34 AM macd2 notifications@github.com wrote:

@gfyoung https://github.com/gfyoung Sure here is the Versions:

‘dependencies’: {‘pandas’: ‘0.23.4’, ‘pytest’: ‘3.4.0’, ‘pip’: ‘18.1’, ‘setuptools’: ‘40.6.3’, ‘Cython’: ‘0.29.3’, ‘numpy’: ‘1.16.0’, ‘scipy’: ‘1.2.0’, ‘pyarrow’: None, ‘xarray’: None, ‘IPython’: ‘6.5.0’, ‘sphinx’: None, ‘patsy’: ‘0.5.0’, ‘dateutil’: ‘2.7.5’, ‘pytz’: ‘2018.7’, ‘blosc’: None, ‘bottleneck’: ‘1.2.1’, ‘tables’: ‘3.4.4’, ‘numexpr’: ‘2.6.8’, ‘feather’: None, ‘matplotlib’: ‘3.0.2’, ‘openpyxl’: ‘2.5.12’, ‘xlrd’: ‘1.1.0’, ‘xlwt’: ‘1.3.0’, ‘xlsxwriter’: ‘0.7.3’, ‘lxml’: ‘4.1.1’, ‘bs4’: ‘4.4.1’, ‘html5lib’: ‘1.0b8’, ‘sqlalchemy’: ‘1.2.15’, ‘pymysql’: ‘0.9.2’, ‘psycopg2’: ‘2.7.6.1 (dt dec pq3 ext lo64)’, ‘jinja2’: ‘2.10’, ‘s3fs’: None, ‘fastparquet’: None, ‘pandas_gbq’: None, ‘pandas_datareader’: ‘0.7.0’}

unfortunately i can not share the file but i think the issues indeed comes from numpy as @vvvlc https://github.com/vvvlc said here is an other issue on their git:

nipy/nibabel#697 https://github.com/nipy/nibabel/issues/697

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pandas-dev/pandas/issues/24839#issuecomment-456043488, or mute the thread https://github.com/notifications/unsubscribe-auth/ABQHIqiIyuE-lmMlN2Ep-4htyMrTBdefks5vFaWqgaJpZM4aJQqs .

What versions of pytables and numpy reproduce this? Is it specific to the data?

with

pandas: 0.24.1
numpy: 1.16.2
tables: 3.4.4

this doesn’t raise,

In [8]: df = pd.DataFrame({"A": [1, 2]})

In [9]: df.to_hdf('x.hdf5', key='x')

In [10]: pd.read_hdf('x.hdf5', 'x', mode='r')
Out[10]:
   A
0  1
1  2

FYI, pytables 3.5.0 and 3.5.1 are on PyPI with the fix from the pytables side.

@dev72 ok but how about old HDF files that already exist?

you can avoid the error: ValueError: cannot set WRITEABLE flag to True of this array passing format='table' to HDFStore.append or HDFStore.put when you save data with pandas.

This will likely solve your problem, tested with pandas 0.24 and numpy 1.16+