pandas: BUG: Pyarrow dependency warning starts with newline which makes it impossible to filter out by message with -W or PYTHONWARNINGS
Pandas version checks
-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
I have confirmed this bug exists on the main branch of pandas.
Issue Description
I would expect that the Pyarrow warning is easily ignorable by message i.e. something like the following does not produce any warning:
python -W 'ignore:Pyarrow:DeprecationWarning' -c 'import pandas'
The issue is that because the warning starts with a newline it can not be targetted by message. This seems to be the case of any warning that starts with some kind of whitespace character.
I tried a variety of things like -W 'ignore:\nPyarrow:DeprecationWarning'
but I could not make it work. I think the reason is the “ignoring any whitespace at the start or end of message” from the warnings documentation:
In -W and PYTHONWARNINGS, message is a literal string that the start of the warning message must contain (case-insensitively), ignoring any whitespace at the start or end of message.
In scikit-learn tests warnings are turned into errors with pytest -W
and we would like to use -W
too to ignore the Pyarrow dependency warning for now. A possible work-around is to use filterwarnings which accept a regex, see Note in https://docs.pytest.org/en/stable/how-to/capture-warnings.html#controlling-warnings for more details, but being able to use -W
or PYTHONWARNINGS
seems desirable e.g. to control warnings from the command-line outside of pytest.
Expected Behavior
There is an easy way with -W
or PYTHONWARNINGS
to ignore the Pyarrow DeprecationWarning e.g.
python -W 'ignore:Pyarrow:DeprecationWarning' -c 'import pandas'
Installed Versions
pandas : 2.2.0 numpy : 1.26.3 pytz : 2023.3 dateutil : 2.8.2 setuptools : 68.0.0 pip : 23.2 Cython : 0.29.35 pytest : 7.4.0 hypothesis : None sphinx : 6.0.0 blosc : None feather : None xlsxwriter : None lxml.etree : None html5lib : None pymysql : None psycopg2 : None jinja2 : 3.1.2 IPython : 8.18.0 pandas_datareader : None adbc-driver-postgresql: None adbc-driver-sqlite : None bs4 : None bottleneck : None dataframe-api-compat : None fastparquet : None fsspec : None gcsfs : None matplotlib : 3.8.0 numba : None numexpr : None odfpy : None openpyxl : None pandas_gbq : None pyarrow : None pyreadstat : None python-calamine : None pyxlsb : None s3fs : None scipy : 1.12.0 sqlalchemy : None tables : None tabulate : None xarray : None xlrd : None zstandard : 0.21.0 tzdata : 2023.3 qtpy : None pyqt5 : None
About this issue
- Original URL
- State: closed
- Created 5 months ago
- Comments: 16 (11 by maintainers)
Preemptively +100 to remove the deprecation warning in 2.2.1
@mimizone you probably know this already but you don’t have to use the full message, it only needs to match at the beginning. Since “https://” is towards the end of the message, I don’t think it matters.
For example if there was not the newline at the start you could ignore the message like this:
or