pandas: DataFrame.eval errors with AttributeError: 'UnaryOp'
Code Sample, a copy-pastable example if possible
def test_unary():
df = pd.DataFrame({'x': np.array([0.11, 0], dtype=np.float32)})
res = df.eval('(x > 0.1) | (x < -0.1)')
assert np.array_equal(res, np.array([True, False])), res
Problem description
This is related to #11235. on python 3.6, pandas 20.1, this raises an error the traceback ends with:
File ".../envs/py3/lib/python3.6/site-packages/pandas/core/computation/expr.py", line 370, in _maybe_downcast_constants
name = self.env.add_tmp(np.float32(right.value))
AttributeError: 'UnaryOp' object has no attribute 'value'
In that case the right is -(0.1)
INSTALLED VERSIONS
------------------
commit: None
python: 3.6.1.final.0
python-bits: 64
OS: Linux
OS-release: 4.8.0-49-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
pandas: 0.20.1 pytest: None pip: 9.0.1 setuptools: 27.2.0 Cython: 0.25.2 numpy: 1.12.1 scipy: 0.19.0 xarray: None IPython: 6.0.0 sphinx: None patsy: None dateutil: 2.6.0 pytz: 2017.2 blosc: None bottleneck: None tables: None numexpr: None feather: None matplotlib: 2.0.2 openpyxl: None xlrd: None xlwt: None xlsxwriter: None lxml: None bs4: None html5lib: None sqlalchemy: None pymysql: None psycopg2: None jinja2: None s3fs: None pandas_gbq: None pandas_datareader: None
Another example:
>>> df = pd.DataFrame({'x':[1,2,3,4,5]})
>>> df.eval('x.shift(-1)')
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 4
- Comments: 23 (6 by maintainers)
Commits related to this issue
- Fix #16363: Prevent visit_BinOp from accessing value on UnaryOp — committed to alexcwatt/pandas by alexcwatt 5 years ago
- BUG: Fix #16363 - Prevent visit_BinOp from accessing value on UnaryOp — committed to alexcwatt/pandas by alexcwatt 5 years ago
- BUG: Fix #16363 - Prevent visit_BinOp from accessing value on UnaryOp (#25928) — committed to pandas-dev/pandas by alexcwatt 5 years ago
- BUG: Fix #16363 - Prevent visit_BinOp from accessing value on UnaryOp (#25928) — committed to yhaque1213/pandas by alexcwatt 5 years ago
- Fixing Unary Op evaluate issue Summary: Without this diff, the critical_path_analysis on some traces fails with the error: `'UnaryOp' object has no attribute 'evaluate'` This is apparently a pandas ... — committed to amoghavs/HolisticTraceAnalysis by deleted user 4 months ago
- Fixing Unary Op evaluate issue (#112) Summary: Without this diff, the critical_path_analysis on some traces fails with the error: `'UnaryOp' object has no attribute 'evaluate'` This is apparently a... — committed to amoghavs/HolisticTraceAnalysis by deleted user 4 months ago
- Fixing Unary Op evaluate issue (#112) Summary: Without this diff, the critical_path_analysis on some traces fails with the error: `'UnaryOp' object has no attribute 'evaluate'` This is apparently a... — committed to amoghavs/HolisticTraceAnalysis by deleted user 4 months ago
- Fixing Unary Op evaluate issue (#112) Summary: Pull Request resolved: https://github.com/facebookresearch/HolisticTraceAnalysis/pull/112 Without this diff, the critical_path_analysis on some traces ... — committed to facebookresearch/HolisticTraceAnalysis by deleted user 4 months ago
I ran into this recently and would like to help with a patch. As best I can tell, the problem is that
_maybe_downcast_constantsnot only tries to downcast constants but alsoUnaryOp’s, which isn’t possible, sinceUnaryOpinstances don’t have avalueattribute like constants/scalars do.I am new to the pandas code, and the expressions code is a bit tricky, but I think we could catch the
AttributeErrorin_maybe_downcast_constantsor explicitly check in each case thatleftorrighthas the attributevalue.In short, the problem is that an operation like
df.eval(x < -.1)fails whenxis anp.float32because the right side of the equation is seen as aUnaryOpnode instead of as anp.float32and is subjected to_maybe_downcast_constantsbyvisit_BinOp. OTOH,df.eval(x < @y)works wheny = -.1, because pandas doesn’t have to parse it. I think a small change might fix this, but I could be overlooking something bigger and would appreciate feedback.Hi, I am wondering if this is resolved? I’m running into a similar issue using pandas df.query() with negative numbers. Thank you!
Not really a fix. But if you need a workaround just use float64. Worked for me.