You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by GitBox <gi...@apache.org> on 2022/12/19 13:29:11 UTC
[GitHub] [arrow] JeromeLovy opened a new issue, #15027: Documented examples for pyarrow "filtering by expression" fail with TypeError
JeromeLovy opened a new issue, #15027:
URL: https://github.com/apache/arrow/issues/15027
### Describe the usage question you have. Please include as many useful details as possible.
Hello,
I'm trying to perform "filtering by expressions" with pyarrow, by testing the code samples found in the [Compute Functions](https://arrow.apache.org/docs/python/compute.html#filtering-by-expressions) documentation or in the [pyarrow.Table.filter](https://arrow.apache.org/docs/python/generated/pyarrow.Table.html#pyarrow.Table.filter) reference.
Example 1:
```
import pyarrow as pa
import pyarrow.compute as pc
even_filter = (pc.bit_wise_and(pc.field("nums"), pc.scalar(1)) == pc.scalar(0))
table = pa.table({'nums': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
'chars': ["a", "b", "c", "d", "e", "f", "g", "h", "i", "l"]})
table.filter(even_filter)
```
Example 2:
```
import pyarrow as pa
import pandas as pd
df = pd.DataFrame({'year': [2020, 2022, 2019, 2021],
'n_legs': [2, 4, 5, 100],
'animals': ["Flamingo", "Horse", "Brittle stars", "Centipede"]})
table = pa.Table.from_pandas(df)
table.filter(expr)
```
Both examples fail with following exception, apparently raised in `_pack_compute_args()` in `pyarrow/_compute.pyx`:
```
TypeError: Got unexpected argument type <class 'pyarrow._compute.Expression'> for compute function
```
Am I doing something wrong?
Is the documented behavior (Filtering by Expressions) really implemented?
(In `pyarrow/_compute.pyx`, the Cython code for `_pack_compute_args()` doesn't seem to allow an Expression as argument indeed.)
Thanks in advance.
Best,
Jérôme
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on issue #15027: Documented examples for pyarrow "filtering by expression" fail with TypeError
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on issue #15027:
URL: https://github.com/apache/arrow/issues/15027#issuecomment-1357808893
Which version of pyarrow are you using? The docs you mention are for the latest released version 10.0, and I think it is new in that version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] JeromeLovy commented on issue #15027: Documented examples for pyarrow "filtering by expression" fail with TypeError
Posted by GitBox <gi...@apache.org>.
JeromeLovy commented on issue #15027:
URL: https://github.com/apache/arrow/issues/15027#issuecomment-1357909048
Good catch! My bad! 😅 Thanks for the quick hint!
Examples run smoothly with pyarrow v10 indeed. It seems that "filtering by expressions" has been introduced with v9.
As of this writing, my conda tends to install v8.0.0 (when not explicitly requesting `pyarrow==10.0.1`). Hence my confusion: default version of pyarrow installed with my conda environment wasn't matching the documentation I was reading online. 😅
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] JeromeLovy closed issue #15027: Documented examples for pyarrow "filtering by expression" fail with TypeError
Posted by GitBox <gi...@apache.org>.
JeromeLovy closed issue #15027: Documented examples for pyarrow "filtering by expression" fail with TypeError
URL: https://github.com/apache/arrow/issues/15027
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org