You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "JerAguilon (via GitHub)" <gi...@apache.org> on 2024/02/16 08:20:49 UTC

[I] [Pyarrow] Support non-scalar filtering [arrow]

JerAguilon opened a new issue, #40099:
URL: https://github.com/apache/arrow/issues/40099

   ### Describe the enhancement requested
   
   It would be fantastic to be able to run expressions on a dataset with non-scalar expression. As a dummy example, this would return all the rows in which column "foo" is bigger than the last value:
   
   ```
   import pyarrow.compute as pc
   import pyarrow as pa
   
   table = pa.Table.from_arrays([pa.array([1, 5, 3, 4])], names=["foo"])
   expr = pc.field('foo') >= pc.last(pc.field('foo'))
   
   # expected:  pa.Table.from_arrays([pa.array([5, 4])], names=["foo"])
   ```
   
   Today, you'd get something like:
   
   ```
   ArrowInvalid: ExecuteScalarExpression cannot Execute non-scalar expression (foo == last(foo))
   ```
   
   Is there a reason we can only execute scalar expressions? Is there a way today to interweave aggregations in one filter query?
   
   ### Component(s)
   
   Python


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Pyarrow] Support scalar aggregate filtering [arrow]

Posted by "JerAguilon (via GitHub)" <gi...@apache.org>.
JerAguilon commented on issue #40099:
URL: https://github.com/apache/arrow/issues/40099#issuecomment-1949128116

   I realize this is actually a CPP request for `ExecuteScalarExpression`. Changing tags and making a PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Pyarrow] Support scalar aggregate filtering [arrow]

Posted by "JerAguilon (via GitHub)" <gi...@apache.org>.
JerAguilon closed issue #40099: [Pyarrow] Support scalar aggregate filtering 
URL: https://github.com/apache/arrow/issues/40099


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org