You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "larry98 (via GitHub)" <gi...@apache.org> on 2024/07/08 18:36:48 UTC
[I] Support SimplifyWithGuarantee for is_in expressions [arrow]
larry98 opened a new issue, #43187:
URL: https://github.com/apache/arrow/issues/43187
### Describe the enhancement requested
We'd like to use parquet predicate pushdown on `is_in` expressions, but this currently isn't supported in `SimplifyWithGuarantee`. We implemented a proof of concept where we sort and deduplicate the `is_in` expression's value set, then have `SimplifyWithGuarantee` binary search on the inequality bound and slice the value set accordingly. This works well, but I'm not sure what the correct interface for enabling this code path should be. Our current approach adds a new field to `SetLookupOptions` which allows the user to declare whether the value set is pre-sorted and deduplicated.
Any thoughts? I'd be happy to put up a PR if we agree on an interface.
### Component(s)
C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Support SimplifyWithGuarantee for is_in expressions [arrow]
Posted by "larry98 (via GitHub)" <gi...@apache.org>.
larry98 commented on issue #43187:
URL: https://github.com/apache/arrow/issues/43187#issuecomment-2228838350
@mapleFU I opened a PR (https://github.com/apache/arrow/pull/43256) with the change that I was thinking of. I don't fully follow the point about `KnownFieldValues` - I think I may be proposing something simpler.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Support SimplifyWithGuarantee for is_in expressions [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on issue #43187:
URL: https://github.com/apache/arrow/issues/43187#issuecomment-2341809216
Issue resolved by pull request 43761
https://github.com/apache/arrow/pull/43761
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Support SimplifyWithGuarantee for is_in expressions [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz closed issue #43187: [C++] Support SimplifyWithGuarantee for is_in expressions
URL: https://github.com/apache/arrow/issues/43187
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++] Support SimplifyWithGuarantee for is_in expressions [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #43187:
URL: https://github.com/apache/arrow/issues/43187#issuecomment-2223410446
@larry98 feel free to do this and @ me if you finish
This requires some enhancement on `KnownFieldValues` in this function. Besides the syntax of "in" in SQL with null would be a bit hacking, which should be take care of
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org