[GitHub] [arrow] westonpace commented on issue #12568: Support for automatic optimizations

[GitBox <gi...@apache.org>]

westonpace commented on issue #12568:
URL: https://github.com/apache/arrow/issues/12568#issuecomment-1059426493


   This is not an answer exactly but I wanted to mention that there are a number of PRs & JIRAs in Arrow to actively adopt [Substrait](https://substrait.io/) which should open the door for users to plug in their own optimizer.  For example, if rarrow's dplyr bindings produced Substrait plans instead of arrow plans then you could do:
   
   ```
   rarrow-dplyr -> substrait -> any optimizer you want (e.g. duckdb's optimizer, calcite, etc.) -> substrait -> arrow
   ```
   
   Though that is perhaps a bit of an oversimplification as optimizers often would want information back from arrow as well (e.g. table statistics, cost estimates, etc.) 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org