You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/10/21 16:45:15 UTC
[GitHub] [arrow-datafusion] alamb opened a new issue #1162: Add additional algebraic simplifications
alamb opened a new issue #1162:
URL: https://github.com/apache/arrow-datafusion/issues/1162
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Now that we have a nice ConstantPropagator and Simplifier framework, there are additional predicate rewrites we can do to improve query performance
**Describe the solution you'd like**
Implement additional algebraic simplification / rewrite rules to [constant_folding.rs](https://github.com/apache/arrow-datafusion/blob/master/datafusion/src/optimizer/constant_folding.rs) after https://github.com/apache/arrow-datafusion/pull/1153 has been merged
Types of standard rewrites I think it would be good to have:
```
true OR <expr> --> true
false OR <expr> --> expr
true AND <expr> --> <expr>
false AND <expr> --> false
```
There is also an interesting one in https://github.com/apache/arrow-datafusion/pull/1153 that would rotatate expr trees to isolate volatile functions:
```
(rand() + 1) + 2 --> rand() + (1 + 2)
```
The reason for doing that rewrite is that then the constant evaluator could convert to `rand() + 3`
**Describe alternatives you've considered**
There are various automated rewriting rules such as egg / tokomak (drafted in https://github.com/apache/arrow-datafusion/pull/1066 and https://github.com/apache/arrow-datafusion/pull/441 by @Dandandan and @pjmore ) which might be better / more powerful than implementing our own rules
**Additional context**
I am sure other query optimization systems have a good set of rewrite rules that we could investigate. I bet both postgres and spark have interesting optimizations to do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb closed issue #1162: Add additional algebraic simplifications
Posted by GitBox <gi...@apache.org>.
alamb closed issue #1162:
URL: https://github.com/apache/arrow-datafusion/issues/1162
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #1162: Add additional algebraic simplifications
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #1162:
URL: https://github.com/apache/arrow-datafusion/issues/1162#issuecomment-954008545
@matthewmturner I think you might find some interesting work here if you wanted
Specifically the idea would be to add the simplification rules in the description of this PR to the logic in `Simplify` here:
https://github.com/apache/arrow-datafusion/blob/2c8e65bcca9fe41ad80116d99ad974c86cb59654/datafusion/src/optimizer/constant_folding.rs#L140-L230
And then write tests following the pattern here: https://github.com/apache/arrow-datafusion/blob/2c8e65bcca9fe41ad80116d99ad974c86cb59654/datafusion/src/optimizer/constant_folding.rs#L267-L279
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] matthewmturner commented on issue #1162: Add additional algebraic simplifications
Posted by GitBox <gi...@apache.org>.
matthewmturner commented on issue #1162:
URL: https://github.com/apache/arrow-datafusion/issues/1162#issuecomment-958140912
@alamb any next steps here that I can work on?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] matthewmturner commented on issue #1162: Add additional algebraic simplifications
Posted by GitBox <gi...@apache.org>.
matthewmturner commented on issue #1162:
URL: https://github.com/apache/arrow-datafusion/issues/1162#issuecomment-954011669
@alamb thanks! will check it out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb closed issue #1162: Add additional algebraic simplifications
Posted by GitBox <gi...@apache.org>.
alamb closed issue #1162:
URL: https://github.com/apache/arrow-datafusion/issues/1162
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #1162: Add additional algebraic simplifications
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #1162:
URL: https://github.com/apache/arrow-datafusion/issues/1162#issuecomment-958161846
@matthewmturner -- I am not sure -- it might be worth looking at #1066 and #441 to see if the rules in that provide any additional inspiration
There are also transformations of stuff like `(2 + (1 + rand()))` which could be written to `(2 + 1) + rand()` and then constant folded where it wouldn't be folded before.
THough I think the most interesting / useful thing might be https://github.com/apache/arrow-datafusion/issues/1160 but that is likely a bit more involved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] matthewmturner commented on issue #1162: Add additional algebraic simplifications
Posted by GitBox <gi...@apache.org>.
matthewmturner commented on issue #1162:
URL: https://github.com/apache/arrow-datafusion/issues/1162#issuecomment-958140912
@alamb any next steps here that I can work on?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org