You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/05/15 14:02:19 UTC
[GitHub] [arrow-datafusion] andygrove opened a new issue, #2535: Logical optimizer rule "simplify expressions" should not depend on physical expressions
andygrove opened a new issue, #2535:
URL: https://github.com/apache/arrow-datafusion/issues/2535
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
In the "simplify expressions" rule, we evaluate and fold expressions that result in scalar values and we depend on the physical expression crate to do this. I do not see why we cannot do scalar expression evaluation directly without creating physical expressions.
For users of DataFusion that just want SQL planning and logical query plans to use with a different physical plan, I think this would be a better approach,
**Describe the solution you'd like**
Perform scalar expression evaluation directly in the logical plan/expression crate.
**Describe alternatives you've considered**
None
**Additional context**
This is related to https://github.com/apache/arrow-datafusion/issues/2345
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #2535: Logical optimizer rule "simplify expressions" should not depend on physical planner and expressions
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #2535:
URL: https://github.com/apache/arrow-datafusion/issues/2535#issuecomment-1135932491
I think it is important for expression simplification that expressions can be partially evaluated at optimization time.
In terms of evaluating on logical plan `Expr`s directly, I agree there is no theoretical reason this can not be done, but if we did so it will result in dual expression evaluation systems that would have be kept in sync. This might be especially hard if someone wanted to use a different physical expression evaluation system
I understand the desire to remove the direct code dependence on the execution engine from the optimizer -- perhaps we can do so via a trait -- for example, we could pass some sort of `dyn ExprEvaluator` to the optimizer for use in `simplify expressions` and elsewhere.
By default, DataFusion could provide the evaluator based on `PhysicalExpr` -- and for those users of the optimizer itself they could decide the most appropriate way to provide expression evaluation to the optimizer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] andygrove commented on issue #2535: Logical optimizer rule "simplify expressions" should not depend on physical planner and expressions
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #2535:
URL: https://github.com/apache/arrow-datafusion/issues/2535#issuecomment-1144972906
@alamb Now that I have started looking at this more closely it turns out to be less of an issue than I first thought. We just need to move `create_physical_expr` from core to the `physical-expr` crate. I have updated the issue description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb closed issue #2535: Logical optimizer rule "simplify expressions" should not depend on the core datafusion crate
Posted by GitBox <gi...@apache.org>.
alamb closed issue #2535: Logical optimizer rule "simplify expressions" should not depend on the core datafusion crate
URL: https://github.com/apache/arrow-datafusion/issues/2535
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org