You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/11/14 22:15:15 UTC
[GitHub] [arrow-datafusion] andygrove opened a new issue, #4209: Allow optimizer rules to skip optimizing plans
andygrove opened a new issue, #4209:
URL: https://github.com/apache/arrow-datafusion/issues/4209
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Many rules are only applicable to certain types of plan. Currently a rule has two choices if it cannot optimize the given plan:
- Return a clone of the input plan
- Return an Err
**Describe the solution you'd like**
I would like a third option, where the rule can return "not applicable".
**Describe alternatives you've considered**
None
**Additional context**
None
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] HaoYang670 commented on issue #4209: Allow optimizer rules to skip optimizing plans
Posted by GitBox <gi...@apache.org>.
HaoYang670 commented on issue #4209:
URL: https://github.com/apache/arrow-datafusion/issues/4209#issuecomment-1314947347
Wonder if we could also do this update in physical optimizer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] andygrove commented on issue #4209: Allow optimizer rules to skip optimizing plans
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #4209:
URL: https://github.com/apache/arrow-datafusion/issues/4209#issuecomment-1315487638
@HaoYang670 Perhaps we can copy Spark's terminology and have an "analysis phase"? Do you want to file an issue for this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] andygrove commented on issue #4209: Allow optimizer rules to skip optimizing plans
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #4209:
URL: https://github.com/apache/arrow-datafusion/issues/4209#issuecomment-1315488330
> Wonder if we could also do this update in physical optimizer.
Sounds good, but I think that should be a separate issue & PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] HaoYang670 commented on issue #4209: Allow optimizer rules to skip optimizing plans
Posted by GitBox <gi...@apache.org>.
HaoYang670 commented on issue #4209:
URL: https://github.com/apache/arrow-datafusion/issues/4209#issuecomment-1315004883
Another concern (may be not related to this issue) is that should we move the `type_coercion` out from the `optimizer` and put it into a new mod (maybe call it `type_checker`)?
Because what the `type_coercion` does is different from an optimizer, but more like a type-checker in the compiler. It checks the plan, but not optimizes the plan:
```
Logical Plan -->> (type checker / type_coercion) -->> Sensible Logical Plan -->> (Logical optimizer) -->> Optimized Logical Plan
|
\|/
Return error if the plan break the type rules
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] mingmwang commented on issue #4209: Allow optimizer rules to skip optimizing plans
Posted by GitBox <gi...@apache.org>.
mingmwang commented on issue #4209:
URL: https://github.com/apache/arrow-datafusion/issues/4209#issuecomment-1315510211
> Wonder if we could also do this update in physical optimizer.
For physical optimizer, we already have utils to return Option. The 'op' here is a Closure which can return Option if the 'op'
does not apply. It is similar to the Partial Apply Func in Scala.
https://github.com/apache/arrow-datafusion/blob/d72b4f04cf12424a54b22bf376b87065d5fcb6aa/datafusion/core/src/physical_plan/rewrite.rs#L76-L84
@andygrove @HaoYang670
I am not sure whether this is your expections.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb closed issue #4209: Allow optimizer rules to skip optimizing plans
Posted by GitBox <gi...@apache.org>.
alamb closed issue #4209: Allow optimizer rules to skip optimizing plans
URL: https://github.com/apache/arrow-datafusion/issues/4209
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org