You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/07/06 08:22:48 UTC
[GitHub] [doris] jackwener opened a new pull request, #10647: [feature](planner): push sort to olapscan.
jackwener opened a new pull request, #10647:
URL: https://github.com/apache/doris/pull/10647
# Proposed changes
Issue Number: close #10646.
## Problem Summary:
Describe the overview of changes.
push sort to olapscan.
## Checklist(Required)
1. Does it affect the original behavior: (Yes)
2. Has unit tests been added: (No)
3. Has document been added or modified: (No need)
4. Does it need to update dependencies: (No)
5. Are there any changes that cannot be rolled back: (No)
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #10647: [feature](planner): push sort to olapscan.
Posted by GitBox <gi...@apache.org>.
morrySnow commented on code in PR #10647:
URL: https://github.com/apache/doris/pull/10647#discussion_r914581350
##########
fe/fe-core/src/main/java/org/apache/doris/planner/OlapScanNode.java:
##########
@@ -693,6 +708,37 @@ private void computeTabletInfo() throws UserException {
}
}
+ /**
+ * Check Parent sort node can push down to child olap scan.
Review Comment:
add more comment to explain why use prefix matching
##########
fe/fe-core/src/main/java/org/apache/doris/planner/OriginalPlanner.java:
##########
@@ -197,6 +197,28 @@ public void createPlanFragments(StatementBase statement, Analyzer analyzer, TQue
fragments = distributedPlanner.createPlanFragments(singleNodePlan);
}
+ // Push sort node down to the bottom of olapscan.
+ for (PlanFragment fragment : fragments) {
+ if (fragment.getPlanRoot() instanceof SortNode) {
Review Comment:
we need to find a pattern sort(scan), so should use recursive way to do that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] xiaokang commented on a diff in pull request #10647: [feature](planner): push sort to olapscan.
Posted by GitBox <gi...@apache.org>.
xiaokang commented on code in PR #10647:
URL: https://github.com/apache/doris/pull/10647#discussion_r916498906
##########
fe/fe-core/src/main/java/org/apache/doris/planner/OriginalPlanner.java:
##########
@@ -316,6 +320,45 @@ private void pushDownResultFileSink(Analyzer analyzer) {
topPlanFragment.getPlanRoot().resetTupleIds(Lists.newArrayList(fileStatusDesc.getId()));
}
+ /**
+ * Push sort down to olap scan.
+ */
+ private void pushSortToOlapScan() {
+ for (PlanFragment fragment : fragments) {
+ PlanNode node = fragment.getPlanRoot();
+ PlanNode parent = null;
+
+ // OlapScanNode is the last node.
+ // So, just get the last two node and check if they are SortNode and OlapScan.
+ while (node.getChildren().size() != 0) {
+ parent = node;
+ node = node.getChildren().get(0);
+ }
+
+ if (!(node instanceof OlapScanNode) || !(parent instanceof SortNode)) {
+ continue;
+ }
+
+ SortNode sortNode = (SortNode) parent;
+ // Ensure all isAscOrder is same, ande length != 0.
+ if (sortNode.getSortInfo().getIsAscOrder().stream().distinct().count() != 1) {
+ continue;
+ }
+ // If offset > 0, we can't push down sort to olap scan.
Review Comment:
If offset > 0, maybe we can still push down sort to olap scan. Just set scan node's limit to limit + offset.
##########
fe/fe-core/src/main/java/org/apache/doris/planner/OriginalPlanner.java:
##########
@@ -316,6 +320,45 @@ private void pushDownResultFileSink(Analyzer analyzer) {
topPlanFragment.getPlanRoot().resetTupleIds(Lists.newArrayList(fileStatusDesc.getId()));
}
+ /**
+ * Push sort down to olap scan.
+ */
+ private void pushSortToOlapScan() {
+ for (PlanFragment fragment : fragments) {
+ PlanNode node = fragment.getPlanRoot();
+ PlanNode parent = null;
+
+ // OlapScanNode is the last node.
+ // So, just get the last two node and check if they are SortNode and OlapScan.
+ while (node.getChildren().size() != 0) {
+ parent = node;
+ node = node.getChildren().get(0);
+ }
+
+ if (!(node instanceof OlapScanNode) || !(parent instanceof SortNode)) {
+ continue;
+ }
+
+ SortNode sortNode = (SortNode) parent;
+ // Ensure all isAscOrder is same, ande length != 0.
+ if (sortNode.getSortInfo().getIsAscOrder().stream().distinct().count() != 1) {
Review Comment:
we can put all check to method checkPushSort
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #10647: [feature](planner): push limit to olapscan when meet sort.
Posted by GitBox <gi...@apache.org>.
jackwener commented on PR #10647:
URL: https://github.com/apache/doris/pull/10647#issuecomment-1199268347
Merge it into #10694
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] jackwener closed pull request #10647: [feature](planner): push limit to olapscan when meet sort.
Posted by GitBox <gi...@apache.org>.
jackwener closed pull request #10647: [feature](planner): push limit to olapscan when meet sort.
URL: https://github.com/apache/doris/pull/10647
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] jackwener closed pull request #10647: [feature](planner): push limit to olapscan when meet sort.
Posted by GitBox <gi...@apache.org>.
jackwener closed pull request #10647: [feature](planner): push limit to olapscan when meet sort.
URL: https://github.com/apache/doris/pull/10647
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org