You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "englefly (via GitHub)" <gi...@apache.org> on 2024/03/22 10:33:07 UTC
[PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
englefly opened a new pull request, #32697:
URL: https://github.com/apache/doris/pull/32697
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2053001511
<details>
<summary>ClickBench: <b>Total hot run time: 29.91 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit af4efb8900a8ca8c0682ab4576282fb8b57c815d, data reload: false
query1 0.04 0.03 0.04
query2 0.08 0.04 0.04
query3 0.23 0.04 0.05
query4 1.67 0.08 0.07
query5 0.51 0.50 0.48
query6 1.13 0.65 0.65
query7 0.02 0.02 0.01
query8 0.05 0.04 0.04
query9 0.56 0.50 0.51
query10 0.56 0.57 0.55
query11 0.16 0.11 0.11
query12 0.15 0.12 0.12
query13 0.61 0.61 0.59
query14 0.77 0.79 0.78
query15 0.84 0.82 0.83
query16 0.38 0.39 0.38
query17 1.00 0.96 1.00
query18 0.21 0.27 0.23
query19 1.83 1.73 1.82
query20 0.02 0.01 0.01
query21 15.44 0.66 0.66
query22 5.01 6.96 1.40
query23 18.21 1.41 1.31
query24 2.08 0.22 0.21
query25 0.14 0.08 0.08
query26 0.26 0.17 0.17
query27 0.09 0.09 0.08
query28 13.34 1.02 0.98
query29 12.63 3.25 3.26
query30 0.25 0.06 0.05
query31 2.86 0.40 0.38
query32 3.27 0.48 0.47
query33 2.83 2.89 2.84
query34 17.12 4.43 4.38
query35 4.51 4.44 4.45
query36 0.65 0.48 0.46
query37 0.19 0.16 0.16
query38 0.16 0.15 0.14
query39 0.05 0.04 0.04
query40 0.17 0.15 0.15
query41 0.09 0.04 0.05
query42 0.06 0.04 0.04
query43 0.04 0.04 0.04
Total cold run time: 110.27 s
Total hot run time: 29.91 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2052964249
<details>
<summary>TPC-H: <b>Total hot run time: 38552 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit af4efb8900a8ca8c0682ab4576282fb8b57c815d, data reload: false
------ Round 1 ----------------------------------
q1 17607 4379 4229 4229
q2 2014 203 186 186
q3 10440 1195 1129 1129
q4 10190 816 753 753
q5 7557 2725 2670 2670
q6 220 133 133 133
q7 1006 617 603 603
q8 9218 2038 2037 2037
q9 7860 6553 6547 6547
q10 8591 3518 3520 3518
q11 469 245 258 245
q12 507 233 231 231
q13 18045 2988 3011 2988
q14 260 233 235 233
q15 521 480 465 465
q16 542 386 395 386
q17 972 712 689 689
q18 7488 6765 6688 6688
q19 7314 1546 1506 1506
q20 708 332 310 310
q21 3565 2858 2721 2721
q22 360 298 285 285
Total cold run time: 115454 ms
Total hot run time: 38552 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4440 4219 4218 4218
q2 396 279 260 260
q3 2994 2781 2746 2746
q4 1865 1575 1631 1575
q5 5338 5367 5279 5279
q6 209 122 125 122
q7 2251 1905 1874 1874
q8 3222 3345 3279 3279
q9 8572 8598 8798 8598
q10 4055 3857 3948 3857
q11 626 507 510 507
q12 857 645 658 645
q13 16281 3205 3198 3198
q14 323 317 285 285
q15 506 473 520 473
q16 532 467 455 455
q17 1853 1557 1528 1528
q18 8075 7942 7892 7892
q19 1729 1525 1580 1525
q20 2006 1874 1862 1862
q21 5475 5139 5055 5055
q22 609 519 523 519
Total cold run time: 72214 ms
Total hot run time: 55752 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2053007428
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit af4efb8900a8ca8c0682ab4576282fb8b57c815d with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 60 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 34 seconds loaded 861443392 Bytes, about 24 MB/s
Insert into select: 13.5 seconds inserted 10000000 Rows, about 740K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #32697:
URL: https://github.com/apache/doris/pull/32697#discussion_r1557321645
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/ProjectOtherJoinConditionForNestedLoopJoin.java:
##########
@@ -0,0 +1,128 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.rewrite;
+
+import org.apache.doris.nereids.rules.Rule;
+import org.apache.doris.nereids.rules.RuleType;
+import org.apache.doris.nereids.trees.expressions.Alias;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.NamedExpression;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.expressions.visitor.DefaultExpressionRewriter;
+import org.apache.doris.nereids.trees.plans.Plan;
+import org.apache.doris.nereids.trees.plans.logical.LogicalProject;
+
+import com.google.common.collect.Lists;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * join (l_orderkey > n_nationkey + n_regionkey)
+ * +----scan(lineItem)
+ * +----scan(nation)
+ * =>
+ * join(l_orderkey > x)
+ * +----scan(lineItem)
+ * +----project(n_nationkey + n_regionkey as x)
+ * +----scan(nation)
+ */
+public class ProjectOtherJoinConditionForNestedLoopJoin extends OneRewriteRuleFactory {
+ @Override
+ public Rule build() {
+ return logicalJoin().when(
+ join -> join.getHashJoinConjuncts().isEmpty()
+ && !join.isMarkJoin()
+ && !join.getOtherJoinConjuncts().isEmpty()
+ && !join.getJoinReorderContext().isExtractSubExpressionForChild()
+ ).then(
+ join -> {
+ join.getJoinReorderContext().setExtractSubExpressionForChild(true);
+ List<Expression> otherConjuncts = join.getOtherJoinConjuncts();
+ List<Expression> newOtherConjuncts = new ArrayList<>();
+ Set<Slot> leftSlots = join.child(0).getOutputSet();
+ Set<Slot> rightSlots = join.child(1).getOutputSet();
+ ReplacerContext ctx = new ReplacerContext(leftSlots, rightSlots);
+ for (Expression conj : otherConjuncts) {
+ Expression newConj = conj.accept(AliasReplacer.INSTANCE, ctx);
+ newOtherConjuncts.add(newConj);
+ }
+ boolean changed = !ctx.leftAlias.isEmpty() || !ctx.rightAlias.isEmpty();
+ if (changed) {
+ Plan left = join.left();
+ if (!ctx.leftAlias.isEmpty()) {
+ List<NamedExpression> newProjects = Lists.newArrayList(left.getOutput());
+ newProjects.addAll(ctx.leftAlias);
+ left = new LogicalProject<>(newProjects, left);
+ }
+ Plan right = join.right();
+ if (!ctx.rightAlias.isEmpty()) {
+ List<NamedExpression> newProjects = Lists.newArrayList(right.getOutput());
+ newProjects.addAll(ctx.rightAlias);
+ right = new LogicalProject<>(newProjects, right);
+ }
+ join = join.withJoinConjuncts(join.getHashJoinConjuncts(),
+ newOtherConjuncts, join.getJoinReorderContext())
+ .withChildren(Lists.newArrayList(left, right));
Review Comment:
```suggestion
.withChildren(ImmutableList.of(left, right));
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/ProjectOtherJoinConditionForNestedLoopJoin.java:
##########
@@ -0,0 +1,128 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.rewrite;
+
+import org.apache.doris.nereids.rules.Rule;
+import org.apache.doris.nereids.rules.RuleType;
+import org.apache.doris.nereids.trees.expressions.Alias;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.NamedExpression;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.expressions.visitor.DefaultExpressionRewriter;
+import org.apache.doris.nereids.trees.plans.Plan;
+import org.apache.doris.nereids.trees.plans.logical.LogicalProject;
+
+import com.google.common.collect.Lists;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * join (l_orderkey > n_nationkey + n_regionkey)
+ * +----scan(lineItem)
+ * +----scan(nation)
+ * =>
+ * join(l_orderkey > x)
+ * +----scan(lineItem)
+ * +----project(n_nationkey + n_regionkey as x)
+ * +----scan(nation)
+ */
+public class ProjectOtherJoinConditionForNestedLoopJoin extends OneRewriteRuleFactory {
+ @Override
+ public Rule build() {
+ return logicalJoin().when(
+ join -> join.getHashJoinConjuncts().isEmpty()
+ && !join.isMarkJoin()
+ && !join.getOtherJoinConjuncts().isEmpty()
+ && !join.getJoinReorderContext().isExtractSubExpressionForChild()
+ ).then(
+ join -> {
+ join.getJoinReorderContext().setExtractSubExpressionForChild(true);
Review Comment:
why need this? just like push hash join conjuncts, when not changed, we return null then the rule will not fall into dead loop
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/ProjectOtherJoinConditionForNestedLoopJoin.java:
##########
@@ -0,0 +1,128 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.rewrite;
+
+import org.apache.doris.nereids.rules.Rule;
+import org.apache.doris.nereids.rules.RuleType;
+import org.apache.doris.nereids.trees.expressions.Alias;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.NamedExpression;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.expressions.visitor.DefaultExpressionRewriter;
+import org.apache.doris.nereids.trees.plans.Plan;
+import org.apache.doris.nereids.trees.plans.logical.LogicalProject;
+
+import com.google.common.collect.Lists;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * join (l_orderkey > n_nationkey + n_regionkey)
+ * +----scan(lineItem)
+ * +----scan(nation)
+ * =>
+ * join(l_orderkey > x)
+ * +----scan(lineItem)
+ * +----project(n_nationkey + n_regionkey as x)
+ * +----scan(nation)
+ */
+public class ProjectOtherJoinConditionForNestedLoopJoin extends OneRewriteRuleFactory {
+ @Override
+ public Rule build() {
+ return logicalJoin().when(
+ join -> join.getHashJoinConjuncts().isEmpty()
+ && !join.isMarkJoin()
+ && !join.getOtherJoinConjuncts().isEmpty()
+ && !join.getJoinReorderContext().isExtractSubExpressionForChild()
+ ).then(
+ join -> {
+ join.getJoinReorderContext().setExtractSubExpressionForChild(true);
+ List<Expression> otherConjuncts = join.getOtherJoinConjuncts();
+ List<Expression> newOtherConjuncts = new ArrayList<>();
+ Set<Slot> leftSlots = join.child(0).getOutputSet();
+ Set<Slot> rightSlots = join.child(1).getOutputSet();
+ ReplacerContext ctx = new ReplacerContext(leftSlots, rightSlots);
+ for (Expression conj : otherConjuncts) {
+ Expression newConj = conj.accept(AliasReplacer.INSTANCE, ctx);
+ newOtherConjuncts.add(newConj);
+ }
+ boolean changed = !ctx.leftAlias.isEmpty() || !ctx.rightAlias.isEmpty();
+ if (changed) {
+ Plan left = join.left();
+ if (!ctx.leftAlias.isEmpty()) {
+ List<NamedExpression> newProjects = Lists.newArrayList(left.getOutput());
+ newProjects.addAll(ctx.leftAlias);
+ left = new LogicalProject<>(newProjects, left);
+ }
+ Plan right = join.right();
+ if (!ctx.rightAlias.isEmpty()) {
+ List<NamedExpression> newProjects = Lists.newArrayList(right.getOutput());
+ newProjects.addAll(ctx.rightAlias);
+ right = new LogicalProject<>(newProjects, right);
+ }
+ join = join.withJoinConjuncts(join.getHashJoinConjuncts(),
+ newOtherConjuncts, join.getJoinReorderContext())
+ .withChildren(Lists.newArrayList(left, right));
+ }
+ return join;
+ }
+ ).toRule(RuleType.PROJECT_OTHER_JOIN_CONDITION);
+ }
+
+ private static class ReplacerContext {
+ HashMap<Expression, Alias> aliasMap = new HashMap<>();
+ Set<Slot> leftSlots;
+ Set<Slot> rightSlots;
+ Set<Alias> leftAlias = new HashSet<>();
Review Comment:
LinkedHashSet?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2068606249
<details>
<summary>TPC-H: <b>Total hot run time: 38623 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 14589fa31b5ca60828cd873e505762d77b62494b, data reload: false
------ Round 1 ----------------------------------
q1 17605 4386 4253 4253
q2 2015 191 184 184
q3 10449 1138 1196 1138
q4 10184 793 777 777
q5 7506 2716 2644 2644
q6 220 135 135 135
q7 1014 611 592 592
q8 9281 2067 2051 2051
q9 7249 6581 6522 6522
q10 8588 3532 3513 3513
q11 452 245 238 238
q12 439 219 225 219
q13 17940 2942 2961 2942
q14 270 240 233 233
q15 530 478 471 471
q16 521 418 381 381
q17 958 684 697 684
q18 7309 6773 6731 6731
q19 6754 1497 1480 1480
q20 647 325 307 307
q21 3459 2900 2852 2852
q22 339 279 276 276
Total cold run time: 113729 ms
Total hot run time: 38623 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4336 4231 4240 4231
q2 368 281 271 271
q3 2996 2693 2687 2687
q4 1866 1628 1630 1628
q5 5347 5372 5283 5283
q6 208 121 128 121
q7 2219 1881 1882 1881
q8 3193 3364 3331 3331
q9 8563 8542 8667 8542
q10 4060 3864 3994 3864
q11 601 496 502 496
q12 813 618 642 618
q13 17415 3273 3101 3101
q14 309 278 309 278
q15 516 480 475 475
q16 498 449 482 449
q17 1820 1537 1510 1510
q18 8156 8155 7826 7826
q19 1665 1521 1616 1521
q20 2044 1844 1820 1820
q21 7475 4954 4912 4912
q22 579 489 481 481
Total cold run time: 75047 ms
Total hot run time: 55326 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2068631487
<details>
<summary>ClickBench: <b>Total hot run time: 30.11 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 14589fa31b5ca60828cd873e505762d77b62494b, data reload: false
query1 0.04 0.03 0.03
query2 0.09 0.04 0.04
query3 0.23 0.04 0.05
query4 1.68 0.07 0.06
query5 0.50 0.50 0.51
query6 1.48 0.73 0.72
query7 0.02 0.01 0.01
query8 0.05 0.04 0.04
query9 0.53 0.50 0.49
query10 0.55 0.56 0.56
query11 0.15 0.12 0.11
query12 0.15 0.11 0.11
query13 0.59 0.59 0.59
query14 0.76 0.77 0.76
query15 0.82 0.78 0.81
query16 0.36 0.36 0.36
query17 0.92 0.93 0.98
query18 0.20 0.27 0.20
query19 1.79 1.65 1.70
query20 0.01 0.01 0.00
query21 15.42 0.65 0.65
query22 4.40 7.30 1.83
query23 18.33 1.27 1.25
query24 1.29 0.37 0.31
query25 0.14 0.08 0.08
query26 0.26 0.16 0.15
query27 0.08 0.08 0.07
query28 13.26 0.99 0.98
query29 12.62 3.28 3.26
query30 0.26 0.07 0.05
query31 2.86 0.37 0.38
query32 3.29 0.46 0.46
query33 2.83 2.83 2.78
query34 17.09 4.41 4.41
query35 4.47 4.48 4.45
query36 0.63 0.47 0.46
query37 0.19 0.15 0.15
query38 0.15 0.14 0.14
query39 0.04 0.03 0.03
query40 0.16 0.14 0.13
query41 0.09 0.06 0.05
query42 0.05 0.04 0.05
query43 0.04 0.03 0.03
Total cold run time: 108.87 s
Total hot run time: 30.11 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2071210436
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2014799306
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2052991046
<details>
<summary>TPC-DS: <b>Total hot run time: 186154 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit af4efb8900a8ca8c0682ab4576282fb8b57c815d, data reload: false
query1 892 1132 1119 1119
query2 6178 2931 2662 2662
query3 6666 208 206 206
query4 36943 21722 21466 21466
query5 4134 417 404 404
query6 243 184 181 181
query7 4052 310 296 296
query8 227 179 182 179
query9 5885 2490 2485 2485
query10 371 244 240 240
query11 14636 14250 14242 14242
query12 147 95 93 93
query13 1023 386 387 386
query14 8896 6821 6924 6821
query15 214 177 176 176
query16 7301 275 268 268
query17 1655 609 576 576
query18 1632 284 269 269
query19 194 160 151 151
query20 102 88 93 88
query21 205 131 125 125
query22 5022 4934 4853 4853
query23 33741 32903 33322 32903
query24 8663 3046 3058 3046
query25 513 405 406 405
query26 878 165 157 157
query27 2986 393 387 387
query28 6680 2278 2300 2278
query29 857 623 614 614
query30 301 175 179 175
query31 937 770 750 750
query32 61 53 52 52
query33 507 255 262 255
query34 907 511 520 511
query35 803 731 710 710
query36 1132 1015 965 965
query37 111 70 70 70
query38 3751 3655 3657 3655
query39 1634 1590 1601 1590
query40 176 130 130 130
query41 51 47 42 42
query42 111 99 99 99
query43 598 577 606 577
query44 1227 778 766 766
query45 292 284 260 260
query46 1107 759 758 758
query47 2092 1953 1995 1953
query48 407 308 324 308
query49 791 387 377 377
query50 803 422 436 422
query51 6976 6985 6747 6747
query52 105 89 93 89
query53 351 281 278 278
query54 265 234 232 232
query55 79 73 73 73
query56 249 225 233 225
query57 1248 1146 1130 1130
query58 220 205 204 204
query59 3671 3353 3281 3281
query60 256 234 239 234
query61 93 88 96 88
query62 598 438 446 438
query63 305 282 280 280
query64 4172 4003 4098 4003
query65 3100 3035 3059 3035
query66 730 339 330 330
query67 15643 15275 15050 15050
query68 7040 574 573 573
query69 595 313 317 313
query70 1343 1169 1258 1169
query71 456 280 288 280
query72 6489 2679 2439 2439
query73 822 334 338 334
query74 6890 6378 6374 6374
query75 3142 2355 2283 2283
query76 4163 1077 1139 1077
query77 626 253 256 253
query78 11108 10275 10174 10174
query79 3403 536 548 536
query80 2374 441 427 427
query81 550 244 235 235
query82 1413 100 101 100
query83 354 176 175 175
query84 262 90 86 86
query85 1032 271 271 271
query86 472 313 316 313
query87 3768 3528 3554 3528
query88 5616 2458 2452 2452
query89 483 380 381 380
query90 1911 177 176 176
query91 125 101 98 98
query92 58 48 48 48
query93 4998 546 542 542
query94 1073 207 179 179
query95 395 290 297 290
query96 603 278 278 278
query97 2693 2461 2485 2461
query98 235 230 222 222
query99 1223 870 884 870
Total cold run time: 292511 ms
Total hot run time: 186154 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2074061684
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly merged PR #32697:
URL: https://github.com/apache/doris/pull/32697
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on code in PR #32697:
URL: https://github.com/apache/doris/pull/32697#discussion_r1561971258
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/ProjectOtherJoinConditionForNestedLoopJoin.java:
##########
@@ -0,0 +1,128 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.rewrite;
+
+import org.apache.doris.nereids.rules.Rule;
+import org.apache.doris.nereids.rules.RuleType;
+import org.apache.doris.nereids.trees.expressions.Alias;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.NamedExpression;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.expressions.visitor.DefaultExpressionRewriter;
+import org.apache.doris.nereids.trees.plans.Plan;
+import org.apache.doris.nereids.trees.plans.logical.LogicalProject;
+
+import com.google.common.collect.Lists;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+/**
+ * join (l_orderkey > n_nationkey + n_regionkey)
+ * +----scan(lineItem)
+ * +----scan(nation)
+ * =>
+ * join(l_orderkey > x)
+ * +----scan(lineItem)
+ * +----project(n_nationkey + n_regionkey as x)
+ * +----scan(nation)
+ */
+public class ProjectOtherJoinConditionForNestedLoopJoin extends OneRewriteRuleFactory {
+ @Override
+ public Rule build() {
+ return logicalJoin().when(
+ join -> join.getHashJoinConjuncts().isEmpty()
+ && !join.isMarkJoin()
+ && !join.getOtherJoinConjuncts().isEmpty()
+ && !join.getJoinReorderContext().isExtractSubExpressionForChild()
+ ).then(
+ join -> {
+ join.getJoinReorderContext().setExtractSubExpressionForChild(true);
+ List<Expression> otherConjuncts = join.getOtherJoinConjuncts();
+ List<Expression> newOtherConjuncts = new ArrayList<>();
+ Set<Slot> leftSlots = join.child(0).getOutputSet();
+ Set<Slot> rightSlots = join.child(1).getOutputSet();
+ ReplacerContext ctx = new ReplacerContext(leftSlots, rightSlots);
+ for (Expression conj : otherConjuncts) {
+ Expression newConj = conj.accept(AliasReplacer.INSTANCE, ctx);
+ newOtherConjuncts.add(newConj);
+ }
+ boolean changed = !ctx.leftAlias.isEmpty() || !ctx.rightAlias.isEmpty();
+ if (changed) {
+ Plan left = join.left();
+ if (!ctx.leftAlias.isEmpty()) {
+ List<NamedExpression> newProjects = Lists.newArrayList(left.getOutput());
+ newProjects.addAll(ctx.leftAlias);
+ left = new LogicalProject<>(newProjects, left);
+ }
+ Plan right = join.right();
+ if (!ctx.rightAlias.isEmpty()) {
+ List<NamedExpression> newProjects = Lists.newArrayList(right.getOutput());
+ newProjects.addAll(ctx.rightAlias);
+ right = new LogicalProject<>(newProjects, right);
+ }
+ join = join.withJoinConjuncts(join.getHashJoinConjuncts(),
+ newOtherConjuncts, join.getJoinReorderContext())
+ .withChildren(Lists.newArrayList(left, right));
+ }
+ return join;
+ }
+ ).toRule(RuleType.PROJECT_OTHER_JOIN_CONDITION);
+ }
+
+ private static class ReplacerContext {
+ HashMap<Expression, Alias> aliasMap = new HashMap<>();
+ Set<Slot> leftSlots;
+ Set<Slot> rightSlots;
+ Set<Alias> leftAlias = new HashSet<>();
Review Comment:
we do not case about the order here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2014799188
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2014853228
<details>
<summary>TPC-DS: <b>Total hot run time: 186769 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 030bff9a47f12a5f3173b9e18608f7512acf19cf, data reload: false
query1 945 372 369 369
query2 7493 1990 1933 1933
query3 6716 219 221 219
query4 31875 21310 21421 21310
query5 4935 417 427 417
query6 266 174 181 174
query7 4623 295 286 286
query8 230 181 176 176
query9 9393 2339 2336 2336
query10 568 249 246 246
query11 17305 14339 14416 14339
query12 133 94 90 90
query13 1624 433 419 419
query14 11978 11402 10796 10796
query15 269 200 189 189
query16 8198 273 264 264
query17 1934 583 558 558
query18 2103 311 279 279
query19 344 156 155 155
query20 97 83 95 83
query21 209 122 133 122
query22 5017 4845 4828 4828
query23 33345 32581 33013 32581
query24 10722 2895 2924 2895
query25 632 402 396 396
query26 1399 161 159 159
query27 2988 368 363 363
query28 7445 1947 1880 1880
query29 913 663 636 636
query30 304 148 150 148
query31 926 721 753 721
query32 96 57 55 55
query33 780 258 256 256
query34 1063 511 503 503
query35 825 628 607 607
query36 1068 888 896 888
query37 112 78 79 78
query38 3552 3452 3405 3405
query39 1493 1455 1413 1413
query40 219 111 112 111
query41 51 52 47 47
query42 104 94 101 94
query43 489 432 433 432
query44 1221 709 721 709
query45 287 269 266 266
query46 1128 697 718 697
query47 1936 1843 1869 1843
query48 459 355 355 355
query49 1151 348 346 346
query50 760 377 376 376
query51 6648 6517 6510 6510
query52 108 88 94 88
query53 342 277 281 277
query54 1234 438 431 431
query55 81 80 79 79
query56 256 230 243 230
query57 1226 1157 1158 1157
query58 242 212 206 206
query59 2835 2515 2600 2515
query60 276 274 265 265
query61 116 115 116 115
query62 663 453 439 439
query63 303 272 275 272
query64 5852 3995 3981 3981
query65 3141 3047 3070 3047
query66 1434 364 354 354
query67 15564 14806 14815 14806
query68 7213 535 541 535
query69 635 386 384 384
query70 1170 1150 1139 1139
query71 504 276 275 275
query72 6375 2749 2562 2562
query73 736 314 318 314
query74 8302 6683 6704 6683
query75 4381 2915 2918 2915
query76 4576 905 878 878
query77 651 258 255 255
query78 11063 10295 10140 10140
query79 12150 534 516 516
query80 1720 377 384 377
query81 522 215 209 209
query82 865 204 207 204
query83 221 147 151 147
query84 289 81 76 76
query85 1485 324 315 315
query86 468 298 312 298
query87 3786 3562 3550 3550
query88 5571 2360 2363 2360
query89 520 367 364 364
query90 1963 179 177 177
query91 172 135 138 135
query92 65 47 47 47
query93 7565 514 500 500
query94 1208 185 179 179
query95 434 337 339 337
query96 617 273 276 273
query97 3087 2876 2865 2865
query98 232 214 210 210
query99 1267 940 930 930
Total cold run time: 321857 ms
Total hot run time: 186769 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2073883062
run feut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2044354963
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2069048447
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2068623028
<details>
<summary>TPC-DS: <b>Total hot run time: 185835 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 14589fa31b5ca60828cd873e505762d77b62494b, data reload: false
query1 901 358 355 355
query2 6160 2712 2409 2409
query3 6656 204 202 202
query4 22854 21245 21269 21245
query5 4104 402 411 402
query6 285 196 192 192
query7 4587 291 284 284
query8 239 182 184 182
query9 8357 2321 2310 2310
query10 417 245 254 245
query11 14693 14175 14158 14158
query12 129 90 82 82
query13 1627 359 348 348
query14 8765 7901 7905 7901
query15 251 185 187 185
query16 8188 266 259 259
query17 1930 563 564 563
query18 2113 278 283 278
query19 270 151 157 151
query20 91 89 86 86
query21 197 130 129 129
query22 5049 4849 4832 4832
query23 33867 33137 33031 33031
query24 10443 3116 3132 3116
query25 607 402 411 402
query26 727 172 163 163
query27 2426 380 360 360
query28 6062 2073 2073 2073
query29 883 633 630 630
query30 317 180 189 180
query31 1002 790 781 781
query32 92 54 57 54
query33 668 262 257 257
query34 911 508 490 490
query35 860 736 712 712
query36 1109 963 941 941
query37 112 73 71 71
query38 3457 3330 3409 3330
query39 1611 1581 1595 1581
query40 177 125 133 125
query41 46 42 42 42
query42 105 98 95 95
query43 567 538 576 538
query44 1098 754 766 754
query45 276 270 276 270
query46 1098 775 727 727
query47 2031 1929 1950 1929
query48 382 312 314 312
query49 835 397 396 396
query50 786 395 429 395
query51 6923 6788 6688 6688
query52 101 85 83 83
query53 339 278 274 274
query54 323 234 232 232
query55 80 74 73 73
query56 256 225 234 225
query57 1169 1126 1127 1126
query58 227 195 204 195
query59 3424 3309 3090 3090
query60 253 235 229 229
query61 86 84 83 83
query62 623 429 439 429
query63 304 275 276 275
query64 4641 3932 3946 3932
query65 3099 3047 3056 3047
query66 750 322 335 322
query67 15571 15054 14950 14950
query68 7848 543 530 530
query69 550 301 309 301
query70 1284 1148 1150 1148
query71 1489 1270 1264 1264
query72 6598 2603 2455 2455
query73 745 319 317 317
query74 6763 6498 6460 6460
query75 4151 2615 2654 2615
query76 5114 987 991 987
query77 610 262 267 262
query78 10890 10225 10205 10205
query79 9863 537 514 514
query80 1743 425 424 424
query81 528 246 246 246
query82 867 93 95 93
query83 201 161 160 160
query84 269 82 80 80
query85 1183 299 251 251
query86 439 261 311 261
query87 3579 3299 3311 3299
query88 4867 2284 2274 2274
query89 542 382 365 365
query90 1939 182 177 177
query91 121 95 96 95
query92 54 44 45 44
query93 7158 496 498 496
query94 1058 179 177 177
query95 382 290 299 290
query96 615 254 256 254
query97 3130 2941 2922 2922
query98 233 214 220 214
query99 1245 849 882 849
Total cold run time: 294645 ms
Total hot run time: 185835 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2074061721
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2051370887
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2052838911
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2014834525
<details>
<summary>TPC-H: <b>Total hot run time: 38282 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 030bff9a47f12a5f3173b9e18608f7512acf19cf, data reload: false
------ Round 1 ----------------------------------
q1 17681 4309 4176 4176
q2 2114 161 156 156
q3 10670 1213 1225 1213
q4 10249 830 824 824
q5 7469 3053 3048 3048
q6 206 124 122 122
q7 1066 584 565 565
q8 9336 2039 1978 1978
q9 7644 6601 6587 6587
q10 8479 3428 3590 3428
q11 436 228 232 228
q12 417 203 202 202
q13 17798 2876 2872 2872
q14 242 202 225 202
q15 509 463 472 463
q16 518 378 375 375
q17 959 522 625 522
q18 7337 6538 6578 6538
q19 2673 1373 1449 1373
q20 561 277 256 256
q21 3559 2842 3014 2842
q22 373 312 320 312
Total cold run time: 110296 ms
Total hot run time: 38282 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4155 4089 4101 4089
q2 335 235 235 235
q3 2990 2836 2809 2809
q4 1829 1561 1527 1527
q5 5314 5360 5377 5360
q6 194 118 117 117
q7 2225 1851 1867 1851
q8 3164 3318 3317 3317
q9 8740 8738 8713 8713
q10 3845 3796 3793 3793
q11 562 456 434 434
q12 722 540 540 540
q13 16905 2888 2824 2824
q14 296 254 251 251
q15 515 461 454 454
q16 466 431 419 419
q17 1749 1525 1467 1467
q18 7420 7270 7183 7183
q19 1638 1578 1560 1560
q20 1950 1746 1725 1725
q21 4886 4796 4796 4796
q22 553 464 471 464
Total cold run time: 70453 ms
Total hot run time: 53928 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2014858044
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 030bff9a47f12a5f3173b9e18608f7512acf19cf with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 21.5 seconds inserted 10000000 Rows, about 465K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](nereids)project sub expression in other condition for nested loop join [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #32697:
URL: https://github.com/apache/doris/pull/32697#issuecomment-2068555269
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org