You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "cambyzju (via GitHub)" <gi...@apache.org> on 2024/07/05 14:12:30 UTC
[PR] [fix)(colocate join) fix wrong use of colocate join [doris]
cambyzju opened a new pull request, #37361:
URL: https://github.com/apache/doris/pull/37361
## Proposed changes
Issue Number: close https://github.com/apache/doris/issues/36324
<!--Describe your changes.-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2216251407
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #37361:
URL: https://github.com/apache/doris/pull/37361#discussion_r1668256427
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/util/JoinUtils.java:
##########
@@ -300,12 +301,42 @@ public static boolean couldColocateJoin(DistributionSpecHash leftHashSpec, Distr
boolean noNeedCheckColocateGroup = hitSameIndex && (leftTablePartitions.equals(rightTablePartitions))
&& (leftTablePartitions.size() <= 1);
ColocateTableIndex colocateIndex = Env.getCurrentColocateIndex();
- if (noNeedCheckColocateGroup
- || (colocateIndex.isSameGroup(leftTableId, rightTableId)
- && !colocateIndex.isGroupUnstable(colocateIndex.getGroup(leftTableId)))) {
+ if (noNeedCheckColocateGroup) {
return true;
}
- return false;
+ if (!colocateIndex.isSameGroup(leftTableId, rightTableId)
+ || colocateIndex.isGroupUnstable(colocateIndex.getGroup(leftTableId))) {
+ return false;
+ }
+
+ Set<Integer> equalIndices = new HashSet<>();
+ for (Expression expr : conjuncts) {
+ // only simple equal predicate can use colocate join
+ if (!(expr instanceof EqualPredicate)) {
+ return false;
+ }
+ Expression leftChild = ((EqualPredicate) expr).left();
+ Expression rightChild = ((EqualPredicate) expr).right();
+ if (!(leftChild instanceof Slot) || !(rightChild instanceof Slot)) {
+ return false;
+ }
+
+ // on conditions must keep same order as distributed columns
+ Integer leftIndex = leftHashSpec.getExprIdToEquivalenceSet().get(((Slot) leftChild).getExprId());
+ Integer rightIndex = rightHashSpec.getExprIdToEquivalenceSet().get(((Slot) rightChild).getExprId());
+ if (leftIndex != rightIndex) {
Review Comment:
use Objects.equals to avoid npe when leftIndex is null
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju commented on code in PR #37361:
URL: https://github.com/apache/doris/pull/37361#discussion_r1668534880
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/util/JoinUtils.java:
##########
@@ -300,12 +301,42 @@ public static boolean couldColocateJoin(DistributionSpecHash leftHashSpec, Distr
boolean noNeedCheckColocateGroup = hitSameIndex && (leftTablePartitions.equals(rightTablePartitions))
&& (leftTablePartitions.size() <= 1);
ColocateTableIndex colocateIndex = Env.getCurrentColocateIndex();
- if (noNeedCheckColocateGroup
- || (colocateIndex.isSameGroup(leftTableId, rightTableId)
- && !colocateIndex.isGroupUnstable(colocateIndex.getGroup(leftTableId)))) {
+ if (noNeedCheckColocateGroup) {
return true;
}
- return false;
+ if (!colocateIndex.isSameGroup(leftTableId, rightTableId)
+ || colocateIndex.isGroupUnstable(colocateIndex.getGroup(leftTableId))) {
+ return false;
+ }
+
+ Set<Integer> equalIndices = new HashSet<>();
+ for (Expression expr : conjuncts) {
+ // only simple equal predicate can use colocate join
+ if (!(expr instanceof EqualPredicate)) {
+ return false;
+ }
+ Expression leftChild = ((EqualPredicate) expr).left();
+ Expression rightChild = ((EqualPredicate) expr).right();
Review Comment:
Got it, maybe it is the reason why lots of regression tests of tcpds + tpch shape failed, I will try to fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2211046716
<details>
<summary>ClickBench: <b>Total hot run time: 30.5 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4fdf27de97b8661b00312f04825ec11c21b9c4aa, data reload: false
query1 0.04 0.03 0.03
query2 0.07 0.04 0.04
query3 0.23 0.05 0.05
query4 1.68 0.07 0.08
query5 0.51 0.50 0.48
query6 1.12 0.72 0.72
query7 0.02 0.01 0.01
query8 0.05 0.04 0.05
query9 0.54 0.48 0.50
query10 0.54 0.53 0.56
query11 0.16 0.11 0.12
query12 0.15 0.12 0.12
query13 0.59 0.59 0.59
query14 0.78 0.78 0.78
query15 0.84 0.81 0.80
query16 0.35 0.38 0.37
query17 0.95 0.94 1.05
query18 0.22 0.24 0.24
query19 1.89 1.79 1.75
query20 0.01 0.01 0.00
query21 15.40 0.73 0.66
query22 4.25 6.53 1.93
query23 18.27 1.26 1.21
query24 2.13 0.24 0.22
query25 0.15 0.08 0.08
query26 0.28 0.21 0.21
query27 0.45 0.23 0.23
query28 13.21 1.02 1.00
query29 12.64 3.30 3.27
query30 0.25 0.06 0.05
query31 2.85 0.39 0.39
query32 3.29 0.47 0.46
query33 2.87 2.91 2.94
query34 16.99 4.35 4.34
query35 4.42 4.41 4.37
query36 0.64 0.47 0.46
query37 0.18 0.15 0.15
query38 0.15 0.15 0.14
query39 0.04 0.03 0.04
query40 0.15 0.13 0.12
query41 0.08 0.05 0.04
query42 0.06 0.05 0.04
query43 0.04 0.04 0.04
Total cold run time: 109.53 s
Total hot run time: 30.5 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju commented on code in PR #37361:
URL: https://github.com/apache/doris/pull/37361#discussion_r1669547634
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/util/JoinUtils.java:
##########
@@ -300,12 +301,42 @@ public static boolean couldColocateJoin(DistributionSpecHash leftHashSpec, Distr
boolean noNeedCheckColocateGroup = hitSameIndex && (leftTablePartitions.equals(rightTablePartitions))
&& (leftTablePartitions.size() <= 1);
ColocateTableIndex colocateIndex = Env.getCurrentColocateIndex();
- if (noNeedCheckColocateGroup
- || (colocateIndex.isSameGroup(leftTableId, rightTableId)
- && !colocateIndex.isGroupUnstable(colocateIndex.getGroup(leftTableId)))) {
+ if (noNeedCheckColocateGroup) {
return true;
}
- return false;
+ if (!colocateIndex.isSameGroup(leftTableId, rightTableId)
+ || colocateIndex.isGroupUnstable(colocateIndex.getGroup(leftTableId))) {
+ return false;
+ }
+
+ Set<Integer> equalIndices = new HashSet<>();
+ for (Expression expr : conjuncts) {
+ // only simple equal predicate can use colocate join
+ if (!(expr instanceof EqualPredicate)) {
+ return false;
+ }
+ Expression leftChild = ((EqualPredicate) expr).left();
+ Expression rightChild = ((EqualPredicate) expr).right();
+ if (!(leftChild instanceof Slot) || !(rightChild instanceof Slot)) {
+ return false;
+ }
+
+ // on conditions must keep same order as distributed columns
+ Integer leftIndex = leftHashSpec.getExprIdToEquivalenceSet().get(((Slot) leftChild).getExprId());
+ Integer rightIndex = rightHashSpec.getExprIdToEquivalenceSet().get(((Slot) rightChild).getExprId());
+ if (leftIndex != rightIndex) {
Review Comment:
done
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/util/JoinUtils.java:
##########
@@ -300,12 +301,42 @@ public static boolean couldColocateJoin(DistributionSpecHash leftHashSpec, Distr
boolean noNeedCheckColocateGroup = hitSameIndex && (leftTablePartitions.equals(rightTablePartitions))
&& (leftTablePartitions.size() <= 1);
ColocateTableIndex colocateIndex = Env.getCurrentColocateIndex();
- if (noNeedCheckColocateGroup
- || (colocateIndex.isSameGroup(leftTableId, rightTableId)
- && !colocateIndex.isGroupUnstable(colocateIndex.getGroup(leftTableId)))) {
+ if (noNeedCheckColocateGroup) {
return true;
}
- return false;
+ if (!colocateIndex.isSameGroup(leftTableId, rightTableId)
+ || colocateIndex.isGroupUnstable(colocateIndex.getGroup(leftTableId))) {
+ return false;
+ }
+
+ Set<Integer> equalIndices = new HashSet<>();
+ for (Expression expr : conjuncts) {
+ // only simple equal predicate can use colocate join
+ if (!(expr instanceof EqualPredicate)) {
+ return false;
+ }
+ Expression leftChild = ((EqualPredicate) expr).left();
+ Expression rightChild = ((EqualPredicate) expr).right();
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2213436866
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2216359149
<details>
<summary>ClickBench: <b>Total hot run time: 30.76 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dc4dfe8086fa434223ac5fdeee17732c92eb75ef, data reload: false
query1 0.03 0.03 0.03
query2 0.07 0.04 0.04
query3 0.22 0.04 0.05
query4 1.68 0.08 0.07
query5 0.51 0.48 0.48
query6 1.14 0.73 0.73
query7 0.02 0.01 0.01
query8 0.06 0.04 0.05
query9 0.56 0.49 0.50
query10 0.55 0.53 0.55
query11 0.15 0.11 0.12
query12 0.15 0.13 0.12
query13 0.59 0.58 0.58
query14 0.77 0.77 0.77
query15 0.85 0.81 0.80
query16 0.35 0.36 0.38
query17 1.00 1.04 0.97
query18 0.22 0.21 0.22
query19 1.76 1.70 1.74
query20 0.01 0.01 0.02
query21 15.39 0.76 0.66
query22 4.24 6.82 2.17
query23 18.28 1.42 1.34
query24 2.13 0.22 0.22
query25 0.16 0.08 0.09
query26 0.29 0.20 0.21
query27 0.45 0.22 0.23
query28 13.27 1.03 1.01
query29 12.61 3.31 3.24
query30 0.25 0.06 0.05
query31 2.86 0.38 0.39
query32 3.29 0.49 0.48
query33 2.93 2.85 2.85
query34 17.23 4.36 4.32
query35 4.38 4.42 4.42
query36 0.64 0.47 0.48
query37 0.18 0.15 0.16
query38 0.16 0.14 0.14
query39 0.04 0.03 0.04
query40 0.15 0.12 0.13
query41 0.09 0.04 0.04
query42 0.06 0.04 0.04
query43 0.04 0.04 0.04
Total cold run time: 109.81 s
Total hot run time: 30.76 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #37361:
URL: https://github.com/apache/doris/pull/37361#discussion_r1668321261
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/util/JoinUtils.java:
##########
@@ -300,12 +301,42 @@ public static boolean couldColocateJoin(DistributionSpecHash leftHashSpec, Distr
boolean noNeedCheckColocateGroup = hitSameIndex && (leftTablePartitions.equals(rightTablePartitions))
&& (leftTablePartitions.size() <= 1);
ColocateTableIndex colocateIndex = Env.getCurrentColocateIndex();
- if (noNeedCheckColocateGroup
- || (colocateIndex.isSameGroup(leftTableId, rightTableId)
- && !colocateIndex.isGroupUnstable(colocateIndex.getGroup(leftTableId)))) {
+ if (noNeedCheckColocateGroup) {
return true;
}
- return false;
+ if (!colocateIndex.isSameGroup(leftTableId, rightTableId)
+ || colocateIndex.isGroupUnstable(colocateIndex.getGroup(leftTableId))) {
+ return false;
+ }
+
+ Set<Integer> equalIndices = new HashSet<>();
+ for (Expression expr : conjuncts) {
+ // only simple equal predicate can use colocate join
+ if (!(expr instanceof EqualPredicate)) {
+ return false;
+ }
+ Expression leftChild = ((EqualPredicate) expr).left();
+ Expression rightChild = ((EqualPredicate) expr).right();
Review Comment:
not always left for left child, right for right child
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2213386356
<details>
<summary>TPC-DS: <b>Total hot run time: 174178 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d6d6d906cb3cd5da073b1fb1ccd2f4fb796f8726, data reload: false
query1 919 370 354 354
query2 6193 2309 2240 2240
query3 6643 206 215 206
query4 28401 17488 17376 17376
query5 3927 485 479 479
query6 256 178 169 169
query7 4604 303 314 303
query8 325 301 295 295
query9 8501 2394 2376 2376
query10 579 300 306 300
query11 10642 9948 10058 9948
query12 137 88 83 83
query13 1654 376 382 376
query14 10173 7564 7514 7514
query15 235 187 186 186
query16 7742 298 311 298
query17 1820 548 541 541
query18 1773 285 276 276
query19 199 146 151 146
query20 92 80 81 80
query21 208 130 129 129
query22 4239 4096 4045 4045
query23 34017 33695 33881 33695
query24 10991 2775 2805 2775
query25 660 391 428 391
query26 1605 162 156 156
query27 3054 325 335 325
query28 7453 2169 2113 2113
query29 985 662 680 662
query30 255 158 151 151
query31 1007 752 768 752
query32 101 56 54 54
query33 752 301 309 301
query34 954 503 496 496
query35 748 676 644 644
query36 1153 974 960 960
query37 159 78 83 78
query38 2995 2838 2770 2770
query39 867 813 802 802
query40 268 125 130 125
query41 52 52 54 52
query42 122 98 100 98
query43 626 526 520 520
query44 1214 742 737 737
query45 198 159 163 159
query46 1082 728 719 719
query47 1863 1758 1746 1746
query48 389 306 302 302
query49 930 420 430 420
query50 759 381 385 381
query51 6860 6812 6779 6779
query52 107 89 94 89
query53 356 292 291 291
query54 833 461 455 455
query55 72 73 75 73
query56 298 266 269 266
query57 1181 1024 1021 1021
query58 284 257 247 247
query59 3475 3181 3107 3107
query60 302 284 283 283
query61 101 97 97 97
query62 595 440 438 438
query63 310 283 287 283
query64 10320 2163 7395 2163
query65 3147 3115 3127 3115
query66 1253 367 325 325
query67 15454 14937 15230 14937
query68 4516 538 549 538
query69 572 402 361 361
query70 1133 1107 1079 1079
query71 403 284 283 283
query72 6982 5299 5585 5299
query73 728 337 322 322
query74 6050 5511 5495 5495
query75 3495 2782 2796 2782
query76 2758 866 887 866
query77 435 307 324 307
query78 9505 9185 8938 8938
query79 2626 513 514 513
query80 2290 488 478 478
query81 597 221 214 214
query82 737 103 108 103
query83 301 178 172 172
query84 269 88 85 85
query85 2186 325 313 313
query86 487 314 311 311
query87 3342 3188 3101 3101
query88 3993 2465 2457 2457
query89 472 375 383 375
query90 1835 189 188 188
query91 135 102 101 101
query92 63 49 48 48
query93 1937 522 513 513
query94 1234 210 209 209
query95 399 318 310 310
query96 596 272 277 272
query97 3192 3023 3027 3023
query98 228 207 204 204
query99 1187 850 845 845
Total cold run time: 284152 ms
Total hot run time: 174178 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2210953891
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2212911362
<details>
<summary>TPC-DS: <b>Total hot run time: 174074 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a573581077a07713ae662fdcd6980bb5c2f22583, data reload: false
query1 906 382 360 360
query2 6431 2511 2375 2375
query3 6632 201 213 201
query4 28291 17723 17385 17385
query5 3647 505 470 470
query6 268 179 175 175
query7 4599 298 290 290
query8 307 295 286 286
query9 8624 2375 2354 2354
query10 585 296 277 277
query11 11664 10223 10214 10214
query12 121 88 82 82
query13 1645 387 376 376
query14 10154 7598 7695 7598
query15 275 181 185 181
query16 7912 314 315 314
query17 1795 562 547 547
query18 1980 295 282 282
query19 207 155 155 155
query20 92 85 84 84
query21 212 135 129 129
query22 4520 4132 3932 3932
query23 34256 33760 33693 33693
query24 10857 2901 2822 2822
query25 587 425 419 419
query26 721 163 159 159
query27 2246 330 330 330
query28 6066 2130 2125 2125
query29 903 658 686 658
query30 248 155 155 155
query31 972 806 769 769
query32 102 53 57 53
query33 689 315 312 312
query34 974 508 502 502
query35 765 644 627 627
query36 1109 971 941 941
query37 136 77 81 77
query38 2922 2802 2779 2779
query39 871 795 802 795
query40 214 133 126 126
query41 52 50 49 49
query42 124 97 108 97
query43 606 565 548 548
query44 1083 728 733 728
query45 203 155 157 155
query46 1074 722 734 722
query47 1842 1781 1765 1765
query48 379 311 294 294
query49 852 413 411 411
query50 791 392 409 392
query51 6875 6743 6781 6743
query52 101 94 91 91
query53 356 284 291 284
query54 904 450 450 450
query55 74 74 70 70
query56 282 270 262 262
query57 1114 1050 1059 1050
query58 242 241 240 240
query59 3699 3136 3241 3136
query60 325 273 288 273
query61 94 92 90 90
query62 590 437 450 437
query63 315 292 289 289
query64 9162 2234 1619 1619
query65 3131 3105 3114 3105
query66 730 327 323 323
query67 15771 14853 14916 14853
query68 5134 523 530 523
query69 739 434 351 351
query70 1139 1150 1092 1092
query71 508 281 276 276
query72 8425 5725 5418 5418
query73 772 328 328 328
query74 6023 5607 5494 5494
query75 5102 2743 2755 2743
query76 4153 993 962 962
query77 740 296 290 290
query78 9606 8941 8882 8882
query79 7207 514 518 514
query80 1303 484 476 476
query81 584 222 226 222
query82 1531 106 108 106
query83 456 183 160 160
query84 276 88 89 88
query85 1419 308 308 308
query86 457 316 324 316
query87 3301 3103 3133 3103
query88 4987 2467 2477 2467
query89 489 397 382 382
query90 1944 187 192 187
query91 132 101 110 101
query92 60 48 47 47
query93 6153 501 492 492
query94 1283 207 214 207
query95 403 318 316 316
query96 604 275 272 272
query97 3217 3024 3020 3020
query98 220 198 194 194
query99 1136 852 903 852
Total cold run time: 295779 ms
Total hot run time: 174074 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2212917798
<details>
<summary>ClickBench: <b>Total hot run time: 31.02 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a573581077a07713ae662fdcd6980bb5c2f22583, data reload: false
query1 0.04 0.04 0.04
query2 0.08 0.04 0.03
query3 0.23 0.05 0.05
query4 1.68 0.07 0.07
query5 0.51 0.48 0.49
query6 1.16 0.73 0.73
query7 0.02 0.01 0.01
query8 0.05 0.04 0.04
query9 0.55 0.51 0.49
query10 0.54 0.54 0.55
query11 0.15 0.11 0.12
query12 0.15 0.12 0.13
query13 0.59 0.58 0.59
query14 0.77 0.77 0.76
query15 0.84 0.82 0.82
query16 0.36 0.36 0.37
query17 0.97 1.00 1.02
query18 0.25 0.22 0.26
query19 1.80 1.69 1.70
query20 0.01 0.01 0.00
query21 15.42 0.75 0.67
query22 4.45 6.75 2.20
query23 18.30 1.38 1.28
query24 2.16 0.22 0.22
query25 0.15 0.09 0.08
query26 0.28 0.20 0.21
query27 0.44 0.23 0.23
query28 13.27 1.01 1.00
query29 12.61 3.33 3.31
query30 0.25 0.06 0.05
query31 2.87 0.39 0.39
query32 3.26 0.48 0.47
query33 2.84 3.00 2.91
query34 17.06 4.40 4.39
query35 4.40 4.47 4.46
query36 0.66 0.47 0.48
query37 0.18 0.15 0.16
query38 0.16 0.14 0.13
query39 0.04 0.03 0.04
query40 0.15 0.12 0.13
query41 0.09 0.04 0.05
query42 0.06 0.04 0.04
query43 0.05 0.04 0.04
Total cold run time: 109.9 s
Total hot run time: 31.02 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju merged PR #37361:
URL: https://github.com/apache/doris/pull/37361
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2216349259
<details>
<summary>TPC-DS: <b>Total hot run time: 173973 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dc4dfe8086fa434223ac5fdeee17732c92eb75ef, data reload: false
query1 924 379 370 370
query2 6440 2305 2248 2248
query3 6635 209 213 209
query4 28185 17502 17305 17305
query5 3659 483 481 481
query6 280 167 179 167
query7 4583 296 273 273
query8 296 276 287 276
query9 8486 2453 2426 2426
query10 571 298 282 282
query11 10860 10027 9978 9978
query12 119 91 86 86
query13 1660 378 371 371
query14 10109 7650 7744 7650
query15 245 195 189 189
query16 7766 322 311 311
query17 1814 572 545 545
query18 1840 297 288 288
query19 205 153 158 153
query20 100 83 85 83
query21 214 136 128 128
query22 4342 4104 3989 3989
query23 33886 33653 33628 33628
query24 10916 2923 2876 2876
query25 626 416 440 416
query26 708 160 152 152
query27 2168 278 273 273
query28 5841 2122 2112 2112
query29 916 648 661 648
query30 257 160 159 159
query31 983 782 788 782
query32 97 57 57 57
query33 801 328 305 305
query34 984 493 498 493
query35 730 581 595 581
query36 1102 986 944 944
query37 143 81 87 81
query38 2937 2803 2776 2776
query39 905 857 818 818
query40 198 121 117 117
query41 49 45 51 45
query42 114 99 97 97
query43 568 553 543 543
query44 1227 726 723 723
query45 194 155 158 155
query46 1075 715 701 701
query47 1868 1763 1774 1763
query48 374 294 304 294
query49 837 443 437 437
query50 773 388 400 388
query51 6800 6685 6748 6685
query52 109 92 93 92
query53 366 283 289 283
query54 944 448 431 431
query55 80 74 75 74
query56 297 262 262 262
query57 1127 1044 1041 1041
query58 252 253 261 253
query59 3364 3144 2999 2999
query60 308 280 296 280
query61 97 95 95 95
query62 799 653 658 653
query63 322 297 295 295
query64 9118 2216 1629 1629
query65 3133 3163 3119 3119
query66 711 328 330 328
query67 15791 15151 14926 14926
query68 7325 521 538 521
query69 772 439 358 358
query70 1179 1114 1064 1064
query71 489 290 272 272
query72 8968 5463 5488 5463
query73 840 322 317 317
query74 5996 5609 5503 5503
query75 4349 2666 2678 2666
query76 4524 942 894 894
query77 783 303 306 303
query78 9801 8990 8856 8856
query79 8493 510 524 510
query80 1117 467 479 467
query81 586 224 222 222
query82 748 141 129 129
query83 309 168 170 168
query84 278 85 88 85
query85 1092 328 369 328
query86 381 333 326 326
query87 3300 3109 3076 3076
query88 4900 2367 2351 2351
query89 547 382 378 378
query90 2004 193 190 190
query91 134 104 107 104
query92 62 48 50 48
query93 7056 496 493 493
query94 1252 209 206 206
query95 414 316 311 311
query96 617 271 261 261
query97 3215 2996 3003 2996
query98 212 197 197 197
query99 1556 1293 1301 1293
Total cold run time: 297527 ms
Total hot run time: 173973 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2212896420
<details>
<summary>TPC-H: <b>Total hot run time: 39757 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a573581077a07713ae662fdcd6980bb5c2f22583, data reload: false
------ Round 1 ----------------------------------
q1 17638 4365 4311 4311
q2 2041 200 199 199
q3 10432 1146 1174 1146
q4 10203 770 725 725
q5 7513 2665 2590 2590
q6 222 142 142 142
q7 958 594 601 594
q8 9230 2058 2066 2058
q9 8823 6509 6496 6496
q10 9011 3730 3742 3730
q11 460 258 245 245
q12 538 230 228 228
q13 17864 2998 3003 2998
q14 267 216 238 216
q15 531 481 503 481
q16 516 394 380 380
q17 952 708 618 618
q18 7939 7473 7445 7445
q19 5581 1506 1477 1477
q20 650 311 330 311
q21 5013 3034 3877 3034
q22 397 333 350 333
Total cold run time: 116779 ms
Total hot run time: 39757 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4430 4256 4244 4244
q2 378 270 269 269
q3 2961 2846 2872 2846
q4 2003 1745 1707 1707
q5 5670 5482 5467 5467
q6 229 130 131 130
q7 2152 1873 1926 1873
q8 3232 3364 3370 3364
q9 8719 8682 8870 8682
q10 4064 3868 3691 3691
q11 591 509 497 497
q12 795 653 654 653
q13 16165 3192 3159 3159
q14 314 289 263 263
q15 541 500 493 493
q16 528 461 448 448
q17 1834 1540 1502 1502
q18 7989 7928 7935 7928
q19 1777 1707 1641 1641
q20 2146 1860 1849 1849
q21 5263 5040 4785 4785
q22 611 554 587 554
Total cold run time: 72392 ms
Total hot run time: 56045 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2210983473
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2216334446
<details>
<summary>TPC-H: <b>Total hot run time: 40036 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dc4dfe8086fa434223ac5fdeee17732c92eb75ef, data reload: false
------ Round 1 ----------------------------------
q1 17598 4386 4263 4263
q2 2019 191 193 191
q3 10467 1200 1101 1101
q4 10203 841 723 723
q5 7503 2699 2719 2699
q6 224 139 137 137
q7 977 603 613 603
q8 9230 2087 2091 2087
q9 8930 6517 6505 6505
q10 9019 3807 3730 3730
q11 469 248 249 248
q12 484 233 243 233
q13 18763 2965 3006 2965
q14 271 227 234 227
q15 521 494 488 488
q16 524 383 380 380
q17 969 691 763 691
q18 8024 7503 7440 7440
q19 6636 1516 1473 1473
q20 669 339 354 339
q21 5022 3269 3176 3176
q22 398 337 337 337
Total cold run time: 118920 ms
Total hot run time: 40036 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4375 4209 4248 4209
q2 361 268 268 268
q3 3047 2901 2901 2901
q4 2016 1720 1744 1720
q5 5580 5579 5469 5469
q6 221 134 133 133
q7 2234 1884 1852 1852
q8 3342 3463 3422 3422
q9 8799 8791 8803 8791
q10 4128 3783 3886 3783
q11 597 518 501 501
q12 825 660 626 626
q13 17102 3173 3213 3173
q14 317 280 275 275
q15 532 471 492 471
q16 504 438 426 426
q17 1822 1483 1522 1483
q18 8197 7897 7833 7833
q19 1871 1559 1580 1559
q20 2987 1868 1865 1865
q21 5089 4883 4835 4835
q22 646 547 541 541
Total cold run time: 74592 ms
Total hot run time: 56136 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2212856277
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2211038843
<details>
<summary>TPC-DS: <b>Total hot run time: 172986 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4fdf27de97b8661b00312f04825ec11c21b9c4aa, data reload: false
query1 929 379 373 373
query2 6456 2422 2286 2286
query3 6644 207 226 207
query4 23497 17407 17313 17313
query5 4191 500 494 494
query6 319 187 146 146
query7 4590 289 285 285
query8 335 280 284 280
query9 8434 2399 2404 2399
query10 602 297 278 278
query11 10686 10089 10181 10089
query12 133 92 85 85
query13 1628 389 385 385
query14 9334 7768 7585 7585
query15 253 183 185 183
query16 7593 309 300 300
query17 1825 548 521 521
query18 1875 282 281 281
query19 193 145 152 145
query20 89 79 82 79
query21 209 137 127 127
query22 4483 4408 4317 4317
query23 33634 33155 33119 33119
query24 11035 2898 2822 2822
query25 634 364 364 364
query26 1390 154 154 154
query27 2920 310 317 310
query28 7342 2060 2050 2050
query29 931 628 592 592
query30 283 147 145 145
query31 986 721 755 721
query32 93 57 57 57
query33 762 283 283 283
query34 987 488 486 486
query35 715 610 607 607
query36 1138 898 925 898
query37 151 79 75 75
query38 2894 2732 2779 2732
query39 865 784 797 784
query40 293 131 138 131
query41 59 59 53 53
query42 119 97 103 97
query43 571 540 514 514
query44 1185 728 737 728
query45 197 172 165 165
query46 1077 731 698 698
query47 1846 1805 1750 1750
query48 373 294 304 294
query49 1155 410 425 410
query50 772 386 388 386
query51 6998 6853 6769 6769
query52 99 98 92 92
query53 362 290 288 288
query54 1036 451 451 451
query55 74 75 73 73
query56 312 292 289 289
query57 1127 1052 1045 1045
query58 283 252 255 252
query59 3357 3106 3263 3106
query60 337 296 296 296
query61 117 118 115 115
query62 664 425 452 425
query63 325 293 301 293
query64 10691 2247 1702 1702
query65 3188 3095 3088 3088
query66 1404 352 342 342
query67 15606 15000 15016 15000
query68 6287 550 540 540
query69 714 432 350 350
query70 1187 1053 1155 1053
query71 460 277 274 274
query72 8197 5175 5707 5175
query73 762 319 316 316
query74 5883 5710 5528 5528
query75 4684 2799 2731 2731
query76 3820 957 860 860
query77 699 299 310 299
query78 9823 9005 9001 9001
query79 2935 518 508 508
query80 1379 476 488 476
query81 579 221 224 221
query82 1358 110 106 106
query83 301 167 169 167
query84 273 95 84 84
query85 1546 333 346 333
query86 471 322 304 304
query87 3328 3076 3105 3076
query88 4347 2339 2353 2339
query89 492 386 379 379
query90 1898 187 187 187
query91 135 100 100 100
query92 67 49 51 49
query93 3935 509 501 501
query94 1259 206 209 206
query95 401 311 301 301
query96 605 264 265 264
query97 3231 3069 3038 3038
query98 226 202 196 196
query99 1188 815 866 815
Total cold run time: 287155 ms
Total hot run time: 172986 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju commented on code in PR #37361:
URL: https://github.com/apache/doris/pull/37361#discussion_r1668531439
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/util/JoinUtils.java:
##########
@@ -300,12 +301,42 @@ public static boolean couldColocateJoin(DistributionSpecHash leftHashSpec, Distr
boolean noNeedCheckColocateGroup = hitSameIndex && (leftTablePartitions.equals(rightTablePartitions))
&& (leftTablePartitions.size() <= 1);
ColocateTableIndex colocateIndex = Env.getCurrentColocateIndex();
- if (noNeedCheckColocateGroup
- || (colocateIndex.isSameGroup(leftTableId, rightTableId)
- && !colocateIndex.isGroupUnstable(colocateIndex.getGroup(leftTableId)))) {
+ if (noNeedCheckColocateGroup) {
return true;
}
- return false;
+ if (!colocateIndex.isSameGroup(leftTableId, rightTableId)
+ || colocateIndex.isGroupUnstable(colocateIndex.getGroup(leftTableId))) {
+ return false;
+ }
+
+ Set<Integer> equalIndices = new HashSet<>();
+ for (Expression expr : conjuncts) {
+ // only simple equal predicate can use colocate join
+ if (!(expr instanceof EqualPredicate)) {
+ return false;
+ }
+ Expression leftChild = ((EqualPredicate) expr).left();
+ Expression rightChild = ((EqualPredicate) expr).right();
+ if (!(leftChild instanceof Slot) || !(rightChild instanceof Slot)) {
+ return false;
+ }
+
+ // on conditions must keep same order as distributed columns
+ Integer leftIndex = leftHashSpec.getExprIdToEquivalenceSet().get(((Slot) leftChild).getExprId());
+ Integer rightIndex = rightHashSpec.getExprIdToEquivalenceSet().get(((Slot) rightChild).getExprId());
+ if (leftIndex != rightIndex) {
Review Comment:
Got, I will fix it later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "cambyzju (via GitHub)" <gi...@apache.org>.
cambyzju commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2213319778
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2213362711
<details>
<summary>TPC-H: <b>Total hot run time: 40107 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d6d6d906cb3cd5da073b1fb1ccd2f4fb796f8726, data reload: false
------ Round 1 ----------------------------------
q1 17639 4335 4368 4335
q2 2016 200 189 189
q3 10567 1196 1167 1167
q4 10726 695 767 695
q5 7595 2695 2659 2659
q6 226 138 139 138
q7 973 615 602 602
q8 9610 2131 2122 2122
q9 8965 6636 6643 6636
q10 9135 3690 3707 3690
q11 469 241 245 241
q12 499 245 225 225
q13 17804 2972 3014 2972
q14 276 233 213 213
q15 536 491 490 490
q16 546 398 386 386
q17 983 681 682 681
q18 8095 7444 7391 7391
q19 3909 1490 1497 1490
q20 686 328 328 328
q21 5081 3125 3312 3125
q22 398 332 343 332
Total cold run time: 116734 ms
Total hot run time: 40107 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4395 4303 4315 4303
q2 381 281 276 276
q3 2966 2859 2858 2858
q4 1988 1694 1724 1694
q5 5495 5474 5534 5474
q6 235 135 135 135
q7 2270 1916 1873 1873
q8 3252 3385 3401 3385
q9 8609 8682 8709 8682
q10 4052 3887 3908 3887
q11 605 498 480 480
q12 824 652 642 642
q13 17850 3139 3236 3139
q14 309 288 281 281
q15 544 500 491 491
q16 507 476 442 442
q17 1842 1497 1499 1497
q18 8264 8064 7843 7843
q19 3989 1563 1554 1554
q20 2153 1861 1808 1808
q21 5144 4943 4866 4866
q22 635 555 554 554
Total cold run time: 76309 ms
Total hot run time: 56164 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2213407348
<details>
<summary>ClickBench: <b>Total hot run time: 30.64 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d6d6d906cb3cd5da073b1fb1ccd2f4fb796f8726, data reload: false
query1 0.04 0.03 0.03
query2 0.07 0.04 0.04
query3 0.22 0.05 0.05
query4 1.66 0.09 0.10
query5 0.51 0.49 0.50
query6 1.15 0.74 0.72
query7 0.02 0.01 0.02
query8 0.05 0.05 0.05
query9 0.55 0.49 0.47
query10 0.54 0.53 0.56
query11 0.14 0.11 0.12
query12 0.14 0.12 0.13
query13 0.60 0.58 0.58
query14 0.76 0.78 0.78
query15 0.83 0.82 0.81
query16 0.36 0.35 0.37
query17 0.97 1.01 0.95
query18 0.21 0.28 0.22
query19 1.82 1.66 1.78
query20 0.03 0.01 0.00
query21 15.41 0.77 0.65
query22 3.95 6.86 2.11
query23 18.31 1.37 1.19
query24 2.06 0.24 0.23
query25 0.16 0.09 0.09
query26 0.29 0.21 0.21
query27 0.45 0.22 0.22
query28 13.23 1.02 1.00
query29 12.70 3.29 3.27
query30 0.25 0.06 0.05
query31 2.86 0.39 0.38
query32 3.29 0.47 0.46
query33 2.85 2.96 2.91
query34 17.14 4.33 4.34
query35 4.43 4.45 4.45
query36 0.65 0.46 0.49
query37 0.18 0.15 0.16
query38 0.15 0.14 0.14
query39 0.05 0.04 0.04
query40 0.15 0.12 0.12
query41 0.09 0.05 0.04
query42 0.06 0.05 0.06
query43 0.04 0.04 0.03
Total cold run time: 109.42 s
Total hot run time: 30.64 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2213436788
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2213436850
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix)(colocate join) fix wrong use of colocate join [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37361:
URL: https://github.com/apache/doris/pull/37361#issuecomment-2211020832
<details>
<summary>TPC-H: <b>Total hot run time: 39824 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4fdf27de97b8661b00312f04825ec11c21b9c4aa, data reload: false
------ Round 1 ----------------------------------
q1 17627 4538 4309 4309
q2 2039 198 194 194
q3 10521 1271 1106 1106
q4 11227 721 774 721
q5 7614 2650 2672 2650
q6 228 142 141 141
q7 966 618 608 608
q8 9399 2118 2097 2097
q9 9011 6500 6534 6500
q10 8841 3737 3721 3721
q11 460 234 240 234
q12 555 243 227 227
q13 17772 2961 2982 2961
q14 278 224 214 214
q15 530 479 495 479
q16 528 393 382 382
q17 953 637 740 637
q18 8089 7444 7526 7444
q19 7093 1570 1363 1363
q20 698 309 325 309
q21 5015 3277 3203 3203
q22 380 324 336 324
Total cold run time: 119824 ms
Total hot run time: 39824 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4394 4290 4263 4263
q2 389 281 280 280
q3 3031 2754 2702 2702
q4 1857 1582 1590 1582
q5 5346 5259 5287 5259
q6 218 134 129 129
q7 2113 1730 1749 1730
q8 3170 3410 3280 3280
q9 8322 8317 8344 8317
q10 3872 3724 3618 3618
q11 577 492 485 485
q12 801 639 599 599
q13 16843 3023 2959 2959
q14 288 250 253 250
q15 514 485 484 484
q16 466 420 436 420
q17 1778 1500 1539 1500
q18 7688 7558 7476 7476
q19 1743 1581 1464 1464
q20 1986 1808 1756 1756
q21 9679 4763 4745 4745
q22 624 553 534 534
Total cold run time: 75699 ms
Total hot run time: 53832 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org