You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "XieJiann (via GitHub)" <gi...@apache.org> on 2023/12/13 09:33:15 UTC
[PR] [feature](Nereids): put or expansion rule in cost base rewrite job [doris]
XieJiann opened a new pull request, #28348:
URL: https://github.com/apache/doris/pull/28348
## Proposed changes
put or expansion rule in cost base rewrite job
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1854261607
run builall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1859440888
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): add is null in predicate and put or expansion rule in rewriter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1859457660
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit ae77f54a9adf769fc962fd3afca689667b9e8cc5, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4683 4481 4477 4477
q2 365 151 156 151
q3 1460 1257 1280 1257
q4 1116 885 849 849
q5 3147 3128 3165 3128
q6 245 129 130 129
q7 987 491 486 486
q8 2231 2201 2209 2201
q9 6688 6680 6842 6680
q10 3221 3288 3246 3246
q11 327 202 200 200
q12 347 214 207 207
q13 4567 3842 3802 3802
q14 243 212 210 210
q15 568 526 518 518
q16 445 390 381 381
q17 1014 583 511 511
q18 7182 6932 7119 6932
q19 1534 1446 1388 1388
q20 562 317 304 304
q21 3091 2630 2696 2630
q22 342 283 282 282
Total cold run time: 44365 ms
Total hot run time: 39969 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4396 4390 4370 4370
q2 270 166 173 166
q3 3537 3533 3529 3529
q4 2383 2367 2364 2364
q5 5737 5731 5722 5722
q6 241 122 123 122
q7 2372 1899 1860 1860
q8 3526 3540 3519 3519
q9 8948 8969 8933 8933
q10 3916 4018 3983 3983
q11 496 385 392 385
q12 767 606 602 602
q13 4274 3579 3594 3579
q14 290 257 258 257
q15 574 525 532 525
q16 501 468 463 463
q17 1886 1869 1854 1854
q18 8756 8145 8258 8145
q19 1724 1724 1763 1724
q20 2262 1949 1936 1936
q21 6517 6203 6181 6181
q22 509 420 413 413
Total cold run time: 63882 ms
Total hot run time: 60632 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on code in PR #28348:
URL: https://github.com/apache/doris/pull/28348#discussion_r1427496409
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/OrExpansion.java:
##########
@@ -72,7 +72,7 @@ public class OrExpansion extends OneExplorationRuleFactory {
@Override
public Rule build() {
- return logicalJoin().when(JoinUtils::shouldNestedLoopJoin)
+ return logicalJoin(any(), any()).when(JoinUtils::shouldNestedLoopJoin)
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1853784571
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.69 seconds
stream load tsv: 591 seconds loaded 74807831229 Bytes, about 120 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.6 seconds inserted 10000000 Rows, about 337K ops/s
storage size: 17224213461 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1855188919
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.54 seconds
stream load tsv: 592 seconds loaded 74807831229 Bytes, about 120 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size: 17221251426 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1855086101
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit d86bae96b450d5b9ada18e1c1dfa03b2915b12c6, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4779 4468 4460 4460
q2 371 152 159 152
q3 1491 1233 1237 1233
q4 1119 932 864 864
q5 3215 3212 3199 3199
q6 246 128 128 128
q7 1002 496 480 480
q8 2224 2225 2216 2216
q9 6760 6703 6725 6703
q10 3273 3283 3264 3264
q11 324 211 198 198
q12 362 206 216 206
q13 4576 3841 3775 3775
q14 243 212 213 212
q15 576 531 540 531
q16 442 382 391 382
q17 1027 641 607 607
q18 7076 6690 6681 6681
q19 1568 1510 1253 1253
q20 524 306 283 283
q21 3073 2665 2671 2665
q22 356 278 285 278
Total cold run time: 44627 ms
Total hot run time: 39770 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4422 4392 4423 4392
q2 269 164 175 164
q3 3523 3509 3499 3499
q4 2422 2404 2407 2404
q5 5746 5760 5762 5760
q6 237 121 123 121
q7 2356 1830 1845 1830
q8 3540 3546 3550 3546
q9 9108 9068 9102 9068
q10 3920 4009 4027 4009
q11 514 381 390 381
q12 771 607 608 607
q13 4352 3551 3547 3547
q14 288 253 252 252
q15 578 524 528 524
q16 517 481 474 474
q17 1917 1867 1864 1864
q18 8542 8176 7977 7977
q19 1768 1754 1766 1754
q20 2246 1945 1954 1945
q21 6568 6189 6154 6154
q22 504 422 419 419
Total cold run time: 64108 ms
Total hot run time: 60691 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1857187402
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): add is null in predicate and put or expansion rule in rewriter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1859641887
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on code in PR #28348:
URL: https://github.com/apache/doris/pull/28348#discussion_r1427480947
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/Rewriter.java:
##########
@@ -405,6 +406,8 @@ private static List<RewriteJob> getWholeTreeRewriteJobs(List<RewriteJob> jobs) {
topic("rewrite cte sub-tree",
custom(RuleType.REWRITE_CTE_CHILDREN, () -> new RewriteCteChildren(jobs))
),
+ topic("or expansion",
Review Comment:
Because `or expansion` could generate new CTE, it may break the assumption that the CTE anchor always is at the top of the plam
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/Rewriter.java:
##########
@@ -405,6 +406,8 @@ private static List<RewriteJob> getWholeTreeRewriteJobs(List<RewriteJob> jobs) {
topic("rewrite cte sub-tree",
custom(RuleType.REWRITE_CTE_CHILDREN, () -> new RewriteCteChildren(jobs))
),
+ topic("or expansion",
Review Comment:
Because `or expansion` could generate new CTE, it may break the assumption that the CTE anchor always is at the top of the plan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on code in PR #28348:
URL: https://github.com/apache/doris/pull/28348#discussion_r1427480947
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/Rewriter.java:
##########
@@ -405,6 +406,8 @@ private static List<RewriteJob> getWholeTreeRewriteJobs(List<RewriteJob> jobs) {
topic("rewrite cte sub-tree",
custom(RuleType.REWRITE_CTE_CHILDREN, () -> new RewriteCteChildren(jobs))
),
+ topic("or expansion",
Review Comment:
Because `or expansion` could generate new CTE, it may break the assumption that the CTE anchor always is at the top of the plan in the rewritten CTE children
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1857516949
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1853718627
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "xzj7019 (via GitHub)" <gi...@apache.org>.
xzj7019 commented on code in PR #28348:
URL: https://github.com/apache/doris/pull/28348#discussion_r1426502206
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/executor/Rewriter.java:
##########
@@ -405,6 +406,8 @@ private static List<RewriteJob> getWholeTreeRewriteJobs(List<RewriteJob> jobs) {
topic("rewrite cte sub-tree",
custom(RuleType.REWRITE_CTE_CHILDREN, () -> new RewriteCteChildren(jobs))
),
+ topic("or expansion",
Review Comment:
what is the reason to put it here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): add is null in predicate and put or expansion rule in rewriter [doris]
Posted by "jackwener (via GitHub)" <gi...@apache.org>.
jackwener merged PR #28348:
URL: https://github.com/apache/doris/pull/28348
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1854261760
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1854400195
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.77 seconds
stream load tsv: 590 seconds loaded 74807831229 Bytes, about 120 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 28.2 seconds inserted 10000000 Rows, about 354K ops/s
storage size: 17218137664 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1854375464
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 89ce4dec12b6e2d24ca57a5af2aaab35f77ee1f5, data reload: false
run tpch-sf100 query with default conf and session variables
q1 50237 31876 31858 31858
q2 374 135 148 135
q3 1488 1279 1198 1198
q4 1116 912 905 905
q5 3165 3195 3210 3195
q6 260 138 152 138
q7 989 500 480 480
q8 2140 2189 2179 2179
q9 6959 6843 6886 6843
q10 3246 3299 3308 3299
q11 344 223 227 223
q12 350 208 210 208
q13 4606 4371 3842 3842
q14 424 317 317 317
q15 788 1314 1347 1314
q16 438 392 388 388
q17 1022 615 521 521
q18 7381 7012 6894 6894
q19 1548 1452 1420 1420
q20 1810 326 400 326
q21 3076 2610 2650 2610
q22 347 282 281 281
Total cold run time: 92108 ms
Total hot run time: 68574 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 30837 31869 31830 31830
q2 279 165 170 165
q3 3607 3627 3598 3598
q4 2384 2372 2368 2368
q5 5741 5750 5755 5750
q6 240 130 135 130
q7 3033 3404 3373 3373
q8 3540 3530 3544 3530
q9 9175 9146 9187 9146
q10 3952 4052 4091 4052
q11 807 917 1375 917
q12 784 626 604 604
q13 4302 3541 3554 3541
q14 521 442 434 434
q15 1108 1346 1344 1344
q16 496 452 457 452
q17 1890 1877 1831 1831
q18 8717 8145 8167 8145
q19 1758 1797 1771 1771
q20 2892 2622 2704 2622
q21 6511 6164 6145 6145
q22 508 419 415 415
Total cold run time: 93082 ms
Total hot run time: 92163 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1857646461
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): add is null in predicate and put or expansion rule in rewriter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1859483058
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.3 seconds
stream load tsv: 585 seconds loaded 74807831229 Bytes, about 121 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.1 seconds inserted 10000000 Rows, about 355K ops/s
storage size: 17219678315 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1855178907
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit d1110b9960b9403134fb2ea7e3631073c3161fa9, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4723 4478 4492 4478
q2 378 157 163 157
q3 1463 1266 1238 1238
q4 1135 910 859 859
q5 3197 3210 3202 3202
q6 246 125 125 125
q7 1027 489 477 477
q8 2181 2222 2195 2195
q9 6704 6656 6622 6622
q10 3232 3280 3268 3268
q11 320 197 201 197
q12 348 220 203 203
q13 4539 3831 3752 3752
q14 242 220 213 213
q15 577 527 517 517
q16 444 386 386 386
q17 1012 574 567 567
q18 7091 6704 6896 6704
q19 1536 1455 1362 1362
q20 550 327 308 308
q21 3044 2654 2665 2654
q22 348 278 286 278
Total cold run time: 44337 ms
Total hot run time: 39762 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4407 4419 4385 4385
q2 268 162 169 162
q3 3507 3493 3491 3491
q4 2403 2375 2399 2375
q5 5754 5741 5764 5741
q6 233 123 122 122
q7 2364 1834 1842 1834
q8 3536 3520 3548 3520
q9 9039 9001 8993 8993
q10 3930 4004 3992 3992
q11 505 375 380 375
q12 758 590 588 588
q13 4272 3590 3582 3582
q14 294 261 260 260
q15 568 515 514 514
q16 528 476 458 458
q17 1895 1860 1852 1852
q18 8491 8095 8166 8095
q19 1749 1750 1739 1739
q20 2241 1945 1935 1935
q21 6495 6116 6142 6116
q22 508 426 422 422
Total cold run time: 63745 ms
Total hot run time: 60551 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): add is null in predicate and put or expansion rule in rewriter [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1859641821
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1855056492
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1857697814
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.52 seconds
stream load tsv: 585 seconds loaded 74807831229 Bytes, about 121 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17219582542 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1855057768
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): add is null in predicate and put or expansion rule in rewriter [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1859478716
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit ae77f54a9adf769fc962fd3afca689667b9e8cc5, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4723 4467 4528 4467
q2 367 149 158 149
q3 1461 1215 1246 1215
q4 1109 939 881 881
q5 3165 3168 3150 3150
q6 250 127 129 127
q7 1016 486 500 486
q8 2198 2196 2180 2180
q9 6688 6705 6680 6680
q10 3227 3278 3260 3260
q11 328 211 204 204
q12 352 213 208 208
q13 4559 3770 3818 3770
q14 244 215 218 215
q15 576 527 521 521
q16 443 389 385 385
q17 1022 628 571 571
q18 7210 6888 6983 6888
q19 1507 1452 1386 1386
q20 547 291 314 291
q21 3096 2607 2666 2607
q22 353 278 283 278
Total cold run time: 44441 ms
Total hot run time: 39919 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4431 4412 4394 4394
q2 266 164 168 164
q3 3541 3525 3525 3525
q4 2390 2371 2370 2370
q5 5760 5742 5749 5742
q6 243 123 121 121
q7 2394 1855 1887 1855
q8 3524 3523 3522 3522
q9 9023 8934 8996 8934
q10 3938 3979 4005 3979
q11 499 387 372 372
q12 777 604 587 587
q13 4290 3581 3535 3535
q14 288 256 268 256
q15 572 524 515 515
q16 506 464 475 464
q17 1870 1848 1864 1848
q18 8668 8316 8373 8316
q19 1746 1744 1744 1744
q20 2278 1962 1947 1947
q21 6494 6166 6157 6157
q22 509 425 426 425
Total cold run time: 64007 ms
Total hot run time: 60772 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #28348:
URL: https://github.com/apache/doris/pull/28348#issuecomment-1855141570
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feature](Nereids): put or expansion rule in rewriter [doris]
Posted by "xzj7019 (via GitHub)" <gi...@apache.org>.
xzj7019 commented on code in PR #28348:
URL: https://github.com/apache/doris/pull/28348#discussion_r1426494587
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/OrExpansion.java:
##########
@@ -72,7 +72,7 @@ public class OrExpansion extends OneExplorationRuleFactory {
@Override
public Rule build() {
- return logicalJoin().when(JoinUtils::shouldNestedLoopJoin)
+ return logicalJoin(any(), any()).when(JoinUtils::shouldNestedLoopJoin)
Review Comment:
better to move this file from exploration to rewriter dir?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org