You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "XieJiann (via GitHub)" <gi...@apache.org> on 2024/07/05 08:04:40 UTC
[PR] [fix](Nereids): Enhance LogicalJoin input validation when push dow agg [doris]
XieJiann opened a new pull request, #37343:
URL: https://github.com/apache/doris/pull/37343
## Proposed changes
This PR refines the LogicalJoin class by introducing robust input validation. Key improvements:
* Implement precise checks for join input validity
* Ensure consistency between input slots and output sets
* Gracefully handle various join scenarios (left/right)
These enhancements bolster query integrity and optimize join operations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](Nereids): Enhance LogicalJoin input validation when push dow agg [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37343:
URL: https://github.com/apache/doris/pull/37343#issuecomment-2210538203
<details>
<summary>ClickBench: <b>Total hot run time: 30.46 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f73f8ff7bc0ebaa8554769a83f25c9860b88f0bc, data reload: false
query1 0.04 0.03 0.04
query2 0.07 0.04 0.04
query3 0.22 0.04 0.05
query4 1.68 0.07 0.09
query5 0.51 0.49 0.48
query6 1.13 0.72 0.73
query7 0.02 0.02 0.02
query8 0.05 0.04 0.05
query9 0.56 0.47 0.50
query10 0.53 0.54 0.53
query11 0.15 0.11 0.12
query12 0.15 0.12 0.12
query13 0.61 0.59 0.58
query14 0.75 0.78 0.78
query15 0.85 0.80 0.82
query16 0.37 0.35 0.37
query17 0.95 0.97 0.98
query18 0.22 0.25 0.26
query19 1.85 1.67 1.77
query20 0.01 0.01 0.01
query21 15.42 0.76 0.65
query22 4.12 7.84 1.91
query23 18.28 1.31 1.19
query24 1.92 0.27 0.23
query25 0.15 0.09 0.08
query26 0.30 0.21 0.21
query27 0.45 0.22 0.22
query28 13.24 1.02 0.99
query29 12.56 3.30 3.29
query30 0.25 0.06 0.05
query31 2.87 0.40 0.39
query32 3.26 0.48 0.47
query33 3.00 2.89 2.87
query34 17.48 4.41 4.31
query35 4.46 4.49 4.49
query36 0.65 0.46 0.47
query37 0.18 0.15 0.15
query38 0.15 0.14 0.14
query39 0.05 0.04 0.04
query40 0.15 0.13 0.12
query41 0.09 0.05 0.04
query42 0.06 0.05 0.04
query43 0.04 0.04 0.04
Total cold run time: 109.85 s
Total hot run time: 30.46 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](Nereids): Enhance LogicalJoin input validation when push dow agg [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37343:
URL: https://github.com/apache/doris/pull/37343#issuecomment-2210510820
<details>
<summary>TPC-H: <b>Total hot run time: 40076 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f73f8ff7bc0ebaa8554769a83f25c9860b88f0bc, data reload: false
------ Round 1 ----------------------------------
q1 17597 4412 4274 4274
q2 2020 214 187 187
q3 10453 1209 1254 1209
q4 10202 812 784 784
q5 7508 2713 2678 2678
q6 216 136 136 136
q7 954 596 611 596
q8 9243 2111 2085 2085
q9 9048 6503 6476 6476
q10 9044 3749 3728 3728
q11 449 232 235 232
q12 463 224 225 224
q13 17780 2954 2987 2954
q14 269 219 219 219
q15 534 478 490 478
q16 499 376 365 365
q17 969 690 727 690
q18 8050 7564 7467 7467
q19 4220 1497 1552 1497
q20 654 328 316 316
q21 4901 3144 3911 3144
q22 374 339 337 337
Total cold run time: 115447 ms
Total hot run time: 40076 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4356 4223 4220 4220
q2 361 260 270 260
q3 2978 2839 2883 2839
q4 1989 1728 1698 1698
q5 5744 5549 5490 5490
q6 238 132 131 131
q7 2187 1893 1895 1893
q8 3304 3469 3400 3400
q9 8769 8661 8811 8661
q10 4088 3919 3692 3692
q11 573 509 501 501
q12 808 663 668 663
q13 17267 3183 3162 3162
q14 324 269 281 269
q15 524 490 477 477
q16 481 443 449 443
q17 1812 1611 1519 1519
q18 8136 7944 7824 7824
q19 1788 1623 1645 1623
q20 2190 1842 1858 1842
q21 5275 4927 4896 4896
q22 602 547 575 547
Total cold run time: 73794 ms
Total hot run time: 56050 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](Nereids): Enhance LogicalJoin input validation when push down agg [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann merged PR #37343:
URL: https://github.com/apache/doris/pull/37343
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](Nereids): Enhance LogicalJoin input validation when push dow agg [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37343:
URL: https://github.com/apache/doris/pull/37343#issuecomment-2210842878
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](Nereids): Enhance LogicalJoin input validation when push dow agg [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #37343:
URL: https://github.com/apache/doris/pull/37343#issuecomment-2210842943
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](Nereids): Enhance LogicalJoin input validation when push dow agg [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37343:
URL: https://github.com/apache/doris/pull/37343#issuecomment-2210398268
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](Nereids): Enhance LogicalJoin input validation when push dow agg [doris]
Posted by "XieJiann (via GitHub)" <gi...@apache.org>.
XieJiann commented on PR #37343:
URL: https://github.com/apache/doris/pull/37343#issuecomment-2210398475
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](Nereids): Enhance LogicalJoin input validation when push dow agg [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #37343:
URL: https://github.com/apache/doris/pull/37343#issuecomment-2210528963
<details>
<summary>TPC-DS: <b>Total hot run time: 172690 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f73f8ff7bc0ebaa8554769a83f25c9860b88f0bc, data reload: false
query1 909 367 367 367
query2 6463 2290 2300 2290
query3 6627 213 221 213
query4 28437 17272 17501 17272
query5 3678 491 505 491
query6 260 171 165 165
query7 4581 296 299 296
query8 322 277 272 272
query9 8534 2421 2400 2400
query10 595 290 289 289
query11 10688 9942 10084 9942
query12 120 89 83 83
query13 1660 384 368 368
query14 10259 7468 7138 7138
query15 237 180 180 180
query16 7756 295 303 295
query17 1367 536 533 533
query18 1959 274 285 274
query19 191 151 149 149
query20 91 79 79 79
query21 206 134 122 122
query22 4332 3946 3995 3946
query23 34005 34509 33918 33918
query24 11041 2888 2817 2817
query25 585 398 380 380
query26 716 156 160 156
query27 2308 317 321 317
query28 6138 2166 2161 2161
query29 874 640 617 617
query30 252 148 148 148
query31 992 786 748 748
query32 93 54 56 54
query33 698 298 306 298
query34 890 486 495 486
query35 771 647 637 637
query36 1126 968 985 968
query37 147 84 83 83
query38 2932 2838 2844 2838
query39 895 847 873 847
query40 213 130 166 130
query41 51 51 55 51
query42 116 100 98 98
query43 590 543 542 542
query44 1129 736 726 726
query45 192 155 158 155
query46 1073 740 708 708
query47 1836 1794 1750 1750
query48 369 306 298 298
query49 831 396 412 396
query50 769 387 380 380
query51 6799 7117 6749 6749
query52 110 91 96 91
query53 351 291 285 285
query54 876 443 437 437
query55 75 74 72 72
query56 282 255 263 255
query57 1166 1048 1057 1048
query58 259 242 256 242
query59 3383 3123 3040 3040
query60 310 299 276 276
query61 100 92 94 92
query62 567 466 433 433
query63 321 286 294 286
query64 9521 2175 1626 1626
query65 3190 3103 3129 3103
query66 677 318 325 318
query67 15378 15130 14830 14830
query68 4487 532 530 530
query69 438 366 314 314
query70 1179 1030 1135 1030
query71 381 284 281 281
query72 7008 5472 5345 5345
query73 727 327 321 321
query74 5991 5488 5443 5443
query75 3348 2693 2719 2693
query76 2507 878 926 878
query77 467 311 297 297
query78 9713 9313 8818 8818
query79 2693 515 536 515
query80 2045 489 479 479
query81 596 220 222 220
query82 789 112 103 103
query83 271 173 173 173
query84 281 83 94 83
query85 1952 316 302 302
query86 474 333 324 324
query87 3285 3087 3099 3087
query88 4536 2353 2377 2353
query89 472 375 375 375
query90 1677 189 189 189
query91 132 102 100 100
query92 62 50 52 50
query93 3741 520 508 508
query94 842 212 214 212
query95 415 318 319 318
query96 589 263 266 263
query97 3172 3009 3023 3009
query98 215 195 198 195
query99 1154 828 837 828
Total cold run time: 279908 ms
Total hot run time: 172690 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org