You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "englefly (via GitHub)" <gi...@apache.org> on 2024/09/04 02:06:07 UTC
[PR] [feat](nereids) update how to table row count, and disable join reorder if any table row count is not available [doris]
englefly opened a new pull request, #40349:
URL: https://github.com/apache/doris/pull/40349
## Proposed changes
1. table row count
there are 2 sources for table row count, one is analyzed result, another is from BE report.
But neither of them is accurate. the priorities of them from high to low:
user injected > BE report > analyzed
2. join reorder
if a table is not analyzed, and its row count is not fully reported, optimizer will disable join reorder.
fully report means after a table is newly created or truncated, all tablets row count are reported.
Issue Number: close #xxx
<!--Describe your changes.-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2362915634
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2365228191
<details>
<summary>TPC-DS: <b>Total hot run time: 191636 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 61283d9616ad04f43ab7ee55dc8ee7dab59c8165, data reload: false
query1 960 387 380 380
query2 6175 2012 1976 1976
query3 8676 189 200 189
query4 33848 23412 23377 23377
query5 3439 502 487 487
query6 273 167 164 164
query7 4181 296 322 296
query8 281 209 209 209
query9 9591 2684 2664 2664
query10 467 290 269 269
query11 18145 15138 15299 15138
query12 157 101 94 94
query13 1525 432 422 422
query14 8559 7462 7427 7427
query15 278 174 171 171
query16 8129 460 410 410
query17 1802 623 587 587
query18 2156 317 321 317
query19 368 154 153 153
query20 121 117 117 117
query21 208 106 109 106
query22 4679 4454 4543 4454
query23 35547 34801 35050 34801
query24 11204 2948 2902 2902
query25 533 410 414 410
query26 771 161 164 161
query27 2201 290 282 282
query28 7108 2463 2462 2462
query29 645 429 426 426
query30 271 155 152 152
query31 1010 775 792 775
query32 104 53 55 53
query33 744 292 299 292
query34 890 477 485 477
query35 847 747 729 729
query36 1051 924 933 924
query37 154 87 87 87
query38 3978 3904 3877 3877
query39 1507 1452 1404 1404
query40 205 98 99 98
query41 51 51 49 49
query42 120 92 92 92
query43 518 481 475 475
query44 1218 815 798 798
query45 210 163 165 163
query46 1139 743 740 740
query47 1890 1827 1815 1815
query48 471 363 375 363
query49 868 411 405 405
query50 813 408 416 408
query51 6981 6796 6931 6796
query52 94 89 85 85
query53 252 197 171 171
query54 1144 449 463 449
query55 73 73 80 73
query56 292 253 260 253
query57 1171 1106 1093 1093
query58 230 231 224 224
query59 3063 2798 2872 2798
query60 284 269 270 269
query61 101 107 106 106
query62 804 672 665 665
query63 212 184 183 183
query64 3928 652 664 652
query65 3237 3133 3197 3133
query66 685 298 297 297
query67 16316 16058 15561 15561
query68 5380 571 570 570
query69 640 414 330 330
query70 1162 1046 1129 1046
query71 452 268 265 265
query72 7590 3917 3982 3917
query73 788 322 323 322
query74 9882 9019 9145 9019
query75 4200 2651 2635 2635
query76 4272 965 906 906
query77 710 288 294 288
query78 9854 9203 9594 9203
query79 2386 549 582 549
query80 965 434 446 434
query81 552 242 239 239
query82 511 142 140 140
query83 335 138 132 132
query84 288 78 79 78
query85 1056 291 283 283
query86 306 300 301 300
query87 4465 4349 4260 4260
query88 4329 2320 2303 2303
query89 412 279 281 279
query90 1814 187 180 180
query91 194 143 139 139
query92 62 48 48 48
query93 5055 533 540 533
query94 543 288 300 288
query95 342 248 255 248
query96 639 279 279 279
query97 3255 3122 3243 3122
query98 208 199 197 197
query99 1539 1271 1269 1269
Total cold run time: 303139 ms
Total hot run time: 191636 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2367134873
<details>
<summary>TPC-DS: <b>Total hot run time: 191617 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b09ea841c140f2667cdcbf613ade46ead076fe96, data reload: false
query1 935 378 380 378
query2 6391 2102 1992 1992
query3 8685 190 201 190
query4 34148 23470 23439 23439
query5 3444 452 448 448
query6 273 198 174 174
query7 4195 303 291 291
query8 295 228 221 221
query9 9658 2629 2633 2629
query10 472 274 259 259
query11 17900 15211 15263 15211
query12 160 92 94 92
query13 1523 402 390 390
query14 8549 7373 7246 7246
query15 252 164 171 164
query16 8053 430 460 430
query17 1676 602 586 586
query18 2148 318 318 318
query19 350 163 152 152
query20 120 114 118 114
query21 219 106 105 105
query22 5000 4503 4652 4503
query23 35001 34463 34297 34297
query24 11123 3017 2840 2840
query25 633 398 401 398
query26 1165 157 157 157
query27 2316 278 279 278
query28 7776 2454 2413 2413
query29 823 425 415 415
query30 258 158 153 153
query31 989 805 780 780
query32 91 53 54 53
query33 755 283 285 283
query34 915 479 474 474
query35 877 735 715 715
query36 1065 917 1017 917
query37 164 86 90 86
query38 4060 3907 3928 3907
query39 1521 1436 1421 1421
query40 208 97 98 97
query41 51 48 49 48
query42 113 95 91 91
query43 506 474 474 474
query44 1249 825 803 803
query45 198 162 163 162
query46 1136 744 727 727
query47 1904 1850 1890 1850
query48 445 347 353 347
query49 870 393 397 393
query50 806 395 402 395
query51 6932 6841 6897 6841
query52 95 83 85 83
query53 243 180 172 172
query54 1131 448 452 448
query55 74 72 79 72
query56 279 271 248 248
query57 1242 1071 1084 1071
query58 226 218 223 218
query59 3062 2811 2956 2811
query60 294 274 260 260
query61 106 106 103 103
query62 809 665 662 662
query63 208 186 177 177
query64 3998 649 624 624
query65 3205 3159 3185 3159
query66 767 294 295 294
query67 15854 15522 15976 15522
query68 4887 578 558 558
query69 539 316 298 298
query70 1217 1134 1122 1122
query71 444 272 285 272
query72 7412 4010 4045 4010
query73 765 335 321 321
query74 10309 9102 9075 9075
query75 3473 2627 2665 2627
query76 3254 988 944 944
query77 589 303 290 290
query78 9844 10774 9701 9701
query79 1338 547 549 547
query80 817 429 463 429
query81 577 243 241 241
query82 307 141 143 141
query83 275 134 134 134
query84 268 79 76 76
query85 773 306 277 277
query86 481 304 303 303
query87 4450 4334 4319 4319
query88 3064 2280 2294 2280
query89 380 285 285 285
query90 1897 185 189 185
query91 202 144 141 141
query92 60 47 51 47
query93 1050 525 532 525
query94 792 292 300 292
query95 353 246 260 246
query96 599 270 278 270
query97 3257 3106 3193 3106
query98 217 198 197 197
query99 1585 1307 1287 1287
Total cold run time: 295464 ms
Total hot run time: 191617 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2367138733
<details>
<summary>ClickBench: <b>Total hot run time: 32.38 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b09ea841c140f2667cdcbf613ade46ead076fe96, data reload: false
query1 0.04 0.05 0.04
query2 0.07 0.03 0.02
query3 0.23 0.06 0.06
query4 1.64 0.10 0.10
query5 0.50 0.50 0.51
query6 1.15 0.71 0.71
query7 0.02 0.01 0.01
query8 0.03 0.03 0.03
query9 0.57 0.50 0.49
query10 0.56 0.58 0.55
query11 0.14 0.11 0.11
query12 0.13 0.10 0.10
query13 0.60 0.59 0.58
query14 2.97 2.97 3.02
query15 0.88 0.82 0.81
query16 0.40 0.38 0.38
query17 0.96 1.02 1.05
query18 0.20 0.19 0.20
query19 2.01 1.80 2.00
query20 0.01 0.02 0.01
query21 15.36 0.56 0.58
query22 2.25 2.04 1.68
query23 17.37 0.88 0.78
query24 2.51 1.22 0.79
query25 0.23 0.20 0.15
query26 0.30 0.13 0.14
query27 0.03 0.04 0.04
query28 11.50 1.13 1.18
query29 13.35 3.29 3.26
query30 0.24 0.05 0.06
query31 2.87 0.38 0.37
query32 3.29 0.47 0.47
query33 3.03 3.04 3.00
query34 17.05 4.39 4.34
query35 4.35 4.38 4.41
query36 0.67 0.47 0.48
query37 0.09 0.06 0.06
query38 0.05 0.03 0.04
query39 0.03 0.02 0.02
query40 0.15 0.12 0.12
query41 0.07 0.02 0.02
query42 0.03 0.02 0.02
query43 0.03 0.03 0.03
Total cold run time: 107.96 s
Total hot run time: 32.38 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2345609830
<details>
<summary>TPC-H: <b>Total hot run time: 39981 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4c5f044ea0e8082fff517399b66309a70ce2c13d, data reload: false
------ Round 1 ----------------------------------
q1 17604 4345 4206 4206
q2 1994 311 307 307
q3 12123 1260 1456 1260
q4 10502 998 1054 998
q5 7728 3229 3144 3144
q6 226 140 138 138
q7 1038 621 629 621
q8 9463 2028 1994 1994
q9 6842 6355 6254 6254
q10 7025 2521 2510 2510
q11 438 262 254 254
q12 410 230 226 226
q13 17765 3050 3034 3034
q14 285 249 251 249
q15 553 503 494 494
q16 540 441 427 427
q17 987 952 954 952
q18 7460 6743 6866 6743
q19 1379 1262 1234 1234
q20 598 349 336 336
q21 3911 3600 3629 3600
q22 1087 1002 1000 1000
Total cold run time: 109958 ms
Total hot run time: 39981 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4224 4158 4167 4158
q2 342 229 237 229
q3 2931 2903 2926 2903
q4 1963 2022 2052 2022
q5 5770 5681 5750 5681
q6 221 135 132 132
q7 2196 1765 1777 1765
q8 3336 3435 3502 3435
q9 8845 8834 8835 8834
q10 3550 3635 3589 3589
q11 581 485 480 480
q12 809 630 625 625
q13 8576 3146 3255 3146
q14 318 302 276 276
q15 544 513 505 505
q16 549 461 457 457
q17 1812 1807 1807 1807
q18 8436 8068 7950 7950
q19 1789 1778 1739 1739
q20 2154 1882 1898 1882
q21 5823 5676 5673 5673
q22 1086 1019 1008 1008
Total cold run time: 65855 ms
Total hot run time: 58296 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on code in PR #40349:
URL: https://github.com/apache/doris/pull/40349#discussion_r1755970262
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/NereidsPlanner.java:
##########
@@ -288,6 +296,18 @@ private Plan planWithoutLock(
return physicalPlan;
}
+ private List<LogicalOlapScan> getAllOlapScans(Plan plan) {
Review Comment:
the meaning of -1 is explained in comments for getOlapTableRowCount()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2350888652
run p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2345644021
<details>
<summary>TPC-DS: <b>Total hot run time: 196276 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4c5f044ea0e8082fff517399b66309a70ce2c13d, data reload: false
query1 917 393 394 393
query2 6288 1738 1747 1738
query3 8667 199 198 198
query4 32075 24242 23934 23934
query5 4712 531 531 531
query6 288 195 178 178
query7 4191 314 300 300
query8 302 233 228 228
query9 8336 2437 2416 2416
query10 467 291 289 289
query11 16126 15459 15764 15459
query12 163 105 101 101
query13 1530 411 389 389
query14 10187 6796 7083 6796
query15 210 171 176 171
query16 7536 483 457 457
query17 1149 584 574 574
query18 1802 324 340 324
query19 222 166 164 164
query20 130 120 119 119
query21 217 116 112 112
query22 4814 4725 4591 4591
query23 35283 34199 34369 34199
query24 10608 3263 3164 3164
query25 618 402 388 388
query26 725 161 155 155
query27 2081 293 295 293
query28 5106 2031 2005 2005
query29 870 417 403 403
query30 242 154 154 154
query31 991 757 762 757
query32 99 56 54 54
query33 617 301 295 295
query34 941 469 481 469
query35 880 732 718 718
query36 1079 900 906 900
query37 141 81 86 81
query38 3961 3900 3889 3889
query39 1524 1442 1388 1388
query40 201 118 115 115
query41 46 46 45 45
query42 124 100 94 94
query43 481 421 440 421
query44 1190 766 745 745
query45 200 167 168 167
query46 1160 870 854 854
query47 1846 1764 1820 1764
query48 368 290 287 287
query49 847 450 459 450
query50 1027 443 445 443
query51 7119 6894 6809 6809
query52 99 90 90 90
query53 272 187 184 184
query54 798 469 481 469
query55 77 79 77 77
query56 326 274 265 265
query57 1212 1056 1122 1056
query58 249 235 233 233
query59 2845 2526 2453 2453
query60 320 274 274 274
query61 107 101 101 101
query62 919 680 668 668
query63 226 195 188 188
query64 3560 716 663 663
query65 3249 3212 3186 3186
query66 763 335 339 335
query67 15900 15772 15550 15550
query68 3143 892 892 892
query69 450 331 337 331
query70 1172 1150 1145 1145
query71 366 354 362 354
query72 5910 3491 3497 3491
query73 593 584 583 583
query74 9205 9221 9109 9109
query75 3271 3186 3143 3143
query76 1720 901 895 895
query77 452 423 423 423
query78 9554 9358 9388 9358
query79 998 987 957 957
query80 878 862 844 844
query81 448 283 283 283
query82 272 272 271 271
query83 195 194 191 191
query84 207 109 107 107
query85 715 428 454 428
query86 320 316 322 316
query87 4558 4542 4425 4425
query88 4122 4055 4024 4024
query89 399 384 392 384
query90 1243 323 322 322
query91 127 124 123 123
query92 80 77 80 77
query93 1073 1064 1071 1064
query94 568 400 388 388
query95 473 434 442 434
query96 477 472 471 471
query97 3141 3179 3118 3118
query98 237 240 232 232
query99 1645 1353 1333 1333
Total cold run time: 279638 ms
Total hot run time: 196276 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2362681947
<details>
<summary>TPC-DS: <b>Total hot run time: 192112 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4a712b05146e722ef81023371d7dd89eb275a9b7, data reload: false
query1 930 387 416 387
query2 6351 2157 2060 2060
query3 8691 203 209 203
query4 34200 23571 23459 23459
query5 3490 475 498 475
query6 283 177 174 174
query7 4201 317 316 316
query8 295 230 244 230
query9 9463 2699 2716 2699
query10 455 283 285 283
query11 18031 15241 15402 15241
query12 158 99 100 99
query13 1569 429 409 409
query14 9774 7751 7198 7198
query15 247 166 179 166
query16 7957 509 457 457
query17 1661 615 590 590
query18 2137 330 326 326
query19 367 156 156 156
query20 124 112 126 112
query21 216 112 105 105
query22 4745 4545 4471 4471
query23 35145 34593 34515 34515
query24 11162 2952 2940 2940
query25 620 404 403 403
query26 1148 169 162 162
query27 2214 298 299 298
query28 7724 2448 2428 2428
query29 825 438 428 428
query30 258 161 161 161
query31 1022 778 799 778
query32 99 56 60 56
query33 733 305 328 305
query34 911 491 496 491
query35 893 728 736 728
query36 1100 940 949 940
query37 156 88 89 88
query38 4029 3895 3873 3873
query39 1469 1418 1393 1393
query40 206 98 98 98
query41 51 51 48 48
query42 123 97 97 97
query43 549 500 492 492
query44 1187 817 806 806
query45 195 164 169 164
query46 1158 780 768 768
query47 1914 1781 1857 1781
query48 470 371 383 371
query49 881 430 397 397
query50 820 413 425 413
query51 6992 6926 7019 6926
query52 101 88 88 88
query53 251 176 180 176
query54 1196 458 453 453
query55 81 79 82 79
query56 310 266 263 263
query57 1185 1092 1076 1076
query58 236 229 235 229
query59 3290 3175 3108 3108
query60 283 267 260 260
query61 106 102 100 100
query62 800 671 643 643
query63 222 194 184 184
query64 3898 688 631 631
query65 3305 3163 3208 3163
query66 735 296 297 296
query67 15672 15443 15463 15443
query68 4406 595 574 574
query69 447 314 298 298
query70 1204 1130 1062 1062
query71 393 271 283 271
query72 6237 4027 4053 4027
query73 786 329 335 329
query74 9471 9000 9086 9000
query75 3362 2661 2712 2661
query76 2690 965 982 965
query77 424 298 298 298
query78 9843 9116 9065 9065
query79 1701 557 543 543
query80 1026 453 455 453
query81 592 236 246 236
query82 726 147 148 147
query83 219 139 136 136
query84 249 82 79 79
query85 1311 303 283 283
query86 386 275 307 275
query87 4426 4270 4366 4270
query88 3258 2376 2373 2373
query89 394 288 287 287
query90 1716 190 183 183
query91 203 142 146 142
query92 56 51 48 48
query93 1733 557 544 544
query94 739 291 288 288
query95 359 260 264 260
query96 614 280 283 280
query97 3238 3110 3109 3109
query98 220 202 192 192
query99 1697 1317 1278 1278
Total cold run time: 295196 ms
Total hot run time: 192112 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2357458367
<details>
<summary>ClickBench: <b>Total hot run time: 32.65 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 603a31adf3385c8dc6f08873e8eaa597314be95f, data reload: false
query1 0.05 0.04 0.04
query2 0.07 0.03 0.03
query3 0.23 0.06 0.06
query4 1.64 0.09 0.10
query5 0.49 0.50 0.50
query6 1.14 0.73 0.73
query7 0.02 0.01 0.01
query8 0.04 0.03 0.03
query9 0.56 0.51 0.50
query10 0.57 0.55 0.55
query11 0.14 0.11 0.11
query12 0.13 0.10 0.10
query13 0.62 0.59 0.58
query14 3.02 2.98 2.97
query15 0.88 0.82 0.82
query16 0.36 0.39 0.38
query17 1.04 1.05 1.00
query18 0.21 0.21 0.20
query19 1.89 1.87 2.02
query20 0.02 0.01 0.01
query21 15.35 0.60 0.59
query22 2.60 2.01 2.15
query23 17.17 0.86 0.74
query24 3.09 0.68 1.80
query25 0.20 0.09 0.05
query26 0.57 0.14 0.13
query27 0.04 0.04 0.04
query28 10.27 1.10 1.06
query29 12.53 3.28 3.26
query30 0.25 0.06 0.06
query31 2.88 0.38 0.38
query32 3.27 0.46 0.47
query33 3.03 3.02 3.03
query34 16.81 4.39 4.43
query35 4.42 4.43 4.45
query36 0.65 0.47 0.47
query37 0.08 0.06 0.06
query38 0.05 0.04 0.03
query39 0.03 0.02 0.02
query40 0.16 0.12 0.12
query41 0.08 0.02 0.03
query42 0.03 0.02 0.02
query43 0.04 0.03 0.03
Total cold run time: 106.72 s
Total hot run time: 32.65 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2338085362
<details>
<summary>TPC-DS: <b>Total hot run time: 186844 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2c9998ac9f7f66ddd55e69fbe64ec9ccd434f6a0, data reload: false
query1 915 379 393 379
query2 6493 2012 1961 1961
query3 6651 208 218 208
query4 34318 23207 23052 23052
query5 4160 510 482 482
query6 262 172 185 172
query7 4593 295 289 289
query8 316 242 222 222
query9 8488 2486 2499 2486
query10 427 272 274 272
query11 16678 14972 14935 14935
query12 150 103 123 103
query13 1639 392 357 357
query14 11367 7168 6893 6893
query15 279 173 176 173
query16 7632 450 454 450
query17 1570 568 549 549
query18 1486 286 307 286
query19 296 148 141 141
query20 119 118 112 112
query21 208 110 102 102
query22 4596 4187 4069 4069
query23 34255 33234 33985 33234
query24 11051 2900 2786 2786
query25 624 375 398 375
query26 1172 151 154 151
query27 2346 273 272 272
query28 7242 2033 2030 2030
query29 839 437 428 428
query30 309 165 155 155
query31 1023 775 804 775
query32 102 59 62 59
query33 769 297 295 295
query34 960 479 483 479
query35 874 737 716 716
query36 1052 935 915 915
query37 163 92 91 91
query38 4007 3881 3819 3819
query39 1462 1388 1407 1388
query40 201 122 118 118
query41 51 50 49 49
query42 121 96 95 95
query43 513 471 477 471
query44 1265 767 745 745
query45 198 172 170 170
query46 1140 756 739 739
query47 1856 1792 1789 1789
query48 358 294 290 290
query49 1115 437 438 437
query50 810 429 425 425
query51 6939 6915 6831 6831
query52 100 89 90 89
query53 256 188 182 182
query54 950 466 465 465
query55 81 77 72 72
query56 276 295 264 264
query57 1181 1051 1064 1051
query58 252 224 231 224
query59 2945 2778 2724 2724
query60 297 267 271 267
query61 104 105 108 105
query62 826 661 673 661
query63 215 184 185 184
query64 4443 719 655 655
query65 3215 3185 3144 3144
query66 1356 339 338 338
query67 15643 15230 15248 15230
query68 3543 592 559 559
query69 399 281 280 280
query70 1183 1136 1126 1126
query71 338 270 271 270
query72 6438 4037 3998 3998
query73 739 325 322 322
query74 9090 8891 8755 8755
query75 3408 2743 2641 2641
query76 2016 1050 971 971
query77 512 312 315 312
query78 9652 9079 8991 8991
query79 1018 533 526 526
query80 702 504 562 504
query81 452 235 227 227
query82 249 138 142 138
query83 180 153 160 153
query84 237 80 77 77
query85 688 293 275 275
query86 322 276 306 276
query87 4396 4205 4257 4205
query88 3227 2268 2263 2263
query89 376 286 295 286
query90 1855 213 194 194
query91 125 99 149 99
query92 61 49 48 48
query93 1035 554 541 541
query94 813 299 299 299
query95 360 259 254 254
query96 586 259 267 259
query97 3231 3114 3075 3075
query98 214 201 198 198
query99 1429 1279 1247 1247
Total cold run time: 286474 ms
Total hot run time: 186844 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2331905107
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2345185364
<details>
<summary>TPC-H: <b>Total hot run time: 39891 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1e008b682b1245f10eacf6eeed3c52230e77553e, data reload: false
------ Round 1 ----------------------------------
q1 17591 4258 4142 4142
q2 2002 307 309 307
q3 12157 1267 1376 1267
q4 10527 935 1049 935
q5 7720 3232 3176 3176
q6 230 143 143 143
q7 1047 622 628 622
q8 9463 2025 1999 1999
q9 6786 6358 6315 6315
q10 7011 2548 2511 2511
q11 438 256 259 256
q12 411 238 237 237
q13 17753 3045 3032 3032
q14 285 253 248 248
q15 558 499 494 494
q16 513 428 424 424
q17 982 956 941 941
q18 7436 6781 6929 6781
q19 1368 1248 1232 1232
q20 617 339 333 333
q21 3885 3511 3500 3500
q22 1081 1030 996 996
Total cold run time: 109861 ms
Total hot run time: 39891 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4158 4196 4197 4196
q2 335 232 239 232
q3 2916 2859 2887 2859
q4 1976 1984 2074 1984
q5 5838 5722 5723 5722
q6 221 134 139 134
q7 2152 1772 1751 1751
q8 3315 3407 3441 3407
q9 8844 8845 8830 8830
q10 3558 3635 3592 3592
q11 572 478 476 476
q12 788 603 678 603
q13 8446 3157 3266 3157
q14 326 285 276 276
q15 571 505 502 502
q16 512 461 482 461
q17 1785 1768 1772 1768
q18 8347 8084 7902 7902
q19 1780 1742 1751 1742
q20 2130 1916 1866 1866
q21 5897 5735 5720 5720
q22 1111 1039 1013 1013
Total cold run time: 65578 ms
Total hot run time: 58193 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2362673560
<details>
<summary>TPC-H: <b>Total hot run time: 41629 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4a712b05146e722ef81023371d7dd89eb275a9b7, data reload: false
------ Round 1 ----------------------------------
q1 17580 7330 7252 7252
q2 2020 279 272 272
q3 12283 1062 1238 1062
q4 10557 787 751 751
q5 7754 3119 3074 3074
q6 235 153 153 153
q7 1014 617 626 617
q8 9663 2085 2051 2051
q9 8267 6394 6386 6386
q10 7028 2332 2285 2285
q11 428 250 255 250
q12 405 221 218 218
q13 17778 3031 2982 2982
q14 248 209 211 209
q15 575 514 517 514
q16 686 627 631 627
q17 973 832 830 830
q18 7366 6679 6705 6679
q19 1402 1023 1107 1023
q20 571 300 290 290
q21 4116 3092 3214 3092
q22 1115 1022 1012 1012
Total cold run time: 112064 ms
Total hot run time: 41629 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7202 7240 7258 7240
q2 335 234 234 234
q3 3049 2997 2933 2933
q4 2114 1818 1750 1750
q5 5602 5593 5745 5593
q6 231 144 144 144
q7 2192 1869 1823 1823
q8 3304 3456 3442 3442
q9 8844 8856 8844 8844
q10 3429 3495 3488 3488
q11 582 501 485 485
q12 827 645 619 619
q13 9631 3178 3151 3151
q14 311 273 273 273
q15 592 537 533 533
q16 714 674 687 674
q17 1821 1656 1571 1571
q18 8287 7690 7909 7690
q19 1730 1726 1498 1498
q20 2119 1887 1897 1887
q21 5480 5392 5451 5392
q22 1157 1032 1065 1032
Total cold run time: 69553 ms
Total hot run time: 60296 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on code in PR #40349:
URL: https://github.com/apache/doris/pull/40349#discussion_r1767909315
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/stats/StatsCalculator.java:
##########
@@ -205,6 +210,23 @@ public Map<String, ColumnStatistic> getTotalColumnStatisticMap() {
return totalColumnStatisticMap;
}
+ /**
+ * disable join reorder if any table row count is not available.
+ */
+ public static void disableJoinReorderIfTableRowCountNotAvailable(
+ List<LogicalOlapScan> scans, CascadesContext context) {
+ StatsCalculator calculator = new StatsCalculator(context);
+ for (LogicalOlapScan scan : scans) {
+ double rowCount = calculator.getOlapTableRowCount(scan);
+ if (rowCount == -1 && ConnectContext.get() != null) {
+ LOG.info("disable join reorder since row count not available: "
+ + scan.getTable().getNameWithFullQualifiers());
+ ConnectContext.get().getSessionVariable().setDisableJoinReorder(true);
Review Comment:
this is statement level. it has no impact on other queries.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2365229702
<details>
<summary>ClickBench: <b>Total hot run time: 32.71 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 61283d9616ad04f43ab7ee55dc8ee7dab59c8165, data reload: false
query1 0.05 0.05 0.04
query2 0.06 0.02 0.03
query3 0.23 0.06 0.06
query4 1.67 0.10 0.10
query5 0.51 0.49 0.48
query6 1.14 0.73 0.73
query7 0.02 0.01 0.02
query8 0.04 0.04 0.03
query9 0.57 0.51 0.52
query10 0.55 0.56 0.56
query11 0.16 0.11 0.10
query12 0.14 0.10 0.10
query13 0.60 0.59 0.59
query14 3.07 2.93 2.96
query15 0.89 0.82 0.81
query16 0.38 0.39 0.38
query17 0.99 1.07 1.02
query18 0.19 0.20 0.19
query19 1.94 1.87 2.04
query20 0.02 0.01 0.01
query21 15.35 0.57 0.58
query22 3.40 2.00 1.57
query23 17.30 0.93 0.90
query24 2.48 1.20 1.02
query25 0.24 0.23 0.08
query26 0.41 0.14 0.13
query27 0.03 0.04 0.04
query28 11.04 1.09 1.07
query29 12.55 3.30 3.32
query30 0.25 0.06 0.05
query31 2.89 0.38 0.36
query32 3.29 0.47 0.47
query33 2.98 3.04 3.07
query34 17.03 4.36 4.42
query35 4.46 4.44 4.49
query36 0.68 0.48 0.48
query37 0.08 0.05 0.06
query38 0.05 0.03 0.03
query39 0.03 0.02 0.02
query40 0.16 0.12 0.12
query41 0.07 0.02 0.02
query42 0.02 0.02 0.02
query43 0.04 0.03 0.03
Total cold run time: 108.05 s
Total hot run time: 32.71 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2327858768
<details>
<summary>TPC-DS: <b>Total hot run time: 197281 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4b3661cd3256aee78fe50faf5ad9461dd4d1d253, data reload: false
query1 954 405 388 388
query2 12600 8737 8792 8737
query3 10679 217 223 217
query4 33617 23231 23247 23231
query5 3565 482 483 482
query6 255 171 169 169
query7 3932 306 290 290
query8 260 210 218 210
query9 8379 2521 2497 2497
query10 444 279 275 275
query11 17524 15074 14952 14952
query12 134 101 103 101
query13 1527 401 376 376
query14 9328 7111 7521 7111
query15 264 188 185 185
query16 7870 514 481 481
query17 1604 604 587 587
query18 2025 302 294 294
query19 295 153 146 146
query20 131 116 115 115
query21 208 108 105 105
query22 4452 4363 4439 4363
query23 34267 34258 35114 34258
query24 8436 2933 2818 2818
query25 529 374 390 374
query26 677 151 152 151
query27 1972 269 271 269
query28 5907 2061 2038 2038
query29 637 406 422 406
query30 260 151 148 148
query31 984 763 760 760
query32 89 49 54 49
query33 612 302 285 285
query34 857 480 480 480
query35 849 723 722 722
query36 1087 953 942 942
query37 143 86 94 86
query38 4025 3931 3783 3783
query39 1464 1395 1433 1395
query40 195 115 109 109
query41 48 43 49 43
query42 112 95 93 93
query43 511 467 478 467
query44 1091 736 732 732
query45 202 165 163 163
query46 1095 744 731 731
query47 1895 1792 1812 1792
query48 366 304 304 304
query49 837 417 420 417
query50 804 429 413 413
query51 7073 6879 6734 6734
query52 98 91 89 89
query53 247 177 177 177
query54 794 465 452 452
query55 75 74 75 74
query56 278 254 255 254
query57 1186 1071 1087 1071
query58 259 234 245 234
query59 3024 2777 2810 2777
query60 284 268 265 265
query61 101 100 99 99
query62 763 645 653 645
query63 211 179 181 179
query64 3428 673 689 673
query65 3224 3146 3158 3146
query66 685 332 329 329
query67 15382 15383 15204 15204
query68 4330 575 557 557
query69 415 288 293 288
query70 1222 1159 1071 1071
query71 333 279 259 259
query72 6334 3998 3967 3967
query73 742 328 340 328
query74 9106 8812 8919 8812
query75 3358 2696 2681 2681
query76 2022 1047 935 935
query77 422 313 322 313
query78 9722 10912 10878 10878
query79 994 556 533 533
query80 722 546 511 511
query81 556 233 228 228
query82 286 148 154 148
query83 237 149 145 145
query84 224 74 74 74
query85 683 286 280 280
query86 325 301 297 297
query87 4385 4244 4336 4244
query88 2967 2353 2338 2338
query89 383 290 288 288
query90 1749 193 189 189
query91 122 97 98 97
query92 55 49 48 48
query93 1032 532 537 532
query94 686 291 302 291
query95 350 249 301 249
query96 584 275 269 269
query97 3271 3039 3147 3039
query98 212 199 189 189
query99 1529 1265 1261 1261
Total cold run time: 286468 ms
Total hot run time: 197281 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2357442718
<details>
<summary>TPC-H: <b>Total hot run time: 41751 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 603a31adf3385c8dc6f08873e8eaa597314be95f, data reload: false
------ Round 1 ----------------------------------
q1 18224 7422 7312 7312
q2 3079 160 174 160
q3 11775 1205 1176 1176
q4 10880 712 723 712
q5 7851 3171 3112 3112
q6 234 153 152 152
q7 1028 629 618 618
q8 9436 2106 2029 2029
q9 6886 6420 6420 6420
q10 7018 2248 2262 2248
q11 444 252 260 252
q12 414 225 221 221
q13 17790 3004 2940 2940
q14 236 226 217 217
q15 573 516 514 514
q16 702 616 639 616
q17 977 848 840 840
q18 7374 6639 6802 6639
q19 1418 1104 979 979
q20 593 300 285 285
q21 4200 3341 3299 3299
q22 1092 1010 1034 1010
Total cold run time: 112224 ms
Total hot run time: 41751 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7258 7193 7254 7193
q2 326 228 229 228
q3 2869 2776 2809 2776
q4 1960 1733 1708 1708
q5 5398 5396 5433 5396
q6 222 144 142 142
q7 2104 1740 1722 1722
q8 3174 3299 3317 3299
q9 8359 8426 8385 8385
q10 3392 3388 3352 3352
q11 574 476 471 471
q12 789 587 578 578
q13 7182 3000 3009 3000
q14 288 251 262 251
q15 564 514 511 511
q16 700 650 674 650
q17 1756 1559 1540 1540
q18 7731 7437 7432 7432
q19 1688 1578 1576 1576
q20 2023 1800 1807 1800
q21 5529 5333 5101 5101
q22 1099 1026 990 990
Total cold run time: 64985 ms
Total hot run time: 58101 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2363045561
<details>
<summary>TPC-H: <b>Total hot run time: 41816 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 61283d9616ad04f43ab7ee55dc8ee7dab59c8165, data reload: false
------ Round 1 ----------------------------------
q1 17806 7406 7243 7243
q2 2019 296 273 273
q3 12224 1060 1223 1060
q4 10578 769 885 769
q5 7743 3137 3108 3108
q6 235 153 152 152
q7 1020 626 596 596
q8 9441 2064 2039 2039
q9 6869 6527 6431 6431
q10 7022 2291 2279 2279
q11 443 254 252 252
q12 411 223 214 214
q13 17785 2974 3018 2974
q14 250 214 208 208
q15 558 551 525 525
q16 703 630 622 622
q17 982 821 813 813
q18 7329 6751 6741 6741
q19 1393 991 1014 991
q20 590 281 286 281
q21 3972 3285 3336 3285
q22 1101 1002 960 960
Total cold run time: 110474 ms
Total hot run time: 41816 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7239 7238 7210 7210
q2 347 246 248 246
q3 3157 2974 2921 2921
q4 2062 1841 1760 1760
q5 5660 5572 5714 5572
q6 231 141 150 141
q7 2199 1771 1834 1771
q8 3325 3462 3428 3428
q9 8804 8909 8802 8802
q10 3513 3515 3463 3463
q11 582 497 478 478
q12 794 607 598 598
q13 9275 3160 3213 3160
q14 322 266 287 266
q15 583 522 515 515
q16 731 677 683 677
q17 1883 1622 1577 1577
q18 8384 7841 7931 7841
q19 1717 1591 1562 1562
q20 2092 1920 1863 1863
q21 5396 5538 5336 5336
q22 1175 1073 1033 1033
Total cold run time: 69471 ms
Total hot run time: 60220 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #40349:
URL: https://github.com/apache/doris/pull/40349#discussion_r1767880271
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/NereidsPlanner.java:
##########
@@ -256,6 +259,17 @@ private Plan planWithoutLock(
}
}
+ // if we cannot get table row count, skip join reorder
+ // except:
+ // 1. user set leading hint
+ // 2. ut test. In ut test, FeConstants.enableInternalSchemaDb is false or FeConstants.runningUnitTest is true
+ if (FeConstants.enableInternalSchemaDb && !FeConstants.runningUnitTest
+ && !cascadesContext.isLeadingDisableJoinReorder()) {
+ List<LogicalOlapScan> scans = cascadesContext.getRewritePlan()
+ .collectToList(LogicalOlapScan.class::isInstance);
Review Comment:
only need olapscan, how about other scan?
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/stats/StatsCalculator.java:
##########
@@ -205,6 +210,23 @@ public Map<String, ColumnStatistic> getTotalColumnStatisticMap() {
return totalColumnStatisticMap;
}
+ /**
+ * disable join reorder if any table row count is not available.
+ */
+ public static void disableJoinReorderIfTableRowCountNotAvailable(
+ List<LogicalOlapScan> scans, CascadesContext context) {
+ StatsCalculator calculator = new StatsCalculator(context);
+ for (LogicalOlapScan scan : scans) {
+ double rowCount = calculator.getOlapTableRowCount(scan);
+ if (rowCount == -1 && ConnectContext.get() != null) {
Review Comment:
-1 is a magic number could we add a static variable `UNKNOWN_ROW_COUNT` to static utils and replace all magic number `-1` by the static variable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2362685969
<details>
<summary>ClickBench: <b>Total hot run time: 33 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4a712b05146e722ef81023371d7dd89eb275a9b7, data reload: false
query1 0.04 0.04 0.03
query2 0.06 0.02 0.02
query3 0.22 0.06 0.06
query4 1.66 0.10 0.10
query5 0.51 0.52 0.53
query6 1.13 0.72 0.72
query7 0.02 0.01 0.01
query8 0.03 0.03 0.03
query9 0.57 0.50 0.50
query10 0.54 0.59 0.55
query11 0.15 0.10 0.11
query12 0.14 0.11 0.11
query13 0.61 0.60 0.60
query14 3.11 2.95 3.00
query15 0.90 0.83 0.85
query16 0.38 0.39 0.42
query17 1.06 1.03 1.06
query18 0.20 0.19 0.19
query19 2.01 1.87 2.02
query20 0.01 0.01 0.01
query21 15.37 0.61 0.62
query22 3.45 2.13 2.07
query23 17.54 0.74 0.76
query24 3.27 0.94 1.14
query25 0.23 0.22 0.11
query26 0.41 0.14 0.14
query27 0.04 0.04 0.04
query28 10.39 1.09 1.06
query29 12.54 3.26 3.23
query30 0.25 0.05 0.05
query31 2.90 0.40 0.38
query32 3.26 0.47 0.48
query33 3.01 3.07 3.04
query34 17.04 4.35 4.37
query35 4.48 4.41 4.37
query36 0.69 0.49 0.52
query37 0.09 0.06 0.06
query38 0.04 0.03 0.04
query39 0.03 0.02 0.02
query40 0.16 0.13 0.12
query41 0.08 0.02 0.03
query42 0.04 0.02 0.02
query43 0.03 0.02 0.02
Total cold run time: 108.69 s
Total hot run time: 33 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "xzj7019 (via GitHub)" <gi...@apache.org>.
xzj7019 commented on code in PR #40349:
URL: https://github.com/apache/doris/pull/40349#discussion_r1767896292
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/stats/StatsCalculator.java:
##########
@@ -205,6 +210,23 @@ public Map<String, ColumnStatistic> getTotalColumnStatisticMap() {
return totalColumnStatisticMap;
}
+ /**
+ * disable join reorder if any table row count is not available.
+ */
+ public static void disableJoinReorderIfTableRowCountNotAvailable(
+ List<LogicalOlapScan> scans, CascadesContext context) {
+ StatsCalculator calculator = new StatsCalculator(context);
+ for (LogicalOlapScan scan : scans) {
+ double rowCount = calculator.getOlapTableRowCount(scan);
+ if (rowCount == -1 && ConnectContext.get() != null) {
+ LOG.info("disable join reorder since row count not available: "
+ + scan.getTable().getNameWithFullQualifiers());
+ ConnectContext.get().getSessionVariable().setDisableJoinReorder(true);
Review Comment:
This sets session variable directly, should use disableNereidsJoinReorderOnce?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on code in PR #40349:
URL: https://github.com/apache/doris/pull/40349#discussion_r1767909315
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/stats/StatsCalculator.java:
##########
@@ -205,6 +210,23 @@ public Map<String, ColumnStatistic> getTotalColumnStatisticMap() {
return totalColumnStatisticMap;
}
+ /**
+ * disable join reorder if any table row count is not available.
+ */
+ public static void disableJoinReorderIfTableRowCountNotAvailable(
+ List<LogicalOlapScan> scans, CascadesContext context) {
+ StatsCalculator calculator = new StatsCalculator(context);
+ for (LogicalOlapScan scan : scans) {
+ double rowCount = calculator.getOlapTableRowCount(scan);
+ if (rowCount == -1 && ConnectContext.get() != null) {
+ LOG.info("disable join reorder since row count not available: "
+ + scan.getTable().getNameWithFullQualifiers());
+ ConnectContext.get().getSessionVariable().setDisableJoinReorder(true);
Review Comment:
this is statement level. it has no impact on other queries.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2359863668
<details>
<summary>ClickBench: <b>Total hot run time: 33.06 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a5d7a05f22338b4b9214968e0f3c040ac6c67485, data reload: false
query1 0.04 0.05 0.04
query2 0.06 0.03 0.03
query3 0.23 0.06 0.06
query4 1.64 0.10 0.10
query5 0.51 0.51 0.49
query6 1.15 0.73 0.72
query7 0.02 0.01 0.02
query8 0.03 0.03 0.03
query9 0.56 0.48 0.50
query10 0.54 0.56 0.55
query11 0.15 0.10 0.10
query12 0.13 0.11 0.11
query13 0.61 0.60 0.59
query14 3.01 2.99 2.93
query15 0.90 0.81 0.83
query16 0.41 0.37 0.37
query17 1.08 0.97 1.05
query18 0.20 0.19 0.20
query19 1.95 1.85 2.00
query20 0.02 0.01 0.00
query21 15.35 0.59 0.61
query22 2.40 2.83 2.22
query23 17.28 0.74 0.99
query24 3.25 1.87 0.82
query25 0.20 0.20 0.10
query26 0.47 0.15 0.14
query27 0.03 0.04 0.04
query28 10.00 1.09 1.08
query29 12.52 3.38 3.40
query30 0.25 0.06 0.06
query31 2.87 0.39 0.38
query32 3.24 0.47 0.48
query33 2.99 3.00 3.03
query34 17.17 4.43 4.40
query35 4.49 4.43 4.43
query36 0.68 0.49 0.48
query37 0.08 0.06 0.06
query38 0.04 0.04 0.03
query39 0.04 0.02 0.02
query40 0.15 0.13 0.12
query41 0.08 0.02 0.02
query42 0.04 0.02 0.02
query43 0.03 0.03 0.03
Total cold run time: 106.89 s
Total hot run time: 33.06 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2365223851
<details>
<summary>TPC-H: <b>Total hot run time: 41602 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 61283d9616ad04f43ab7ee55dc8ee7dab59c8165, data reload: false
------ Round 1 ----------------------------------
q1 17593 7332 7268 7268
q2 2022 290 282 282
q3 12196 1055 1200 1055
q4 10566 816 768 768
q5 7761 3084 3074 3074
q6 235 151 147 147
q7 1003 608 601 601
q8 9438 2017 2017 2017
q9 6782 6432 6389 6389
q10 7001 2295 2284 2284
q11 429 245 251 245
q12 412 223 219 219
q13 17779 2955 2984 2955
q14 242 212 222 212
q15 587 528 528 528
q16 663 643 605 605
q17 953 787 767 767
q18 7237 6678 6802 6678
q19 1423 971 1097 971
q20 601 287 278 278
q21 3976 3263 3305 3263
q22 1105 996 996 996
Total cold run time: 110004 ms
Total hot run time: 41602 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7258 7249 7244 7244
q2 326 230 248 230
q3 3031 3023 2877 2877
q4 2085 1831 1748 1748
q5 5613 5584 5661 5584
q6 239 150 147 147
q7 2227 1815 1826 1815
q8 3291 3434 3426 3426
q9 8689 8898 8750 8750
q10 3508 3455 3453 3453
q11 565 485 465 465
q12 828 621 657 621
q13 9598 3178 3132 3132
q14 314 267 287 267
q15 573 528 534 528
q16 726 672 672 672
q17 1782 1585 1577 1577
q18 8184 7998 7899 7899
q19 1721 1415 1580 1415
q20 2089 1863 1880 1863
q21 5500 5405 5475 5405
q22 1145 1045 1059 1045
Total cold run time: 69292 ms
Total hot run time: 60163 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on code in PR #40349:
URL: https://github.com/apache/doris/pull/40349#discussion_r1755970170
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/stats/StatsCalculator.java:
##########
@@ -205,6 +210,22 @@ public Map<String, ColumnStatistic> getTotalColumnStatisticMap() {
return totalColumnStatisticMap;
}
+ /**
+ * disable join reorder if any table row count is not available.
+ */
+ public static void disableJoinReorderIfTableRowCountNotAvailable(
+ List<LogicalOlapScan> scans, CascadesContext context) {
+ StatsCalculator calculator = new StatsCalculator(context);
+ for (LogicalOlapScan scan : scans) {
+ double rowCount = calculator.getOlapTableRowCount(scan);
+ if (rowCount == -1 && ConnectContext.get() != null) {
Review Comment:
the meaning of -1 is explained in comments for getOlapTableRowCount()
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/NereidsPlanner.java:
##########
@@ -288,6 +296,18 @@ private Plan planWithoutLock(
return physicalPlan;
}
+ private List<LogicalOlapScan> getAllOlapScans(Plan plan) {
Review Comment:
the meaning of -1 is explained in comments for getOlapTableRowCount()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2343485941
<details>
<summary>TPC-DS: <b>Total hot run time: 193377 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a656d37037e55e4611f223e3bb02a171a9b99f11, data reload: false
query1 904 383 405 383
query2 6402 1895 1846 1846
query3 8658 191 199 191
query4 32025 23263 23236 23236
query5 3610 484 501 484
query6 262 163 162 162
query7 4160 300 285 285
query8 278 212 229 212
query9 8404 2483 2484 2483
query10 431 268 263 263
query11 16611 15020 15197 15020
query12 142 99 99 99
query13 1523 404 367 367
query14 9284 7290 6882 6882
query15 248 167 174 167
query16 7998 464 454 454
query17 1591 561 543 543
query18 2078 290 280 280
query19 189 150 142 142
query20 120 111 111 111
query21 208 103 102 102
query22 4608 4411 4480 4411
query23 35004 34006 34109 34006
query24 11106 2955 2966 2955
query25 666 396 403 396
query26 1226 164 170 164
query27 2419 301 282 282
query28 6973 2036 2034 2034
query29 803 394 393 393
query30 257 153 154 153
query31 968 757 722 722
query32 107 54 52 52
query33 723 289 289 289
query34 902 477 466 466
query35 864 732 734 732
query36 1060 924 957 924
query37 155 91 87 87
query38 3915 3880 3952 3880
query39 1469 1421 1405 1405
query40 205 117 119 117
query41 52 49 47 47
query42 124 98 96 96
query43 494 463 471 463
query44 1268 751 740 740
query45 195 168 167 167
query46 1110 728 751 728
query47 1853 1764 1811 1764
query48 387 298 289 289
query49 880 443 451 443
query50 838 411 426 411
query51 7009 6810 6878 6810
query52 105 88 88 88
query53 252 186 183 183
query54 878 471 462 462
query55 80 75 76 75
query56 292 281 268 268
query57 1206 1067 1095 1067
query58 248 236 252 236
query59 2974 2737 2782 2737
query60 309 279 283 279
query61 141 125 135 125
query62 778 647 665 647
query63 225 197 189 189
query64 4140 794 746 746
query65 3234 3152 3146 3146
query66 757 352 369 352
query67 15791 15298 15357 15298
query68 3036 868 861 861
query69 427 328 337 328
query70 1149 1169 1173 1169
query71 345 350 347 347
query72 6152 3659 3683 3659
query73 588 572 579 572
query74 8939 8953 8833 8833
query75 3119 2943 3004 2943
query76 1728 852 837 837
query77 443 397 403 397
query78 9338 9342 9548 9342
query79 918 875 861 861
query80 813 815 793 793
query81 452 251 256 251
query82 268 263 264 263
query83 190 189 190 189
query84 194 107 105 105
query85 610 425 390 390
query86 326 335 290 290
query87 4317 4394 4306 4306
query88 4385 4074 4027 4027
query89 369 363 362 362
query90 1008 308 304 304
query91 122 125 117 117
query92 74 76 73 73
query93 930 915 922 915
query94 420 346 334 334
query95 417 409 412 409
query96 469 468 471 468
query97 3089 3081 3100 3081
query98 228 229 223 223
query99 1402 1292 1302 1292
Total cold run time: 280441 ms
Total hot run time: 193377 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2343571069
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2343497960
<details>
<summary>ClickBench: <b>Total hot run time: 32.35 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a656d37037e55e4611f223e3bb02a171a9b99f11, data reload: false
query1 0.05 0.05 0.04
query2 0.08 0.05 0.04
query3 0.23 0.05 0.06
query4 1.67 0.08 0.07
query5 0.53 0.49 0.50
query6 1.14 0.73 0.72
query7 0.02 0.01 0.02
query8 0.06 0.05 0.05
query9 0.54 0.51 0.49
query10 0.55 0.57 0.54
query11 0.15 0.12 0.12
query12 0.15 0.12 0.12
query13 0.60 0.59 0.58
query14 1.39 1.41 1.42
query15 0.86 0.82 0.84
query16 0.38 0.37 0.37
query17 1.07 1.00 1.06
query18 0.17 0.18 0.17
query19 1.92 1.87 1.86
query20 0.02 0.01 0.01
query21 15.40 0.67 0.67
query22 4.18 6.74 2.59
query23 18.34 1.39 1.28
query24 2.11 0.24 0.21
query25 0.16 0.09 0.09
query26 0.26 0.18 0.17
query27 0.08 0.09 0.07
query28 13.15 1.04 1.00
query29 12.59 3.42 3.37
query30 0.24 0.06 0.05
query31 2.89 0.40 0.39
query32 3.23 0.49 0.47
query33 2.98 3.06 3.04
query34 17.02 4.48 4.46
query35 4.47 4.51 4.44
query36 0.66 0.47 0.47
query37 0.17 0.16 0.16
query38 0.18 0.15 0.15
query39 0.04 0.04 0.04
query40 0.16 0.13 0.13
query41 0.11 0.05 0.05
query42 0.06 0.05 0.05
query43 0.04 0.04 0.04
Total cold run time: 110.1 s
Total hot run time: 32.35 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2357411033
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2358381669
<details>
<summary>ClickBench: <b>Total hot run time: 32.18 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 603a31adf3385c8dc6f08873e8eaa597314be95f, data reload: false
query1 0.04 0.05 0.04
query2 0.06 0.03 0.03
query3 0.23 0.06 0.07
query4 1.65 0.09 0.10
query5 0.51 0.51 0.48
query6 1.13 0.72 0.73
query7 0.02 0.02 0.01
query8 0.03 0.03 0.03
query9 0.57 0.49 0.50
query10 0.57 0.57 0.56
query11 0.15 0.11 0.11
query12 0.14 0.10 0.10
query13 0.61 0.60 0.60
query14 3.06 2.99 3.00
query15 0.87 0.81 0.81
query16 0.41 0.39 0.39
query17 1.05 1.07 1.04
query18 0.20 0.20 0.20
query19 1.90 1.82 2.02
query20 0.01 0.01 0.01
query21 15.35 0.58 0.58
query22 2.48 3.13 1.83
query23 17.32 0.84 0.74
query24 2.51 1.25 0.48
query25 0.20 0.13 0.07
query26 0.40 0.14 0.13
query27 0.04 0.05 0.04
query28 11.30 1.09 1.07
query29 12.57 3.31 3.26
query30 0.25 0.06 0.05
query31 2.89 0.40 0.37
query32 3.26 0.46 0.45
query33 3.00 3.05 3.02
query34 17.18 4.40 4.33
query35 4.42 4.41 4.37
query36 0.68 0.50 0.51
query37 0.08 0.06 0.06
query38 0.05 0.03 0.03
query39 0.03 0.03 0.04
query40 0.16 0.13 0.12
query41 0.08 0.02 0.02
query42 0.03 0.02 0.02
query43 0.03 0.03 0.03
Total cold run time: 107.52 s
Total hot run time: 32.18 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2363074847
<details>
<summary>TPC-DS: <b>Total hot run time: 191003 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 61283d9616ad04f43ab7ee55dc8ee7dab59c8165, data reload: false
query1 936 400 403 400
query2 6361 2011 1943 1943
query3 8678 196 199 196
query4 33834 23709 23442 23442
query5 3629 452 458 452
query6 264 170 174 170
query7 4195 303 302 302
query8 299 230 228 228
query9 9422 2662 2656 2656
query10 470 281 279 279
query11 17936 15049 15153 15049
query12 158 96 97 96
query13 1531 430 406 406
query14 8898 7027 7226 7027
query15 244 169 172 169
query16 7318 473 454 454
query17 1621 580 548 548
query18 1900 327 328 327
query19 265 159 149 149
query20 117 126 122 122
query21 214 109 113 109
query22 4899 4643 4456 4456
query23 35979 34568 34362 34362
query24 11234 2926 2906 2906
query25 551 394 407 394
query26 753 157 161 157
query27 2433 282 278 278
query28 7561 2458 2435 2435
query29 663 424 418 418
query30 254 151 156 151
query31 1028 792 811 792
query32 98 54 54 54
query33 736 289 301 289
query34 901 488 498 488
query35 877 748 710 710
query36 1093 957 950 950
query37 144 91 85 85
query38 3987 3926 3852 3852
query39 1510 1437 1410 1410
query40 209 97 97 97
query41 50 49 47 47
query42 113 92 96 92
query43 515 496 470 470
query44 1308 827 799 799
query45 195 162 163 162
query46 1137 746 751 746
query47 1917 1793 1878 1793
query48 463 360 363 360
query49 907 424 404 404
query50 803 412 408 408
query51 7057 7016 6835 6835
query52 102 85 86 85
query53 246 176 173 173
query54 1248 453 444 444
query55 79 73 76 73
query56 290 267 257 257
query57 1184 1124 1097 1097
query58 243 230 232 230
query59 3146 2964 2903 2903
query60 297 264 258 258
query61 104 105 105 105
query62 821 658 678 658
query63 220 184 180 180
query64 4283 648 632 632
query65 3266 3174 3213 3174
query66 867 290 305 290
query67 15889 15733 15559 15559
query68 4838 561 554 554
query69 450 292 295 292
query70 1217 1139 1139 1139
query71 404 273 267 267
query72 6084 3997 4069 3997
query73 775 335 330 330
query74 9944 8921 9029 8921
query75 3368 2635 2631 2631
query76 3040 964 877 877
query77 424 280 284 280
query78 9881 9295 10605 9295
query79 1060 554 549 549
query80 894 430 444 430
query81 556 241 236 236
query82 345 144 140 140
query83 225 141 141 141
query84 250 80 82 80
query85 1354 289 275 275
query86 381 290 300 290
query87 4450 4471 4337 4337
query88 2803 2295 2310 2295
query89 375 280 280 280
query90 1774 184 180 180
query91 179 143 142 142
query92 54 50 46 46
query93 1035 524 525 524
query94 807 287 288 287
query95 346 254 254 254
query96 601 274 284 274
query97 3305 3156 3143 3143
query98 207 202 190 190
query99 1585 1285 1289 1285
Total cold run time: 292961 ms
Total hot run time: 191003 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2365362471
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2365380339
<details>
<summary>ClickBench: <b>Total hot run time: 32.86 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8c688f1e1e8bdcd7ae81bfc8bfdb9212c8622d47, data reload: false
query1 0.05 0.05 0.04
query2 0.07 0.03 0.02
query3 0.23 0.06 0.06
query4 1.66 0.10 0.10
query5 0.52 0.50 0.49
query6 1.13 0.73 0.72
query7 0.02 0.01 0.02
query8 0.04 0.03 0.03
query9 0.57 0.51 0.50
query10 0.54 0.56 0.55
query11 0.14 0.11 0.11
query12 0.14 0.11 0.11
query13 0.61 0.59 0.59
query14 3.10 2.97 2.96
query15 0.89 0.81 0.82
query16 0.39 0.39 0.39
query17 1.06 1.05 1.00
query18 0.19 0.20 0.19
query19 1.93 1.76 2.00
query20 0.01 0.01 0.01
query21 15.35 0.56 0.58
query22 2.30 2.15 2.20
query23 17.29 0.80 0.82
query24 2.49 0.94 0.86
query25 0.22 0.31 0.08
query26 0.23 0.13 0.13
query27 0.04 0.04 0.04
query28 11.24 1.09 1.06
query29 12.58 3.35 3.33
query30 0.24 0.05 0.06
query31 2.89 0.38 0.37
query32 3.27 0.48 0.47
query33 2.98 2.99 3.05
query34 16.99 4.35 4.38
query35 4.36 4.40 4.42
query36 0.68 0.48 0.48
query37 0.08 0.06 0.06
query38 0.05 0.03 0.04
query39 0.03 0.02 0.02
query40 0.16 0.12 0.12
query41 0.07 0.02 0.02
query42 0.03 0.02 0.02
query43 0.03 0.02 0.03
Total cold run time: 106.89 s
Total hot run time: 32.86 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2346251701
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2367126808
<details>
<summary>TPC-H: <b>Total hot run time: 41350 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b09ea841c140f2667cdcbf613ade46ead076fe96, data reload: false
------ Round 1 ----------------------------------
q1 17567 7352 7234 7234
q2 2018 284 283 283
q3 12263 1030 1216 1030
q4 10563 770 697 697
q5 7755 3047 3035 3035
q6 240 156 149 149
q7 1026 619 595 595
q8 9424 2011 2005 2005
q9 6869 6436 6384 6384
q10 6997 2242 2328 2242
q11 431 240 244 240
q12 407 216 208 208
q13 17779 2967 2974 2967
q14 233 207 208 207
q15 593 534 522 522
q16 688 613 616 613
q17 960 820 789 789
q18 7234 6749 6674 6674
q19 1406 991 1029 991
q20 579 290 289 289
q21 3976 3296 3198 3198
q22 1127 998 1004 998
Total cold run time: 110135 ms
Total hot run time: 41350 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7251 7220 7197 7197
q2 331 230 224 224
q3 3021 2956 2907 2907
q4 2035 1756 1813 1756
q5 5607 5574 5628 5574
q6 232 140 145 140
q7 2149 1780 1787 1780
q8 3317 3399 3431 3399
q9 8719 8887 8782 8782
q10 3513 3389 3467 3389
q11 589 490 477 477
q12 787 588 609 588
q13 7794 3190 3159 3159
q14 290 274 271 271
q15 576 545 505 505
q16 714 702 671 671
q17 1779 1560 1597 1560
q18 8180 7766 7769 7766
q19 1743 1627 1580 1580
q20 2133 1869 1900 1869
q21 5600 5060 5510 5060
q22 1153 1049 1056 1049
Total cold run time: 67513 ms
Total hot run time: 59703 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on code in PR #40349:
URL: https://github.com/apache/doris/pull/40349#discussion_r1768043925
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/stats/StatsCalculator.java:
##########
@@ -205,6 +210,23 @@ public Map<String, ColumnStatistic> getTotalColumnStatisticMap() {
return totalColumnStatisticMap;
}
+ /**
+ * disable join reorder if any table row count is not available.
+ */
+ public static void disableJoinReorderIfTableRowCountNotAvailable(
+ List<LogicalOlapScan> scans, CascadesContext context) {
+ StatsCalculator calculator = new StatsCalculator(context);
+ for (LogicalOlapScan scan : scans) {
+ double rowCount = calculator.getOlapTableRowCount(scan);
+ if (rowCount == -1 && ConnectContext.get() != null) {
Review Comment:
this is the return value of OlapTable.getRowCountForIndex().
explained the "-1" at the spec of OlapTable.getRowCountForIndex()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2362591572
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2359849279
<details>
<summary>TPC-H: <b>Total hot run time: 41605 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a5d7a05f22338b4b9214968e0f3c040ac6c67485, data reload: false
------ Round 1 ----------------------------------
q1 17595 7457 7330 7330
q2 1999 290 287 287
q3 12230 1074 1198 1074
q4 10554 725 720 720
q5 7764 3122 3073 3073
q6 242 153 152 152
q7 1030 616 611 611
q8 9453 2086 2032 2032
q9 6815 6474 6381 6381
q10 6983 2289 2307 2289
q11 425 250 251 250
q12 402 215 215 215
q13 17823 3019 3008 3008
q14 265 210 224 210
q15 580 532 504 504
q16 687 594 631 594
q17 985 855 831 831
q18 7309 6660 6658 6658
q19 1399 1014 999 999
q20 564 283 280 280
q21 4073 3105 3172 3105
q22 1136 1002 1015 1002
Total cold run time: 110313 ms
Total hot run time: 41605 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7326 7248 7219 7219
q2 343 253 245 245
q3 3034 3003 2941 2941
q4 2095 1842 1822 1822
q5 5698 5596 5722 5596
q6 235 145 150 145
q7 2191 1792 1819 1792
q8 3328 3475 3438 3438
q9 8790 8877 8875 8875
q10 3413 3448 3482 3448
q11 583 484 480 480
q12 836 646 622 622
q13 9902 3193 3162 3162
q14 322 275 307 275
q15 575 516 548 516
q16 738 683 658 658
q17 1836 1595 1609 1595
q18 8124 7778 7896 7778
q19 1743 1596 1503 1503
q20 2124 1904 1879 1879
q21 5531 5352 5519 5352
q22 1143 1070 1101 1070
Total cold run time: 69910 ms
Total hot run time: 60411 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2367284808
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2332101787
<details>
<summary>TPC-H: <b>Total hot run time: 38406 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4b3661cd3256aee78fe50faf5ad9461dd4d1d253, data reload: false
------ Round 1 ----------------------------------
q1 18159 4531 4346 4346
q2 2040 190 190 190
q3 11707 960 1167 960
q4 10296 761 762 761
q5 7749 2891 2809 2809
q6 233 142 145 142
q7 964 614 593 593
q8 9333 2114 2104 2104
q9 7070 6566 6610 6566
q10 7056 2200 2261 2200
q11 480 241 239 239
q12 392 227 228 227
q13 17754 3079 3091 3079
q14 292 244 245 244
q15 542 495 475 475
q16 561 450 429 429
q17 1005 701 786 701
q18 7366 6888 6962 6888
q19 1385 1046 1095 1046
q20 698 342 328 328
q21 4003 3082 3135 3082
q22 1137 1008 997 997
Total cold run time: 110222 ms
Total hot run time: 38406 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4436 4345 4310 4310
q2 389 288 262 262
q3 2944 2670 2632 2632
q4 1940 1710 1707 1707
q5 5628 5683 5732 5683
q6 238 138 144 138
q7 2257 1863 1876 1863
q8 3350 3445 3486 3445
q9 8853 8827 8854 8827
q10 3627 3394 3410 3394
q11 598 513 505 505
q12 820 645 649 645
q13 13945 3330 3180 3180
q14 326 297 278 278
q15 541 474 506 474
q16 543 503 488 488
q17 1861 1555 1524 1524
q18 8222 7819 7937 7819
q19 1761 1640 1533 1533
q20 2155 1966 1909 1909
q21 5780 5717 5549 5549
q22 1194 1097 1041 1041
Total cold run time: 71408 ms
Total hot run time: 57206 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2332138558
<details>
<summary>ClickBench: <b>Total hot run time: 30.75 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4b3661cd3256aee78fe50faf5ad9461dd4d1d253, data reload: false
query1 0.05 0.05 0.04
query2 0.08 0.04 0.04
query3 0.23 0.05 0.05
query4 1.67 0.07 0.07
query5 0.51 0.50 0.51
query6 1.13 0.74 0.73
query7 0.02 0.01 0.01
query8 0.06 0.05 0.04
query9 0.53 0.49 0.47
query10 0.53 0.56 0.52
query11 0.16 0.12 0.12
query12 0.15 0.12 0.13
query13 0.60 0.59 0.59
query14 1.38 1.41 1.43
query15 0.84 0.83 0.82
query16 0.38 0.36 0.38
query17 1.08 1.01 0.97
query18 0.21 0.20 0.20
query19 1.88 1.78 1.71
query20 0.01 0.00 0.01
query21 15.40 0.66 0.66
query22 4.30 7.91 1.43
query23 18.28 1.32 1.23
query24 2.16 0.22 0.22
query25 0.16 0.08 0.08
query26 0.28 0.18 0.17
query27 0.08 0.07 0.08
query28 13.20 1.03 1.00
query29 12.63 3.36 3.27
query30 0.24 0.05 0.06
query31 2.87 0.42 0.41
query32 3.23 0.49 0.48
query33 2.99 2.98 2.99
query34 17.19 4.46 4.43
query35 4.52 4.54 4.45
query36 0.65 0.46 0.48
query37 0.19 0.16 0.15
query38 0.16 0.15 0.14
query39 0.06 0.03 0.04
query40 0.16 0.13 0.13
query41 0.09 0.05 0.05
query42 0.06 0.05 0.05
query43 0.04 0.04 0.04
Total cold run time: 110.44 s
Total hot run time: 30.75 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2345150616
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2343460523
<details>
<summary>TPC-H: <b>Total hot run time: 38143 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a656d37037e55e4611f223e3bb02a171a9b99f11, data reload: false
------ Round 1 ----------------------------------
q1 17622 4737 4350 4350
q2 1983 331 311 311
q3 12038 954 1154 954
q4 10529 839 729 729
q5 7774 2897 2815 2815
q6 225 136 134 134
q7 952 591 600 591
q8 9326 2117 2037 2037
q9 6982 6547 6544 6544
q10 7032 2273 2216 2216
q11 457 243 248 243
q12 406 230 223 223
q13 17986 3089 3105 3089
q14 284 232 254 232
q15 537 502 489 489
q16 527 423 440 423
q17 977 661 645 645
q18 7397 6799 6921 6799
q19 1390 1077 1009 1009
q20 714 338 330 330
q21 3936 3067 2972 2972
q22 1083 1021 1008 1008
Total cold run time: 110157 ms
Total hot run time: 38143 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4392 4315 4328 4315
q2 384 288 284 284
q3 2918 2647 2687 2647
q4 1970 1723 1677 1677
q5 5674 5720 5767 5720
q6 225 136 142 136
q7 2226 1865 1830 1830
q8 3353 3437 3507 3437
q9 8820 8851 8857 8851
q10 3621 3433 3371 3371
q11 621 538 510 510
q12 850 648 683 648
q13 14919 3269 3312 3269
q14 315 283 293 283
q15 540 504 511 504
q16 540 519 491 491
q17 1869 1561 1553 1553
q18 8328 8000 8026 8000
q19 1780 1632 1546 1546
q20 2182 1891 1950 1891
q21 5785 5397 5606 5397
q22 1111 1064 1058 1058
Total cold run time: 72423 ms
Total hot run time: 57418 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2343696055
<details>
<summary>TPC-DS: <b>Total hot run time: 193048 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 12fd24384b9ed2c7fd8225a861122ea330cf5b12, data reload: false
query1 898 399 394 394
query2 6168 1886 1804 1804
query3 8671 194 200 194
query4 33362 23330 23207 23207
query5 5300 514 508 508
query6 277 162 155 155
query7 4163 291 286 286
query8 288 233 232 232
query9 8485 2485 2486 2485
query10 490 269 277 269
query11 17843 15056 15356 15056
query12 154 123 98 98
query13 1572 389 368 368
query14 10561 6788 7195 6788
query15 256 162 173 162
query16 7856 458 432 432
query17 1579 568 557 557
query18 2039 348 287 287
query19 330 158 153 153
query20 117 112 122 112
query21 210 109 102 102
query22 4797 4310 4321 4310
query23 34521 35710 34212 34212
query24 10909 2927 2956 2927
query25 559 404 405 404
query26 716 156 156 156
query27 2169 299 284 284
query28 6251 2109 2112 2109
query29 698 441 417 417
query30 199 153 160 153
query31 1012 823 796 796
query32 99 54 55 54
query33 689 293 296 293
query34 908 480 466 466
query35 856 716 731 716
query36 1131 907 918 907
query37 142 83 90 83
query38 4006 3926 3907 3907
query39 1437 1403 1403 1403
query40 200 110 115 110
query41 47 46 44 44
query42 113 95 92 92
query43 506 468 459 459
query44 1156 745 734 734
query45 192 177 171 171
query46 1131 750 751 750
query47 1887 1797 1828 1797
query48 364 291 283 283
query49 810 448 449 448
query50 835 408 407 407
query51 7043 6783 6833 6783
query52 104 86 90 86
query53 251 181 178 178
query54 891 469 454 454
query55 76 74 75 74
query56 300 266 260 260
query57 1175 1099 1056 1056
query58 245 237 229 229
query59 2956 2678 2724 2678
query60 304 272 282 272
query61 125 119 118 118
query62 794 665 680 665
query63 232 194 194 194
query64 3881 804 768 768
query65 3305 3207 3185 3185
query66 697 337 346 337
query67 15749 15435 15393 15393
query68 3025 555 567 555
query69 403 280 275 275
query70 1179 1033 1070 1033
query71 339 273 270 270
query72 6167 3982 4068 3982
query73 748 313 322 313
query74 9260 8924 9081 8924
query75 3406 2739 2640 2640
query76 1705 996 997 996
query77 442 317 315 315
query78 9956 10001 9302 9302
query79 1515 868 862 862
query80 993 823 815 815
query81 592 256 263 256
query82 681 265 264 264
query83 231 187 186 186
query84 237 108 104 104
query85 692 449 379 379
query86 426 307 325 307
query87 4485 4298 4345 4298
query88 4509 4100 4076 4076
query89 396 366 363 363
query90 1684 318 309 309
query91 121 121 121 121
query92 83 74 74 74
query93 1338 923 910 910
query94 707 346 352 346
query95 452 409 423 409
query96 470 470 480 470
query97 3156 3130 3113 3113
query98 232 237 231 231
query99 1477 1279 1303 1279
Total cold run time: 288194 ms
Total hot run time: 193048 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2343662586
<details>
<summary>TPC-H: <b>Total hot run time: 37910 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 12fd24384b9ed2c7fd8225a861122ea330cf5b12, data reload: false
------ Round 1 ----------------------------------
q1 17627 4796 4337 4337
q2 1997 307 302 302
q3 12136 972 1066 972
q4 10520 663 755 663
q5 7786 2885 2857 2857
q6 230 139 138 138
q7 963 597 604 597
q8 9343 2072 2066 2066
q9 7173 6565 6523 6523
q10 7048 2233 2188 2188
q11 455 242 245 242
q12 406 228 227 227
q13 19014 3077 3139 3077
q14 276 237 254 237
q15 555 492 492 492
q16 540 475 426 426
q17 1005 724 712 712
q18 7403 6958 6921 6921
q19 1444 794 783 783
q20 776 711 704 704
q21 3829 2390 2481 2390
q22 1093 1056 1057 1056
Total cold run time: 111619 ms
Total hot run time: 37910 ms
----- Round 2, with runtime_filter_mode=off -----
q1 3326 3340 3318 3318
q2 316 297 294 294
q3 2215 2219 2180 2180
q4 1348 1336 1354 1336
q5 4383 4551 4614 4551
q6 292 291 300 291
q7 1942 1894 1825 1825
q8 2423 2376 2384 2376
q9 7321 7329 7356 7329
q10 2912 2966 2865 2865
q11 542 545 530 530
q12 798 731 777 731
q13 15282 3646 3580 3580
q14 301 300 300 300
q15 551 533 519 519
q16 497 492 521 492
q17 1258 1194 1205 1194
q18 7064 7152 7101 7101
q19 967 954 977 954
q20 2025 1978 1961 1961
q21 5161 5260 5176 5176
q22 1141 1142 1140 1140
Total cold run time: 62065 ms
Total hot run time: 50043 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2346314745
<details>
<summary>TPC-H: <b>Total hot run time: 42960 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4c5f044ea0e8082fff517399b66309a70ce2c13d, data reload: false
------ Round 1 ----------------------------------
q1 17621 7376 7265 7265
q2 2046 189 183 183
q3 10464 1399 1344 1344
q4 10593 953 1009 953
q5 8084 3261 3144 3144
q6 245 155 159 155
q7 1040 647 646 646
q8 9455 2004 2031 2004
q9 6827 6291 6297 6291
q10 7033 2508 2514 2508
q11 443 256 260 256
q12 422 233 231 231
q13 17758 3025 3039 3025
q14 288 253 253 253
q15 585 542 515 515
q16 530 437 421 421
q17 1001 948 952 948
q18 7433 6713 6782 6713
q19 1379 1247 1235 1235
q20 609 357 334 334
q21 3898 3544 3548 3544
q22 1099 992 1007 992
Total cold run time: 108853 ms
Total hot run time: 42960 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7235 7187 7168 7168
q2 341 230 234 230
q3 2918 2894 2880 2880
q4 1956 1986 1959 1959
q5 5466 5457 5422 5422
q6 234 147 150 147
q7 2066 1765 1689 1689
q8 3246 3322 3341 3322
q9 8450 8446 8449 8446
q10 3434 3445 3432 3432
q11 583 463 474 463
q12 779 585 591 585
q13 5489 3066 3041 3041
q14 313 268 276 268
q15 586 527 532 527
q16 500 475 461 461
q17 1757 1721 1757 1721
q18 8074 7722 7659 7659
q19 1730 1713 1721 1713
q20 2042 1822 1823 1822
q21 5620 5315 5534 5315
q22 1097 1001 983 983
Total cold run time: 63916 ms
Total hot run time: 59253 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2345655547
<details>
<summary>ClickBench: <b>Total hot run time: 30.85 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4c5f044ea0e8082fff517399b66309a70ce2c13d, data reload: false
query1 0.04 0.04 0.04
query2 0.08 0.05 0.04
query3 0.23 0.04 0.05
query4 1.66 0.07 0.07
query5 0.50 0.51 0.52
query6 1.13 0.73 0.73
query7 0.02 0.01 0.01
query8 0.06 0.05 0.04
query9 0.57 0.52 0.53
query10 0.60 0.57 0.56
query11 0.17 0.12 0.12
query12 0.15 0.13 0.12
query13 0.63 0.61 0.61
query14 1.45 1.44 1.45
query15 0.91 0.88 0.88
query16 0.37 0.37 0.37
query17 1.03 1.04 1.00
query18 0.17 0.16 0.16
query19 2.00 1.84 1.84
query20 0.01 0.01 0.01
query21 15.41 0.67 0.68
query22 4.15 7.59 1.03
query23 17.87 1.40 1.42
query24 2.28 0.22 0.22
query25 0.20 0.09 0.08
query26 0.29 0.19 0.17
query27 0.08 0.07 0.08
query28 13.17 1.00 0.98
query29 12.55 3.37 3.34
query30 0.24 0.05 0.06
query31 2.85 0.42 0.41
query32 3.23 0.50 0.50
query33 3.01 3.04 3.08
query34 15.46 4.33 4.32
query35 4.35 4.36 4.38
query36 0.68 0.50 0.49
query37 0.19 0.16 0.16
query38 0.17 0.15 0.14
query39 0.04 0.04 0.04
query40 0.17 0.13 0.14
query41 0.10 0.05 0.05
query42 0.06 0.05 0.06
query43 0.05 0.04 0.04
Total cold run time: 108.38 s
Total hot run time: 30.85 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2367101984
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2363087330
<details>
<summary>ClickBench: <b>Total hot run time: 32.22 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 61283d9616ad04f43ab7ee55dc8ee7dab59c8165, data reload: false
query1 0.05 0.04 0.04
query2 0.07 0.03 0.03
query3 0.23 0.06 0.06
query4 1.66 0.10 0.10
query5 0.51 0.49 0.50
query6 1.12 0.72 0.73
query7 0.02 0.01 0.02
query8 0.04 0.03 0.03
query9 0.55 0.50 0.49
query10 0.54 0.56 0.55
query11 0.14 0.11 0.10
query12 0.14 0.11 0.11
query13 0.61 0.59 0.60
query14 2.95 3.07 3.01
query15 0.89 0.81 0.83
query16 0.38 0.39 0.38
query17 0.98 0.97 1.06
query18 0.20 0.19 0.20
query19 1.99 1.88 2.01
query20 0.01 0.00 0.01
query21 15.37 0.59 0.55
query22 2.58 1.78 2.22
query23 17.38 0.84 0.65
query24 2.57 1.13 0.75
query25 0.21 0.32 0.04
query26 0.30 0.13 0.13
query27 0.03 0.04 0.04
query28 11.10 1.09 1.07
query29 12.78 3.24 3.21
query30 0.25 0.05 0.06
query31 2.89 0.38 0.38
query32 3.28 0.45 0.45
query33 2.97 3.05 2.96
query34 17.01 4.40 4.42
query35 4.45 4.42 4.42
query36 0.67 0.49 0.47
query37 0.08 0.06 0.06
query38 0.05 0.04 0.04
query39 0.03 0.02 0.03
query40 0.16 0.12 0.12
query41 0.07 0.02 0.02
query42 0.03 0.02 0.02
query43 0.04 0.03 0.03
Total cold run time: 107.38 s
Total hot run time: 32.22 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2367182042
run cloud_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2332126957
<details>
<summary>TPC-DS: <b>Total hot run time: 195568 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4b3661cd3256aee78fe50faf5ad9461dd4d1d253, data reload: false
query1 949 403 421 403
query2 12123 8486 8618 8486
query3 10615 220 220 220
query4 32852 23314 23238 23238
query5 3569 497 504 497
query6 246 167 166 166
query7 3942 294 303 294
query8 281 218 214 214
query9 8533 2492 2499 2492
query10 420 282 281 281
query11 17586 15150 15159 15150
query12 129 102 99 99
query13 1522 375 376 375
query14 8512 7342 7198 7198
query15 223 184 172 172
query16 7897 497 489 489
query17 1562 606 568 568
query18 2164 301 289 289
query19 202 145 141 141
query20 118 106 108 106
query21 204 106 108 106
query22 4493 4133 4083 4083
query23 33936 33859 33672 33672
query24 11245 2930 2926 2926
query25 650 436 440 436
query26 1246 189 160 160
query27 2696 296 297 296
query28 7342 2126 2113 2113
query29 850 433 426 426
query30 280 161 160 160
query31 962 767 775 767
query32 98 53 56 53
query33 748 296 304 296
query34 928 498 509 498
query35 885 742 721 721
query36 1100 1004 990 990
query37 158 89 88 88
query38 4156 3824 3851 3824
query39 1420 1380 1373 1373
query40 201 112 112 112
query41 46 45 43 43
query42 110 94 95 94
query43 489 465 471 465
query44 1218 755 736 736
query45 193 161 164 161
query46 1091 766 795 766
query47 1866 1809 1782 1782
query48 384 298 287 287
query49 858 459 452 452
query50 835 436 417 417
query51 6938 6835 6826 6826
query52 99 87 86 86
query53 247 185 185 185
query54 941 460 471 460
query55 78 77 79 77
query56 277 257 265 257
query57 1211 1125 1093 1093
query58 265 231 243 231
query59 2990 2801 2734 2734
query60 300 266 266 266
query61 101 96 98 96
query62 816 656 662 656
query63 220 190 185 185
query64 5165 662 644 644
query65 3194 3169 3157 3157
query66 1301 364 341 341
query67 15583 15177 15343 15177
query68 3464 626 595 595
query69 399 279 285 279
query70 1166 1105 1133 1105
query71 344 276 278 276
query72 6271 3979 4045 3979
query73 751 324 338 324
query74 9209 8971 8932 8932
query75 3377 2712 2703 2703
query76 1946 998 1017 998
query77 489 328 319 319
query78 9715 9157 9305 9157
query79 1046 541 533 533
query80 706 522 517 517
query81 527 233 227 227
query82 234 152 147 147
query83 169 153 146 146
query84 229 78 74 74
query85 693 291 343 291
query86 307 299 290 290
query87 4396 4161 4362 4161
query88 2902 2347 2349 2347
query89 374 282 287 282
query90 1868 193 193 193
query91 128 101 102 101
query92 60 49 51 49
query93 1042 527 541 527
query94 713 308 309 308
query95 348 264 256 256
query96 592 275 266 266
query97 3221 3063 3105 3063
query98 229 199 202 199
query99 1532 1272 1311 1272
Total cold run time: 292306 ms
Total hot run time: 195568 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2327862688
<details>
<summary>ClickBench: <b>Total hot run time: 31.6 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4b3661cd3256aee78fe50faf5ad9461dd4d1d253, data reload: false
query1 0.05 0.05 0.04
query2 0.08 0.03 0.04
query3 0.22 0.05 0.06
query4 1.68 0.09 0.09
query5 0.52 0.49 0.51
query6 1.13 0.75 0.73
query7 0.02 0.02 0.01
query8 0.05 0.05 0.05
query9 0.54 0.48 0.49
query10 0.54 0.55 0.56
query11 0.16 0.12 0.12
query12 0.16 0.13 0.13
query13 0.62 0.59 0.60
query14 2.11 2.04 2.04
query15 0.84 0.85 0.82
query16 0.37 0.38 0.37
query17 1.06 1.02 1.06
query18 0.22 0.21 0.21
query19 1.87 1.79 1.89
query20 0.01 0.01 0.01
query21 15.40 0.67 0.65
query22 3.85 7.98 1.45
query23 18.28 1.42 1.21
query24 2.12 0.21 0.20
query25 0.15 0.07 0.08
query26 0.27 0.18 0.18
query27 0.08 0.08 0.08
query28 13.27 1.02 1.00
query29 12.62 3.35 3.33
query30 0.24 0.06 0.06
query31 2.87 0.39 0.39
query32 3.25 0.49 0.47
query33 3.04 2.97 3.00
query34 17.20 4.45 4.39
query35 4.43 4.51 4.45
query36 0.66 0.50 0.48
query37 0.18 0.15 0.16
query38 0.15 0.15 0.15
query39 0.05 0.04 0.04
query40 0.15 0.13 0.12
query41 0.09 0.05 0.05
query42 0.06 0.06 0.05
query43 0.05 0.04 0.05
Total cold run time: 110.71 s
Total hot run time: 31.6 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) update how to table row count, and disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2327766240
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) update how to table row count, and disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2327805157
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2365211561
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly merged PR #40349:
URL: https://github.com/apache/doris/pull/40349
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2365379253
<details>
<summary>TPC-DS: <b>Total hot run time: 192623 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8c688f1e1e8bdcd7ae81bfc8bfdb9212c8622d47, data reload: false
query1 935 388 422 388
query2 6225 2048 1950 1950
query3 8682 201 204 201
query4 33906 23618 23413 23413
query5 3445 462 469 462
query6 272 182 176 176
query7 4196 309 314 309
query8 299 237 234 234
query9 9510 2681 2679 2679
query10 458 286 275 275
query11 18028 15357 15276 15276
query12 156 99 96 96
query13 1543 425 412 412
query14 9645 7417 7356 7356
query15 251 166 170 166
query16 8068 458 447 447
query17 1604 589 575 575
query18 2168 314 302 302
query19 361 150 161 150
query20 123 113 141 113
query21 210 104 106 104
query22 4936 4621 4632 4621
query23 35008 35108 34730 34730
query24 11032 3036 2946 2946
query25 546 406 409 406
query26 753 158 166 158
query27 2242 284 284 284
query28 7806 2468 2472 2468
query29 670 432 427 427
query30 260 158 162 158
query31 1026 766 787 766
query32 104 53 54 53
query33 741 311 330 311
query34 918 503 492 492
query35 855 739 719 719
query36 1104 925 957 925
query37 155 87 85 85
query38 4050 3884 3900 3884
query39 1584 1445 1421 1421
query40 209 98 98 98
query41 52 49 52 49
query42 119 96 95 95
query43 510 484 491 484
query44 1229 830 815 815
query45 196 163 162 162
query46 1166 814 755 755
query47 1898 1790 1831 1790
query48 470 377 365 365
query49 918 423 403 403
query50 828 404 417 404
query51 6964 6988 6996 6988
query52 95 88 84 84
query53 256 182 175 175
query54 1238 460 447 447
query55 77 76 75 75
query56 284 251 272 251
query57 1206 1087 1092 1087
query58 237 231 248 231
query59 3063 3018 3028 3018
query60 307 270 274 270
query61 105 104 102 102
query62 815 658 677 658
query63 224 183 185 183
query64 3944 648 642 642
query65 3274 3203 3233 3203
query66 830 298 301 298
query67 15771 15677 15539 15539
query68 4711 600 564 564
query69 534 297 296 296
query70 1210 1129 1142 1129
query71 441 268 277 268
query72 7210 4065 4038 4038
query73 769 325 334 325
query74 10380 9023 8938 8938
query75 3583 2684 2690 2684
query76 3157 940 911 911
query77 628 300 292 292
query78 9902 9116 9200 9116
query79 1608 556 567 556
query80 1039 461 446 446
query81 588 245 242 242
query82 720 157 144 144
query83 296 138 140 138
query84 274 81 80 80
query85 1376 297 277 277
query86 417 283 298 283
query87 4389 4328 4444 4328
query88 3311 2322 2334 2322
query89 405 288 287 287
query90 1857 187 188 187
query91 181 148 144 144
query92 61 48 50 48
query93 2219 547 541 541
query94 891 297 302 297
query95 357 257 251 251
query96 627 274 279 274
query97 3276 3112 3130 3112
query98 226 194 198 194
query99 1515 1302 1308 1302
Total cold run time: 298318 ms
Total hot run time: 192623 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2365375927
<details>
<summary>TPC-H: <b>Total hot run time: 41535 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8c688f1e1e8bdcd7ae81bfc8bfdb9212c8622d47, data reload: false
------ Round 1 ----------------------------------
q1 17576 7390 7304 7304
q2 2009 299 283 283
q3 12177 1058 1240 1058
q4 10563 756 756 756
q5 7751 3050 3053 3050
q6 234 151 148 148
q7 1014 601 598 598
q8 9421 2051 2029 2029
q9 7530 6466 6401 6401
q10 7010 2270 2280 2270
q11 434 249 251 249
q12 415 214 221 214
q13 17772 2991 2996 2991
q14 242 221 217 217
q15 587 519 505 505
q16 694 633 606 606
q17 982 828 824 824
q18 7204 6773 6719 6719
q19 1398 1078 1033 1033
q20 586 290 293 290
q21 3935 3305 3019 3019
q22 1112 971 985 971
Total cold run time: 110646 ms
Total hot run time: 41535 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7305 7253 7256 7253
q2 331 242 244 242
q3 3015 3006 2913 2913
q4 2123 1827 1736 1736
q5 5610 5600 5742 5600
q6 234 153 149 149
q7 2251 1835 1837 1835
q8 3347 3500 3466 3466
q9 8960 8879 8808 8808
q10 3453 3500 3487 3487
q11 574 482 474 474
q12 825 611 631 611
q13 9263 3162 3155 3155
q14 309 297 260 260
q15 584 534 540 534
q16 728 678 682 678
q17 1806 1609 1573 1573
q18 8183 7684 7795 7684
q19 1737 1604 1583 1583
q20 2126 1881 1926 1881
q21 5386 5433 5344 5344
q22 1118 1048 1033 1033
Total cold run time: 69268 ms
Total hot run time: 60299 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2362648085
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2345508159
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2343709286
<details>
<summary>ClickBench: <b>Total hot run time: 32.2 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 12fd24384b9ed2c7fd8225a861122ea330cf5b12, data reload: false
query1 0.05 0.04 0.04
query2 0.09 0.04 0.04
query3 0.22 0.05 0.05
query4 1.66 0.07 0.08
query5 0.50 0.50 0.50
query6 1.12 0.72 0.73
query7 0.02 0.02 0.01
query8 0.06 0.05 0.05
query9 0.54 0.49 0.48
query10 0.54 0.54 0.54
query11 0.15 0.11 0.12
query12 0.15 0.13 0.13
query13 0.61 0.60 0.58
query14 1.37 1.42 1.45
query15 0.86 0.83 0.82
query16 0.37 0.36 0.38
query17 1.07 0.98 1.02
query18 0.18 0.18 0.18
query19 1.97 1.76 1.88
query20 0.01 0.01 0.01
query21 15.41 0.68 0.67
query22 4.22 6.77 2.48
query23 18.26 1.42 1.41
query24 2.13 0.23 0.22
query25 0.16 0.08 0.07
query26 0.27 0.18 0.17
query27 0.08 0.07 0.08
query28 13.22 1.03 1.01
query29 12.62 3.34 3.32
query30 0.25 0.05 0.05
query31 2.89 0.40 0.40
query32 3.23 0.48 0.47
query33 2.94 3.07 3.05
query34 17.01 4.44 4.44
query35 4.47 4.45 4.47
query36 0.67 0.49 0.47
query37 0.18 0.16 0.15
query38 0.15 0.14 0.15
query39 0.05 0.04 0.04
query40 0.16 0.13 0.13
query41 0.09 0.05 0.05
query42 0.06 0.05 0.05
query43 0.05 0.04 0.04
Total cold run time: 110.11 s
Total hot run time: 32.2 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2357578722
run cloud_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2358344469
<details>
<summary>TPC-H: <b>Total hot run time: 41621 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 603a31adf3385c8dc6f08873e8eaa597314be95f, data reload: false
------ Round 1 ----------------------------------
q1 17590 7342 7290 7290
q2 2015 285 277 277
q3 12195 1070 1176 1070
q4 10563 797 721 721
q5 7742 3051 3057 3051
q6 236 151 155 151
q7 1012 605 616 605
q8 9436 2058 2049 2049
q9 6774 6427 6383 6383
q10 6989 2265 2316 2265
q11 426 254 259 254
q12 411 225 218 218
q13 17779 3035 3010 3010
q14 250 215 219 215
q15 581 516 517 516
q16 652 627 605 605
q17 960 843 807 807
q18 7285 6592 6712 6592
q19 1404 1037 1101 1037
q20 585 284 277 277
q21 4002 3312 3232 3232
q22 1113 996 1010 996
Total cold run time: 110000 ms
Total hot run time: 41621 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7323 7292 7289 7289
q2 320 237 239 237
q3 3075 2982 2873 2873
q4 2103 1759 1812 1759
q5 5634 5549 5650 5549
q6 239 149 148 148
q7 2192 1785 1792 1785
q8 3301 3438 3395 3395
q9 8769 8930 8813 8813
q10 3511 3412 3495 3412
q11 584 485 469 469
q12 823 625 610 610
q13 9637 3132 3177 3132
q14 299 287 276 276
q15 573 548 509 509
q16 698 667 678 667
q17 1819 1593 1589 1589
q18 8356 7634 7834 7634
q19 1735 1619 1658 1619
q20 2193 1913 1898 1898
q21 5527 5153 5265 5153
q22 1170 1033 1057 1033
Total cold run time: 69881 ms
Total hot run time: 59849 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2338098118
<details>
<summary>ClickBench: <b>Total hot run time: 31.91 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2c9998ac9f7f66ddd55e69fbe64ec9ccd434f6a0, data reload: false
query1 0.05 0.04 0.04
query2 0.08 0.04 0.04
query3 0.22 0.05 0.05
query4 1.68 0.08 0.08
query5 0.51 0.50 0.50
query6 1.14 0.75 0.73
query7 0.02 0.02 0.01
query8 0.05 0.05 0.05
query9 0.54 0.49 0.49
query10 0.53 0.55 0.54
query11 0.15 0.12 0.12
query12 0.15 0.12 0.12
query13 0.60 0.59 0.59
query14 1.34 1.41 1.42
query15 0.87 0.82 0.80
query16 0.36 0.38 0.37
query17 1.06 1.02 1.06
query18 0.21 0.21 0.20
query19 1.95 1.84 1.79
query20 0.01 0.01 0.01
query21 15.39 0.68 0.67
query22 4.37 6.62 2.23
query23 18.31 1.44 1.42
query24 2.05 0.26 0.22
query25 0.15 0.08 0.08
query26 0.27 0.18 0.18
query27 0.07 0.08 0.08
query28 13.23 1.02 1.00
query29 12.64 3.35 3.35
query30 0.25 0.06 0.05
query31 2.89 0.40 0.39
query32 3.26 0.48 0.47
query33 2.94 3.01 3.03
query34 17.06 4.40 4.34
query35 4.41 4.46 4.38
query36 0.66 0.48 0.49
query37 0.19 0.15 0.15
query38 0.15 0.16 0.15
query39 0.04 0.04 0.04
query40 0.15 0.12 0.12
query41 0.09 0.05 0.05
query42 0.07 0.05 0.05
query43 0.04 0.05 0.04
Total cold run time: 110.2 s
Total hot run time: 31.91 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2327849603
<details>
<summary>TPC-H: <b>Total hot run time: 38114 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4b3661cd3256aee78fe50faf5ad9461dd4d1d253, data reload: false
------ Round 1 ----------------------------------
q1 17630 4503 4338 4338
q2 2030 178 183 178
q3 11879 946 1182 946
q4 10515 779 757 757
q5 7765 2854 2810 2810
q6 227 139 135 135
q7 965 618 612 612
q8 9328 2084 2058 2058
q9 7116 6535 6571 6535
q10 6985 2241 2156 2156
q11 470 241 241 241
q12 398 227 227 227
q13 17758 3099 3063 3063
q14 280 242 223 223
q15 529 487 500 487
q16 596 526 517 517
q17 983 683 707 683
q18 7393 6929 6991 6929
q19 1401 1024 982 982
q20 678 340 332 332
q21 3960 3149 2919 2919
q22 1104 986 1011 986
Total cold run time: 109990 ms
Total hot run time: 38114 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4386 4325 4272 4272
q2 385 260 270 260
q3 2850 2629 2691 2629
q4 1911 1684 1661 1661
q5 5632 5660 5744 5660
q6 243 144 137 137
q7 2236 1854 1792 1792
q8 3358 3452 3491 3452
q9 8816 8845 8799 8799
q10 3572 3410 3359 3359
q11 623 528 496 496
q12 836 657 655 655
q13 16237 3226 3309 3226
q14 334 291 284 284
q15 546 501 477 477
q16 651 581 558 558
q17 1839 1568 1561 1561
q18 8060 7774 7865 7774
q19 1757 1741 1490 1490
q20 2158 1864 1911 1864
q21 5759 5587 5401 5401
q22 1149 1037 1071 1037
Total cold run time: 73338 ms
Total hot run time: 56844 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on code in PR #40349:
URL: https://github.com/apache/doris/pull/40349#discussion_r1768045163
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/NereidsPlanner.java:
##########
@@ -256,6 +259,17 @@ private Plan planWithoutLock(
}
}
+ // if we cannot get table row count, skip join reorder
+ // except:
+ // 1. user set leading hint
+ // 2. ut test. In ut test, FeConstants.enableInternalSchemaDb is false or FeConstants.runningUnitTest is true
+ if (FeConstants.enableInternalSchemaDb && !FeConstants.runningUnitTest
+ && !cascadesContext.isLeadingDisableJoinReorder()) {
+ List<LogicalOlapScan> scans = cascadesContext.getRewritePlan()
+ .collectToList(LogicalOlapScan.class::isInstance);
Review Comment:
this function depends on OlapTable.getRowCountForIndex().
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2357453774
<details>
<summary>TPC-DS: <b>Total hot run time: 193843 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 603a31adf3385c8dc6f08873e8eaa597314be95f, data reload: false
query1 963 385 372 372
query2 6541 2021 1962 1962
query3 6705 215 222 215
query4 34169 23396 23452 23396
query5 4346 498 462 462
query6 263 171 159 159
query7 4611 291 307 291
query8 275 220 226 220
query9 9676 2655 2658 2655
query10 488 307 300 300
query11 18051 15102 15195 15102
query12 150 97 98 97
query13 1620 409 406 406
query14 10913 6758 6978 6758
query15 251 169 184 169
query16 7723 456 504 456
query17 1645 588 550 550
query18 2082 310 302 302
query19 321 150 151 150
query20 114 117 115 115
query21 220 105 112 105
query22 4757 4316 4350 4316
query23 34650 33864 33887 33864
query24 11198 2943 2880 2880
query25 656 420 404 404
query26 1241 161 161 161
query27 2422 287 296 287
query28 7935 2450 2435 2435
query29 850 435 427 427
query30 322 156 154 154
query31 1012 800 814 800
query32 104 59 57 57
query33 776 310 293 293
query34 962 473 494 473
query35 867 727 719 719
query36 1074 922 935 922
query37 162 89 90 89
query38 4003 3902 3943 3902
query39 1509 1449 1403 1403
query40 210 98 99 98
query41 52 52 49 49
query42 127 99 96 96
query43 510 476 475 475
query44 1219 822 806 806
query45 195 165 169 165
query46 1141 759 756 756
query47 1907 1802 1858 1802
query48 467 369 377 369
query49 1176 415 418 415
query50 822 403 407 403
query51 7016 7009 6889 6889
query52 100 90 96 90
query53 256 185 185 185
query54 1355 477 458 458
query55 81 79 87 79
query56 298 282 264 264
query57 1209 1089 1083 1083
query58 252 262 263 262
query59 3100 3050 2837 2837
query60 311 290 288 288
query61 128 228 103 103
query62 866 669 674 669
query63 226 193 193 193
query64 5320 639 635 635
query65 3305 3205 3176 3176
query66 1453 302 304 302
query67 16042 15539 15389 15389
query68 3120 863 829 829
query69 440 341 337 337
query70 1116 1151 1134 1134
query71 331 334 323 323
query72 5968 3389 3352 3352
query73 584 575 575 575
query74 9151 9133 9025 9025
query75 3090 2945 2839 2839
query76 1937 848 864 848
query77 416 360 354 354
query78 10294 9292 9336 9292
query79 897 870 874 870
query80 608 581 570 570
query81 452 248 254 248
query82 235 232 229 229
query83 156 155 164 155
query84 241 112 101 101
query85 677 417 381 381
query86 299 310 302 302
query87 4324 4333 4275 4275
query88 4639 4025 4013 4013
query89 372 364 359 359
query90 1468 316 309 309
query91 163 167 164 164
query92 76 72 71 71
query93 878 892 876 876
query94 559 357 369 357
query95 421 408 399 399
query96 491 480 479 479
query97 3132 3110 3123 3110
query98 221 233 232 232
query99 1392 1271 1300 1271
Total cold run time: 292919 ms
Total hot run time: 193843 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2358257790
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2366829511
run cloud_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "xzj7019 (via GitHub)" <gi...@apache.org>.
xzj7019 commented on code in PR #40349:
URL: https://github.com/apache/doris/pull/40349#discussion_r1770725493
##########
regression-test/suites/nereids_rules_p0/push_down_filter/extract_from_disjunction_in_join.groovy:
##########
@@ -20,7 +20,11 @@ suite("extract_from_disjunction_in_join") {
sql "SET enable_fallback_to_original_planner=false"
sql "set ignore_shape_nodes='PhysicalDistribute,PhysicalProject'"
sql "set disable_nereids_rules=PRUNE_EMPTY_PARTITION"
- sql "set runtime_filter_mode=OFF"
+ sql """
+ set runtime_filter_mode=OFF;
+ set disable_join_reorder=true;
+ set disable_join_reorder=true;
Review Comment:
dup
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2367284839
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2343337164
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2345195088
<details>
<summary>TPC-DS: <b>Total hot run time: 194910 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1e008b682b1245f10eacf6eeed3c52230e77553e, data reload: false
query1 922 390 385 385
query2 6165 1719 1735 1719
query3 8670 195 211 195
query4 33266 23950 24169 23950
query5 4816 529 544 529
query6 274 176 160 160
query7 4173 300 298 298
query8 285 219 237 219
query9 8357 2443 2441 2441
query10 469 286 279 279
query11 15941 15504 15529 15504
query12 157 102 100 100
query13 1548 398 394 394
query14 9487 6860 6733 6733
query15 210 175 172 172
query16 7621 465 451 451
query17 1124 565 551 551
query18 1968 316 309 309
query19 213 151 175 151
query20 125 118 118 118
query21 210 113 105 105
query22 4979 4725 4531 4531
query23 34834 34387 34256 34256
query24 10296 3099 3058 3058
query25 606 389 385 385
query26 720 152 155 152
query27 2088 276 277 276
query28 5181 2016 1995 1995
query29 844 419 404 404
query30 237 157 151 151
query31 980 776 768 768
query32 101 50 56 50
query33 712 299 306 299
query34 908 487 465 465
query35 868 738 724 724
query36 1063 938 916 916
query37 133 80 83 80
query38 4034 3886 3847 3847
query39 1466 1440 1415 1415
query40 209 116 118 116
query41 49 48 48 48
query42 124 101 97 97
query43 477 436 421 421
query44 1243 776 753 753
query45 201 174 177 174
query46 1104 831 814 814
query47 1907 1791 1838 1791
query48 362 291 290 290
query49 853 478 483 478
query50 938 441 450 441
query51 7034 7029 6775 6775
query52 101 94 89 89
query53 258 184 187 184
query54 760 479 470 470
query55 77 77 77 77
query56 292 270 282 270
query57 1189 1072 1096 1072
query58 243 252 261 252
query59 2842 2555 2555 2555
query60 297 268 281 268
query61 101 101 97 97
query62 881 663 668 663
query63 219 195 190 190
query64 3713 664 693 664
query65 3263 3163 3158 3158
query66 757 339 342 339
query67 15947 15492 15514 15492
query68 3081 860 839 839
query69 431 324 337 324
query70 1146 1138 1143 1138
query71 357 348 345 345
query72 5874 3393 3151 3151
query73 578 575 569 569
query74 9188 9020 9114 9020
query75 3195 3089 3070 3070
query76 1712 863 851 851
query77 446 410 403 403
query78 9407 9269 9324 9269
query79 916 909 880 880
query80 842 827 865 827
query81 443 269 271 269
query82 262 267 271 267
query83 190 190 186 186
query84 201 105 106 105
query85 621 412 396 396
query86 308 311 301 301
query87 4388 4347 4350 4347
query88 4082 4013 3996 3996
query89 373 379 367 367
query90 1118 308 303 303
query91 126 120 127 120
query92 77 76 75 75
query93 1030 1022 1020 1020
query94 551 389 366 366
query95 449 414 415 414
query96 463 468 469 468
query97 3118 3137 3110 3110
query98 226 231 226 226
query99 1655 1321 1315 1315
Total cold run time: 278346 ms
Total hot run time: 194910 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2358369669
<details>
<summary>TPC-DS: <b>Total hot run time: 194383 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 603a31adf3385c8dc6f08873e8eaa597314be95f, data reload: false
query1 939 394 389 389
query2 6373 2159 1990 1990
query3 8686 199 201 199
query4 33735 23696 23502 23502
query5 3463 476 461 461
query6 265 176 162 162
query7 4195 301 290 290
query8 296 223 225 223
query9 9308 2663 2646 2646
query10 443 277 277 277
query11 18055 15100 15136 15100
query12 146 99 97 97
query13 1541 404 410 404
query14 9642 7417 7011 7011
query15 271 170 174 170
query16 8019 462 454 454
query17 1641 618 597 597
query18 2094 322 331 322
query19 281 163 169 163
query20 129 114 117 114
query21 224 107 106 106
query22 4849 4310 4524 4310
query23 35177 34479 34326 34326
query24 11048 2971 2813 2813
query25 629 397 406 397
query26 1177 157 158 157
query27 2436 284 281 281
query28 7533 2441 2445 2441
query29 829 434 427 427
query30 267 151 146 146
query31 980 777 783 777
query32 95 54 58 54
query33 762 288 298 288
query34 923 493 488 488
query35 862 720 723 720
query36 1083 947 942 942
query37 162 90 92 90
query38 4071 3934 3971 3934
query39 1484 1397 1418 1397
query40 209 95 97 95
query41 52 48 48 48
query42 123 100 98 98
query43 527 496 478 478
query44 1278 811 797 797
query45 200 163 161 161
query46 1247 766 744 744
query47 1902 1778 1815 1778
query48 465 364 359 359
query49 920 419 419 419
query50 817 391 406 391
query51 6987 6904 6827 6827
query52 102 86 88 86
query53 251 176 173 173
query54 1184 451 444 444
query55 78 77 80 77
query56 301 257 255 255
query57 1194 1084 1066 1066
query58 232 233 228 228
query59 3108 2976 2817 2817
query60 297 265 262 262
query61 107 99 101 99
query62 821 660 654 654
query63 216 185 183 183
query64 3936 652 664 652
query65 3274 3249 3201 3201
query66 1149 292 295 292
query67 15897 15466 15212 15212
query68 3074 852 834 834
query69 449 347 354 347
query70 1191 1137 1166 1137
query71 351 335 339 335
query72 5884 3424 3401 3401
query73 578 572 569 569
query74 9545 8949 8947 8947
query75 3091 2920 2969 2920
query76 1854 886 892 886
query77 406 370 360 360
query78 9365 9246 9312 9246
query79 922 895 888 888
query80 610 572 577 572
query81 459 251 254 251
query82 235 233 239 233
query83 162 160 159 159
query84 237 103 98 98
query85 671 371 359 359
query86 315 316 309 309
query87 4384 4292 4360 4292
query88 4576 4040 4030 4030
query89 370 356 364 356
query90 1533 316 317 316
query91 166 159 165 159
query92 76 74 73 73
query93 896 886 900 886
query94 554 356 373 356
query95 425 416 402 402
query96 485 486 485 485
query97 3135 3111 3149 3111
query98 233 224 219 219
query99 1405 1310 1286 1286
Total cold run time: 288624 ms
Total hot run time: 194383 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2359724267
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2359859511
<details>
<summary>TPC-DS: <b>Total hot run time: 194548 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a5d7a05f22338b4b9214968e0f3c040ac6c67485, data reload: false
query1 945 405 394 394
query2 6405 2080 2043 2043
query3 8674 197 197 197
query4 34148 23646 23399 23399
query5 3445 485 465 465
query6 288 170 164 164
query7 4192 299 296 296
query8 284 218 218 218
query9 9446 2632 2626 2626
query10 443 277 275 275
query11 17916 15204 15252 15204
query12 132 106 93 93
query13 1536 443 424 424
query14 9911 7464 7001 7001
query15 248 173 178 173
query16 8049 445 453 445
query17 1635 604 588 588
query18 2178 320 318 318
query19 363 155 161 155
query20 127 119 123 119
query21 217 116 108 108
query22 4703 4665 4312 4312
query23 35312 34494 34543 34494
query24 11197 3009 2911 2911
query25 631 405 416 405
query26 1144 162 159 159
query27 2398 283 284 283
query28 7570 2452 2429 2429
query29 826 444 430 430
query30 262 152 153 152
query31 978 785 770 770
query32 96 55 58 55
query33 743 305 313 305
query34 899 501 488 488
query35 874 739 705 705
query36 1055 952 950 950
query37 155 89 90 89
query38 3996 3836 3876 3836
query39 1450 1402 1415 1402
query40 211 95 97 95
query41 51 47 48 47
query42 122 96 96 96
query43 537 487 487 487
query44 1361 811 794 794
query45 195 164 166 164
query46 1144 753 767 753
query47 1875 1793 1821 1793
query48 468 365 361 361
query49 914 407 412 407
query50 824 405 415 405
query51 7031 6897 6986 6897
query52 102 88 91 88
query53 259 180 181 180
query54 1120 459 459 459
query55 78 84 76 76
query56 273 248 257 248
query57 1165 1087 1047 1047
query58 238 235 236 235
query59 3164 2912 2951 2912
query60 296 276 273 273
query61 103 101 104 101
query62 807 689 659 659
query63 219 184 186 184
query64 3928 635 618 618
query65 3225 3181 3183 3181
query66 810 290 308 290
query67 15982 15549 15394 15394
query68 3040 573 574 573
query69 438 291 289 289
query70 1134 1084 1101 1084
query71 329 258 265 258
query72 5872 4066 3940 3940
query73 766 319 324 319
query74 9339 8859 8938 8859
query75 3359 2656 2686 2656
query76 2034 944 970 944
query77 402 286 296 286
query78 9815 9315 9488 9315
query79 1491 870 865 865
query80 957 582 576 576
query81 559 255 257 255
query82 654 236 225 225
query83 220 175 159 159
query84 239 102 106 102
query85 739 380 369 369
query86 380 312 291 291
query87 4550 4297 4378 4297
query88 4631 4071 4067 4067
query89 406 367 366 366
query90 1723 308 309 308
query91 165 159 167 159
query92 75 76 74 74
query93 947 899 899 899
query94 743 382 367 367
query95 455 416 413 413
query96 483 482 483 482
query97 3144 3120 3147 3120
query98 225 221 231 221
query99 1472 1294 1286 1286
Total cold run time: 292159 ms
Total hot run time: 194548 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "xzj7019 (via GitHub)" <gi...@apache.org>.
xzj7019 commented on code in PR #40349:
URL: https://github.com/apache/doris/pull/40349#discussion_r1767897301
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/NereidsPlanner.java:
##########
@@ -256,6 +259,17 @@ private Plan planWithoutLock(
}
}
+ // if we cannot get table row count, skip join reorder
+ // except:
+ // 1. user set leading hint
+ // 2. ut test. In ut test, FeConstants.enableInternalSchemaDb is false or FeConstants.runningUnitTest is true
+ if (FeConstants.enableInternalSchemaDb && !FeConstants.runningUnitTest
+ && !cascadesContext.isLeadingDisableJoinReorder()) {
Review Comment:
isLeadingDisableJoinReorder does not include the scenario when user uses "disable_join_reorder" or "ordered" hint.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2362906146
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "morrySnow (via GitHub)" <gi...@apache.org>.
morrySnow commented on code in PR #40349:
URL: https://github.com/apache/doris/pull/40349#discussion_r1755128052
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/NereidsPlanner.java:
##########
@@ -288,6 +296,18 @@ private Plan planWithoutLock(
return physicalPlan;
}
+ private List<LogicalOlapScan> getAllOlapScans(Plan plan) {
Review Comment:
could replace by `plan.collect(LogicalOlapScan.class::isInstance)`
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/stats/StatsCalculator.java:
##########
@@ -205,6 +210,23 @@ public Map<String, ColumnStatistic> getTotalColumnStatisticMap() {
return totalColumnStatisticMap;
}
+ /**
+ * disable join reorder if any table row count is not available.
+ */
+ public static void disableJoinReorderIfTableRowCountNotAvailable(
+ List<LogicalOlapScan> scans, CascadesContext context) {
+ StatsCalculator calculator = new StatsCalculator(context);
+ for (LogicalOlapScan scan : scans) {
+ double rowCount = calculator.getOlapTableRowCount(scan);
+ if (rowCount == -1 && ConnectContext.get() != null) {
Review Comment:
add comment here to explain magic number `-1`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2345199728
<details>
<summary>ClickBench: <b>Total hot run time: 31.4 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1e008b682b1245f10eacf6eeed3c52230e77553e, data reload: false
query1 0.05 0.05 0.05
query2 0.08 0.04 0.05
query3 0.22 0.04 0.04
query4 1.69 0.07 0.07
query5 0.50 0.51 0.51
query6 1.16 0.73 0.72
query7 0.02 0.01 0.02
query8 0.04 0.04 0.05
query9 0.55 0.53 0.51
query10 0.56 0.59 0.56
query11 0.15 0.13 0.12
query12 0.15 0.13 0.12
query13 0.63 0.61 0.60
query14 1.46 1.48 1.46
query15 0.92 0.88 0.88
query16 0.37 0.36 0.36
query17 1.04 1.03 1.02
query18 0.16 0.15 0.16
query19 1.97 1.84 1.82
query20 0.01 0.01 0.01
query21 15.39 0.68 0.68
query22 3.82 7.12 1.63
query23 18.12 1.39 1.33
query24 2.27 0.23 0.22
query25 0.19 0.08 0.08
query26 0.29 0.19 0.18
query27 0.09 0.07 0.07
query28 13.18 1.01 0.98
query29 12.50 3.30 3.30
query30 0.25 0.06 0.05
query31 2.86 0.42 0.41
query32 3.23 0.50 0.49
query33 3.03 3.09 3.09
query34 15.45 4.37 4.34
query35 4.38 4.37 4.40
query36 0.68 0.49 0.49
query37 0.19 0.16 0.15
query38 0.16 0.16 0.15
query39 0.05 0.04 0.04
query40 0.17 0.14 0.13
query41 0.10 0.04 0.05
query42 0.06 0.05 0.05
query43 0.05 0.05 0.04
Total cold run time: 108.24 s
Total hot run time: 31.4 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2338003863
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2338057835
<details>
<summary>TPC-H: <b>Total hot run time: 38290 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2c9998ac9f7f66ddd55e69fbe64ec9ccd434f6a0, data reload: false
------ Round 1 ----------------------------------
q1 18094 4570 4457 4457
q2 2492 204 182 182
q3 11424 1136 1025 1025
q4 10296 803 725 725
q5 7769 2881 2885 2881
q6 234 142 142 142
q7 977 632 607 607
q8 9697 2093 2097 2093
q9 7195 6549 6557 6549
q10 7033 2218 2190 2190
q11 457 242 245 242
q12 403 222 217 217
q13 17781 3072 3087 3072
q14 277 234 245 234
q15 551 484 479 479
q16 530 426 419 419
q17 983 707 737 707
q18 7387 6949 6870 6870
q19 1388 1080 1003 1003
q20 720 334 325 325
q21 4029 3098 2851 2851
q22 1108 1025 1020 1020
Total cold run time: 110825 ms
Total hot run time: 38290 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4401 4313 4304 4304
q2 379 271 287 271
q3 2902 2688 2679 2679
q4 1983 1766 1676 1676
q5 5475 5421 5415 5415
q6 225 131 133 131
q7 2140 1732 1724 1724
q8 3216 3377 3379 3377
q9 8500 8501 8513 8501
q10 3505 3209 3182 3182
q11 596 494 497 494
q12 811 600 602 600
q13 9580 3111 3071 3071
q14 311 274 268 268
q15 524 489 487 487
q16 509 461 458 458
q17 1817 1486 1471 1471
q18 7887 7495 7508 7495
q19 1686 1524 1579 1524
q20 2064 1818 1802 1802
q21 5433 5203 5325 5203
q22 1127 1044 1010 1010
Total cold run time: 65071 ms
Total hot run time: 55143 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "xzj7019 (via GitHub)" <gi...@apache.org>.
xzj7019 commented on code in PR #40349:
URL: https://github.com/apache/doris/pull/40349#discussion_r1751153358
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/stats/StatsCalculator.java:
##########
@@ -205,6 +210,22 @@ public Map<String, ColumnStatistic> getTotalColumnStatisticMap() {
return totalColumnStatisticMap;
}
+ /**
+ * disable join reorder if any table row count is not available.
+ */
+ public static void disableJoinReorderIfTableRowCountNotAvailable(
+ List<LogicalOlapScan> scans, CascadesContext context) {
+ StatsCalculator calculator = new StatsCalculator(context);
+ for (LogicalOlapScan scan : scans) {
+ double rowCount = calculator.getOlapTableRowCount(scan);
+ if (rowCount == -1 && ConnectContext.get() != null) {
Review Comment:
better to add comments here to notice what is the "-1"'s indication.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2340140764
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2346346086
<details>
<summary>TPC-DS: <b>Total hot run time: 195354 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4c5f044ea0e8082fff517399b66309a70ce2c13d, data reload: false
query1 944 377 380 377
query2 6496 1862 1773 1773
query3 6659 222 220 220
query4 26802 23993 24094 23993
query5 5136 553 520 520
query6 258 173 164 164
query7 4590 313 322 313
query8 314 246 226 226
query9 8506 2620 2592 2592
query10 465 286 283 283
query11 16445 15460 15822 15460
query12 166 115 104 104
query13 1696 403 392 392
query14 10903 6969 6732 6732
query15 214 177 172 172
query16 7621 475 485 475
query17 1543 585 576 576
query18 1994 299 291 291
query19 199 152 147 147
query20 118 110 115 110
query21 213 113 107 107
query22 4524 4240 4147 4147
query23 34689 34075 33750 33750
query24 9566 3250 3062 3062
query25 684 416 414 414
query26 1245 163 160 160
query27 2642 289 292 289
query28 6803 2126 2081 2081
query29 976 436 437 436
query30 303 164 155 155
query31 995 794 786 786
query32 106 57 63 57
query33 743 307 313 307
query34 928 501 497 497
query35 860 742 745 742
query36 1065 916 891 891
query37 177 88 85 85
query38 4064 3958 3882 3882
query39 1491 1421 1402 1402
query40 223 118 120 118
query41 51 53 49 49
query42 119 101 100 100
query43 503 457 446 446
query44 1249 811 797 797
query45 200 171 178 171
query46 1121 831 807 807
query47 1904 1792 1793 1792
query48 374 304 304 304
query49 1137 482 466 466
query50 934 447 454 447
query51 6941 7111 6950 6950
query52 103 90 94 90
query53 263 198 190 190
query54 851 476 489 476
query55 81 82 82 82
query56 303 272 278 272
query57 1207 1062 1096 1062
query58 258 256 249 249
query59 2898 2705 2660 2660
query60 315 286 295 286
query61 224 104 103 103
query62 952 676 671 671
query63 224 188 191 188
query64 5307 677 665 665
query65 3274 3239 3206 3206
query66 1460 300 302 300
query67 16391 15768 15684 15684
query68 3349 859 845 845
query69 440 324 326 324
query70 1169 1133 1143 1133
query71 351 340 346 340
query72 6134 3351 3389 3351
query73 591 583 583 583
query74 9380 9102 9158 9102
query75 3115 3024 3016 3016
query76 1960 847 836 836
query77 446 400 390 390
query78 9406 9252 9173 9173
query79 913 903 871 871
query80 879 827 807 807
query81 454 261 275 261
query82 261 262 261 261
query83 191 195 189 189
query84 231 110 107 107
query85 654 413 392 392
query86 318 302 305 302
query87 4381 4298 4476 4298
query88 4202 4137 4128 4128
query89 375 370 366 366
query90 1292 323 312 312
query91 127 125 123 123
query92 79 77 75 75
query93 1032 1052 1018 1018
query94 624 371 385 371
query95 478 425 423 423
query96 475 473 475 473
query97 3119 3133 3164 3133
query98 231 238 284 238
query99 1543 1325 1312 1312
Total cold run time: 280210 ms
Total hot run time: 195354 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2346360678
<details>
<summary>ClickBench: <b>Total hot run time: 31.31 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4c5f044ea0e8082fff517399b66309a70ce2c13d, data reload: false
query1 0.05 0.04 0.04
query2 0.07 0.04 0.04
query3 0.23 0.06 0.06
query4 1.66 0.07 0.09
query5 0.53 0.50 0.51
query6 1.13 0.73 0.72
query7 0.02 0.01 0.02
query8 0.05 0.04 0.04
query9 0.56 0.52 0.52
query10 0.58 0.57 0.56
query11 0.16 0.12 0.12
query12 0.15 0.13 0.13
query13 0.63 0.61 0.61
query14 1.41 1.48 1.47
query15 0.91 0.88 0.88
query16 0.36 0.36 0.36
query17 1.02 1.03 1.05
query18 0.21 0.21 0.21
query19 1.95 1.85 1.80
query20 0.02 0.01 0.01
query21 15.40 0.67 0.64
query22 4.53 7.84 1.52
query23 17.81 1.25 1.28
query24 2.24 0.22 0.22
query25 0.17 0.08 0.09
query26 0.30 0.18 0.17
query27 0.08 0.08 0.08
query28 13.20 1.13 1.09
query29 12.63 3.36 3.32
query30 0.24 0.06 0.05
query31 2.88 0.43 0.41
query32 3.22 0.50 0.49
query33 3.03 3.06 3.05
query34 15.42 4.30 4.30
query35 4.33 4.36 4.36
query36 0.69 0.49 0.50
query37 0.19 0.17 0.16
query38 0.16 0.15 0.15
query39 0.05 0.04 0.04
query40 0.17 0.15 0.13
query41 0.11 0.05 0.05
query42 0.05 0.05 0.04
query43 0.05 0.05 0.04
Total cold run time: 108.65 s
Total hot run time: 31.31 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2343331525
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat](nereids) disable join reorder if any table row count is not available [doris]
Posted by "englefly (via GitHub)" <gi...@apache.org>.
englefly commented on PR #40349:
URL: https://github.com/apache/doris/pull/40349#issuecomment-2343346729
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org