You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Jibing-Li (via GitHub)" <gi...@apache.org> on 2024/07/21 07:04:53 UTC
[PR] [fix](statistics)Remove old partition stats in the next analyze after insert overwrite. [doris]
Jibing-Li opened a new pull request, #38175:
URL: https://github.com/apache/doris/pull/38175
Remove old partition stats in the next analyze after insert overwrite. Because insert overwrite will use new partitions to replace old ones, which means all partitions are new.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Remove old partition stats in the next analyze after insert overwrite. [doris]
Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li commented on PR #38175:
URL: https://github.com/apache/doris/pull/38175#issuecomment-2241503514
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Remove old partition stats in the next analyze after insert overwrite. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38175:
URL: https://github.com/apache/doris/pull/38175#issuecomment-2241519299
<details>
<summary>ClickBench: <b>Total hot run time: 30.48 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 419994054f07520bd0749c6805c53f4a6cb74ce2, data reload: false
query1 0.04 0.03 0.04
query2 0.08 0.04 0.04
query3 0.22 0.05 0.05
query4 1.68 0.09 0.07
query5 0.50 0.48 0.48
query6 1.13 0.72 0.72
query7 0.02 0.01 0.01
query8 0.05 0.04 0.05
query9 0.56 0.50 0.50
query10 0.56 0.56 0.54
query11 0.15 0.11 0.11
query12 0.15 0.12 0.12
query13 0.61 0.58 0.58
query14 0.75 0.78 0.78
query15 0.87 0.81 0.81
query16 0.37 0.37 0.37
query17 1.05 1.05 1.05
query18 0.22 0.22 0.22
query19 1.91 1.83 1.81
query20 0.02 0.01 0.01
query21 15.41 0.75 0.65
query22 4.66 7.09 1.73
query23 18.28 1.41 1.17
query24 2.08 0.24 0.22
query25 0.14 0.09 0.09
query26 0.30 0.21 0.21
query27 0.46 0.24 0.23
query28 13.23 1.02 1.01
query29 12.65 3.27 3.24
query30 0.25 0.06 0.06
query31 2.88 0.39 0.38
query32 3.27 0.48 0.46
query33 2.89 2.91 2.88
query34 17.06 4.38 4.35
query35 4.41 4.40 4.43
query36 0.66 0.48 0.47
query37 0.19 0.17 0.16
query38 0.16 0.14 0.15
query39 0.04 0.03 0.04
query40 0.15 0.12 0.12
query41 0.09 0.05 0.05
query42 0.06 0.05 0.06
query43 0.06 0.04 0.04
Total cold run time: 110.32 s
Total hot run time: 30.48 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Remove old partition stats in the next analyze after insert overwrite. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38175:
URL: https://github.com/apache/doris/pull/38175#issuecomment-2241514682
<details>
<summary>TPC-H: <b>Total hot run time: 39752 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 419994054f07520bd0749c6805c53f4a6cb74ce2, data reload: false
------ Round 1 ----------------------------------
q1 17629 4365 4298 4298
q2 2004 201 188 188
q3 10440 1133 1063 1063
q4 10188 824 792 792
q5 7520 2784 2649 2649
q6 219 135 138 135
q7 952 599 604 599
q8 9213 2060 2071 2060
q9 8621 6558 6545 6545
q10 8848 3796 3787 3787
q11 504 237 236 236
q12 426 227 225 225
q13 17761 2979 3003 2979
q14 281 228 253 228
q15 536 481 494 481
q16 505 393 390 390
q17 971 678 701 678
q18 8146 7484 7324 7324
q19 3682 1535 1352 1352
q20 696 326 318 318
q21 4892 3229 3137 3137
q22 349 293 288 288
Total cold run time: 114383 ms
Total hot run time: 39752 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4322 4213 4220 4213
q2 366 276 274 274
q3 3008 2739 2838 2739
q4 1980 1703 1709 1703
q5 5663 5618 5520 5520
q6 239 139 137 137
q7 2195 1847 1867 1847
q8 3273 3430 3394 3394
q9 8794 8753 8861 8753
q10 4056 3961 3828 3828
q11 631 479 503 479
q12 789 633 680 633
q13 16096 3194 3161 3161
q14 320 298 282 282
q15 533 501 486 486
q16 470 430 451 430
q17 1815 1504 1504 1504
q18 8199 7913 7951 7913
q19 2388 1575 1626 1575
q20 2162 1938 1863 1863
q21 5031 4870 4797 4797
q22 562 484 505 484
Total cold run time: 72892 ms
Total hot run time: 56015 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Remove old partition stats in the next analyze after insert overwrite. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38175:
URL: https://github.com/apache/doris/pull/38175#issuecomment-2241946246
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Remove old partition stats in the next analyze after insert overwrite. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38175:
URL: https://github.com/apache/doris/pull/38175#issuecomment-2241503422
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Remove old partition stats in the next analyze after insert overwrite. [doris]
Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li commented on PR #38175:
URL: https://github.com/apache/doris/pull/38175#issuecomment-2241561117
run cloud_p1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Remove old partition stats in the next analyze after insert overwrite. [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #38175:
URL: https://github.com/apache/doris/pull/38175#issuecomment-2241517399
<details>
<summary>TPC-DS: <b>Total hot run time: 173447 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 419994054f07520bd0749c6805c53f4a6cb74ce2, data reload: false
query1 913 378 369 369
query2 6439 1875 1807 1807
query3 6630 208 228 208
query4 28395 17525 17371 17371
query5 3619 474 491 474
query6 277 177 175 175
query7 4588 286 294 286
query8 247 200 194 194
query9 8583 2462 2446 2446
query10 448 300 273 273
query11 11732 9967 9869 9869
query12 121 89 84 84
query13 1651 376 368 368
query14 10225 7578 7542 7542
query15 227 159 166 159
query16 7760 507 438 438
query17 1558 558 521 521
query18 1926 282 271 271
query19 193 147 149 147
query20 87 84 102 84
query21 206 132 132 132
query22 4301 4068 3970 3970
query23 34092 33747 33695 33695
query24 11257 2880 2972 2880
query25 648 394 383 383
query26 1132 153 155 153
query27 2473 274 279 274
query28 7468 2098 2110 2098
query29 900 637 637 637
query30 259 154 152 152
query31 961 735 776 735
query32 98 59 56 56
query33 750 347 334 334
query34 940 488 500 488
query35 872 778 770 770
query36 1145 999 947 947
query37 158 92 86 86
query38 2989 2859 2817 2817
query39 951 825 811 811
query40 207 117 121 117
query41 44 44 43 43
query42 121 98 97 97
query43 509 472 474 472
query44 1219 732 734 732
query45 192 163 160 160
query46 1084 732 757 732
query47 1825 1754 1779 1754
query48 364 287 295 287
query49 837 430 413 413
query50 788 388 383 383
query51 7045 6868 6735 6735
query52 104 97 94 94
query53 357 291 286 286
query54 918 447 451 447
query55 77 78 75 75
query56 298 279 265 265
query57 1115 1025 1043 1025
query58 261 253 252 252
query59 2905 2627 2874 2627
query60 302 280 325 280
query61 98 94 96 94
query62 786 645 636 636
query63 318 285 290 285
query64 9425 2214 1684 1684
query65 3163 3099 3090 3090
query66 742 323 329 323
query67 15416 14893 14894 14893
query68 4551 540 528 528
query69 719 446 369 369
query70 1160 1117 1152 1117
query71 420 284 284 284
query72 7409 5379 5630 5379
query73 756 326 321 321
query74 6067 5688 5642 5642
query75 3447 2681 2729 2681
query76 2850 959 963 959
query77 630 314 309 309
query78 10011 8903 8943 8903
query79 2311 531 525 525
query80 1315 475 550 475
query81 589 222 217 217
query82 814 138 132 132
query83 271 159 165 159
query84 239 88 85 85
query85 1507 328 294 294
query86 466 323 316 316
query87 3263 3069 3071 3069
query88 3893 2358 2366 2358
query89 475 386 372 372
query90 1753 190 194 190
query91 130 100 100 100
query92 59 48 51 48
query93 2205 517 516 516
query94 1310 290 276 276
query95 401 304 319 304
query96 596 277 270 270
query97 3268 3060 3051 3051
query98 226 203 195 195
query99 1551 1259 1218 1218
Total cold run time: 282401 ms
Total hot run time: 173447 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Remove old partition stats in the next analyze after insert overwrite. [doris]
Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li commented on PR #38175:
URL: https://github.com/apache/doris/pull/38175#issuecomment-2241561069
run feut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Remove old partition stats in the next analyze after insert overwrite. [doris]
Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li commented on PR #38175:
URL: https://github.com/apache/doris/pull/38175#issuecomment-2241615635
run cloud_p1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Remove old partition stats in the next analyze after insert overwrite. [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #38175:
URL: https://github.com/apache/doris/pull/38175#issuecomment-2241997902
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](statistics)Remove old partition stats in the next analyze after insert overwrite. [doris]
Posted by "Jibing-Li (via GitHub)" <gi...@apache.org>.
Jibing-Li merged PR #38175:
URL: https://github.com/apache/doris/pull/38175
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org