You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "eldenmoon (via GitHub)" <gi...@apache.org> on 2024/02/06 08:19:48 UTC
[PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
eldenmoon opened a new pull request, #30904:
URL: https://github.com/apache/doris/pull/30904
…nt (#102)
This addresses the scenario where a rowset cannot be removed.
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_count [doris]
Posted by "xiaokang (via GitHub)" <gi...@apache.org>.
xiaokang commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1933381862
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1929569811
<details>
<summary>ClickBench: <b>Total hot run time: 31.34 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 32a23524fe4a99f1a696776ed3ea5e9a0e78027a, data reload: false
query1 0.04 0.03 0.04
query2 0.07 0.03 0.02
query3 0.23 0.06 0.06
query4 1.66 0.09 0.09
query5 0.55 0.52 0.51
query6 1.18 0.65 0.63
query7 0.01 0.01 0.01
query8 0.04 0.03 0.03
query9 0.55 0.49 0.48
query10 0.55 0.55 0.56
query11 0.13 0.09 0.09
query12 0.12 0.09 0.09
query13 0.61 0.60 0.61
query14 0.78 0.80 0.78
query15 0.80 0.79 0.79
query16 0.40 0.38 0.39
query17 1.05 0.98 1.02
query18 0.23 0.28 0.22
query19 1.89 1.79 1.80
query20 0.02 0.01 0.01
query21 15.40 0.56 0.55
query22 2.56 2.97 2.50
query23 17.26 0.84 0.81
query24 2.61 0.81 0.47
query25 0.30 0.13 0.19
query26 0.37 0.14 0.14
query27 0.06 0.05 0.05
query28 12.09 0.91 0.86
query29 12.51 3.26 3.29
query30 0.64 0.55 0.53
query31 2.78 0.34 0.35
query32 3.35 0.48 0.48
query33 3.21 3.24 3.21
query34 15.84 4.29 4.29
query35 4.31 4.34 4.28
query36 1.09 1.06 1.06
query37 0.07 0.05 0.05
query38 0.04 0.03 0.02
query39 0.02 0.01 0.02
query40 0.16 0.12 0.14
query41 0.08 0.02 0.01
query42 0.03 0.01 0.01
query43 0.03 0.02 0.02
Total cold run time: 105.72 s
Total hot run time: 31.34 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
Posted by "eldenmoon (via GitHub)" <gi...@apache.org>.
eldenmoon commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1929393901
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1929599553
TeamCity be ut coverage result:
Function Coverage: 36.19% (8637/23863)
Line Coverage: 28.21% (70620/250369)
Region Coverage: 27.21% (36441/133927)
Branch Coverage: 23.99% (18666/77814)
Coverage Report: http://coverage.selectdb-in.cc/coverage/32a23524fe4a99f1a696776ed3ea5e9a0e78027a_32a23524fe4a99f1a696776ed3ea5e9a0e78027a/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1929077994
<details>
<summary>ClickBench: <b>Total hot run time: 31.15 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit eeaf78b488f96c522aa3645952f14cd32486f61b, data reload: false
query1 0.03 0.03 0.03
query2 0.06 0.03 0.02
query3 0.23 0.06 0.06
query4 1.67 0.10 0.10
query5 0.52 0.52 0.52
query6 1.18 0.67 0.65
query7 0.02 0.01 0.01
query8 0.03 0.02 0.02
query9 0.55 0.51 0.51
query10 0.55 0.54 0.55
query11 0.12 0.09 0.08
query12 0.10 0.08 0.09
query13 0.61 0.61 0.60
query14 0.79 0.81 0.81
query15 0.78 0.78 0.76
query16 0.40 0.37 0.38
query17 1.01 1.01 1.00
query18 0.22 0.26 0.24
query19 1.80 1.80 1.80
query20 0.01 0.01 0.01
query21 15.40 0.57 0.57
query22 2.54 2.79 1.99
query23 17.33 0.83 0.78
query24 2.39 0.71 0.90
query25 0.36 0.22 0.14
query26 0.53 0.14 0.14
query27 0.06 0.05 0.04
query28 12.35 0.84 0.86
query29 12.52 3.35 3.27
query30 0.57 0.57 0.54
query31 2.78 0.34 0.34
query32 3.38 0.47 0.48
query33 3.22 3.22 3.23
query34 15.81 4.28 4.27
query35 4.31 4.29 4.30
query36 1.08 1.07 1.07
query37 0.07 0.05 0.05
query38 0.03 0.02 0.02
query39 0.02 0.01 0.02
query40 0.16 0.14 0.13
query41 0.07 0.02 0.02
query42 0.03 0.01 0.02
query43 0.02 0.02 0.02
Total cold run time: 105.71 s
Total hot run time: 31.15 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_count [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #30904:
URL: https://github.com/apache/doris/pull/30904
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
Posted by "eldenmoon (via GitHub)" <gi...@apache.org>.
eldenmoon commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1928994776
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_count [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1933406991
<details>
<summary>TPC-DS: <b>Total hot run time: 180875 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 59c54f323b6f549e66b88e51169775c58c89fca4, data reload: false
query1 943 361 352 352
query2 6511 1753 1800 1753
query3 6698 207 205 205
query4 24333 22620 22537 22537
query5 4303 469 467 467
query6 262 167 165 165
query7 4610 315 289 289
query8 256 194 201 194
query9 8461 2848 2848 2848
query10 417 232 218 218
query11 16259 15703 15574 15574
query12 134 81 83 81
query13 1682 429 409 409
query14 9018 7735 7683 7683
query15 219 184 190 184
query16 7343 250 249 249
query17 1410 570 539 539
query18 1957 269 266 266
query19 197 146 157 146
query20 84 82 83 82
query21 190 125 110 110
query22 5072 4796 4863 4796
query23 32536 31532 31596 31532
query24 12701 3399 3443 3399
query25 652 378 369 369
query26 1880 155 159 155
query27 3053 316 315 315
query28 6656 1829 1835 1829
query29 1180 622 620 620
query30 277 142 146 142
query31 977 781 782 781
query32 90 60 58 58
query33 719 232 243 232
query34 1058 483 499 483
query35 946 831 844 831
query36 993 901 869 869
query37 157 55 67 55
query38 3453 3349 3351 3349
query39 1372 1341 1330 1330
query40 304 104 106 104
query41 37 35 33 33
query42 110 95 101 95
query43 473 448 448 448
query44 1064 698 707 698
query45 203 182 173 173
query46 1045 788 759 759
query47 1764 1597 1592 1592
query48 423 344 340 340
query49 1197 311 302 302
query50 771 395 373 373
query51 5325 5213 5172 5172
query52 109 91 91 91
query53 390 294 301 294
query54 291 221 219 219
query55 88 76 80 76
query56 220 212 196 196
query57 1043 976 995 976
query58 215 197 195 195
query59 2273 2289 2141 2141
query60 234 214 216 214
query61 87 87 83 83
query62 603 402 390 390
query63 328 284 281 281
query64 6504 3076 3174 3076
query65 3318 3258 3224 3224
query66 1364 326 324 324
query67 14520 14122 14319 14122
query68 5103 516 540 516
query69 519 361 359 359
query70 1304 1212 1201 1201
query71 377 255 254 254
query72 6334 2745 2614 2614
query73 702 305 307 305
query74 7061 6669 6713 6669
query75 3197 2550 2557 2550
query76 3210 1161 1209 1161
query77 346 237 228 228
query78 9446 8761 8753 8753
query79 968 499 493 493
query80 516 354 346 346
query81 402 206 216 206
query82 165 83 82 82
query83 141 123 125 123
query84 224 79 80 79
query85 755 344 345 344
query86 295 319 302 302
query87 3607 3438 3405 3405
query88 2690 2278 2275 2275
query89 440 353 352 352
query90 1921 165 166 165
query91 154 126 130 126
query92 54 49 48 48
query93 1029 520 476 476
query94 1061 178 183 178
query95 482 369 8651 369
query96 583 262 260 260
query97 4499 4312 4255 4255
query98 216 206 197 197
query99 1060 750 697 697
Total cold run time: 272223 ms
Total hot run time: 180875 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_count [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1931201000
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1929083311
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit eeaf78b488f96c522aa3645952f14cd32486f61b with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.7 seconds inserted 10000000 Rows, about 729K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1928994331
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1929050374
<details>
<summary>TPC-H: <b>Total hot run time: 37484 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eeaf78b488f96c522aa3645952f14cd32486f61b, data reload: false
------ Round 1 ----------------------------------
q1 18143 4725 4475 4475
q2 2202 148 137 137
q3 11331 955 961 955
q4 5052 844 737 737
q5 7915 2915 2918 2915
q6 189 124 127 124
q7 1177 773 763 763
q8 9521 2049 2056 2049
q9 7487 6396 6347 6347
q10 8104 2491 2402 2402
q11 413 210 205 205
q12 753 278 273 273
q13 18025 3310 3343 3310
q14 279 248 255 248
q15 523 494 497 494
q16 474 404 409 404
q17 947 494 520 494
q18 6924 6071 6054 6054
q19 1578 1488 1399 1399
q20 629 364 357 357
q21 6982 3159 3049 3049
q22 808 301 293 293
Total cold run time: 109456 ms
Total hot run time: 37484 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4460 4432 4465 4432
q2 323 245 230 230
q3 3005 2865 2804 2804
q4 1815 1598 1601 1598
q5 5198 5282 5249 5249
q6 191 114 116 114
q7 2176 1784 1728 1728
q8 3139 3277 3301 3277
q9 8314 8377 8354 8354
q10 5812 3546 3550 3546
q11 544 464 459 459
q12 752 586 576 576
q13 6657 3094 3113 3094
q14 264 253 260 253
q15 521 493 491 491
q16 496 479 478 478
q17 1861 1693 1728 1693
q18 8003 7779 7542 7542
q19 9014 1536 1544 1536
q20 2119 1921 1908 1908
q21 4792 4554 4614 4554
q22 540 471 506 471
Total cold run time: 69996 ms
Total hot run time: 54387 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1929409164
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_count [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1933413533
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 59c54f323b6f549e66b88e51169775c58c89fca4 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 60 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 13.5 seconds inserted 10000000 Rows, about 740K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1929069096
<details>
<summary>TPC-DS: <b>Total hot run time: 174687 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eeaf78b488f96c522aa3645952f14cd32486f61b, data reload: false
query1 942 355 337 337
query2 6505 2093 1973 1973
query3 6699 214 206 206
query4 31611 22009 22034 22009
query5 4322 431 414 414
query6 260 163 164 163
query7 4607 287 284 284
query8 243 175 180 175
query9 9255 2290 2281 2281
query10 427 236 232 232
query11 18074 15340 15493 15340
query12 131 77 75 75
query13 1621 411 410 410
query14 9296 7084 7118 7084
query15 262 174 185 174
query16 8138 259 248 248
query17 2091 527 495 495
query18 2104 264 258 258
query19 353 144 141 141
query20 82 82 81 81
query21 201 130 119 119
query22 4946 4678 4680 4678
query23 30763 30035 29976 29976
query24 10644 2731 2778 2731
query25 577 348 346 346
query26 1396 149 151 149
query27 2917 302 311 302
query28 7342 1858 1838 1838
query29 869 618 615 615
query30 285 130 139 130
query31 888 701 729 701
query32 94 53 51 51
query33 735 225 220 220
query34 1091 464 455 455
query35 875 778 775 775
query36 966 959 926 926
query37 121 56 56 56
query38 3195 3138 3121 3121
query39 1296 1252 1233 1233
query40 190 96 91 91
query41 39 34 34 34
query42 103 94 96 94
query43 533 491 503 491
query44 1029 688 712 688
query45 195 181 175 175
query46 1047 663 659 659
query47 1583 1477 1442 1442
query48 411 351 347 347
query49 1106 285 280 280
query50 750 370 387 370
query51 5236 5119 5143 5119
query52 100 87 83 83
query53 337 291 271 271
query54 265 215 216 215
query55 81 78 76 76
query56 219 194 209 194
query57 988 914 916 914
query58 194 176 188 176
query59 2424 2317 2445 2317
query60 236 218 209 209
query61 83 82 89 82
query62 641 378 399 378
query63 294 272 281 272
query64 5528 3687 3481 3481
query65 3204 3214 3218 3214
query66 1009 314 312 312
query67 14174 14314 14104 14104
query68 4302 549 529 529
query69 457 333 331 331
query70 1320 1239 1234 1234
query71 317 248 239 239
query72 5962 2801 2639 2639
query73 697 336 326 326
query74 6631 6237 6246 6237
query75 3002 2315 2304 2304
query76 2545 932 924 924
query77 338 234 225 225
query78 9110 8727 8453 8453
query79 3510 498 495 495
query80 2194 356 341 341
query81 535 196 205 196
query82 1539 82 93 82
query83 358 132 122 122
query84 283 80 79 79
query85 1609 346 329 329
query86 464 283 303 283
query87 3362 3224 3200 3200
query88 4388 2340 2325 2325
query89 441 366 367 366
query90 2003 163 166 163
query91 154 141 124 124
query92 60 44 41 41
query93 4216 508 459 459
query94 1320 172 175 172
query95 7957 360 354 354
query96 606 266 265 265
query97 4241 4110 4121 4110
query98 228 198 196 196
query99 1261 654 715 654
Total cold run time: 293633 ms
Total hot run time: 174687 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1929491278
<details>
<summary>TPC-H: <b>Total hot run time: 37066 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 32a23524fe4a99f1a696776ed3ea5e9a0e78027a, data reload: false
------ Round 1 ----------------------------------
q1 17686 4593 4464 4464
q2 2049 153 136 136
q3 10580 952 915 915
q4 4652 750 755 750
q5 7718 2792 2833 2792
q6 185 123 117 117
q7 1157 730 719 719
q8 9295 2025 2054 2025
q9 7313 6350 6378 6350
q10 8135 2429 2422 2422
q11 420 212 194 194
q12 757 266 291 266
q13 18014 3336 3322 3322
q14 279 263 254 254
q15 531 503 506 503
q16 491 408 417 408
q17 951 586 531 531
q18 6782 5965 5912 5912
q19 1609 1338 1304 1304
q20 569 362 353 353
q21 6897 3167 3033 3033
q22 809 310 296 296
Total cold run time: 106879 ms
Total hot run time: 37066 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4432 4453 4429 4429
q2 331 237 229 229
q3 2983 2861 2814 2814
q4 1887 1625 1630 1625
q5 5209 5228 5272 5228
q6 186 114 114 114
q7 2173 1784 1739 1739
q8 3118 3258 3249 3249
q9 8387 8309 8324 8309
q10 5921 3593 3576 3576
q11 543 478 486 478
q12 721 576 542 542
q13 16920 3097 3059 3059
q14 290 262 267 262
q15 544 495 489 489
q16 518 480 482 480
q17 1858 1689 1682 1682
q18 8048 7814 7515 7515
q19 6675 1553 1568 1553
q20 2140 1917 1895 1895
q21 4737 4590 4652 4590
q22 526 456 493 456
Total cold run time: 78147 ms
Total hot run time: 54313 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1929120655
TeamCity be ut coverage result:
Function Coverage: 36.20% (8629/23839)
Line Coverage: 28.21% (70560/250126)
Region Coverage: 27.22% (36424/133791)
Branch Coverage: 24.00% (18656/77722)
Coverage Report: http://coverage.selectdb-in.cc/coverage/eeaf78b488f96c522aa3645952f14cd32486f61b_eeaf78b488f96c522aa3645952f14cd32486f61b/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1929584228
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 32a23524fe4a99f1a696776ed3ea5e9a0e78027a with default session variables
Stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 13.5 seconds inserted 10000000 Rows, about 740K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_count [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1931200981
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1929008565
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_count [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1933397893
<details>
<summary>TPC-H: <b>Total hot run time: 41474 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 59c54f323b6f549e66b88e51169775c58c89fca4, data reload: false
------ Round 1 ----------------------------------
q1 17660 5018 4905 4905
q2 2040 147 140 140
q3 10570 1041 1000 1000
q4 4658 978 970 970
q5 7711 3225 3268 3225
q6 197 136 133 133
q7 1276 774 757 757
q8 9262 2073 2084 2073
q9 7510 6720 6694 6694
q10 8316 2638 2664 2638
q11 422 214 205 205
q12 754 333 329 329
q13 17988 3677 3673 3673
q14 294 268 252 252
q15 596 508 523 508
q16 471 402 421 402
q17 922 874 821 821
q18 7456 6715 6636 6636
q19 1526 1497 1516 1497
q20 621 365 312 312
q21 6996 4016 3972 3972
q22 873 344 332 332
Total cold run time: 108119 ms
Total hot run time: 41474 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4894 4829 4875 4829
q2 293 184 187 184
q3 3616 3602 3598 3598
q4 2557 2556 2537 2537
q5 5776 5753 5746 5746
q6 211 129 127 127
q7 2255 1682 1687 1682
q8 3035 3125 3111 3111
q9 8766 8777 8726 8726
q10 6780 4251 4252 4251
q11 533 378 383 378
q12 761 542 536 536
q13 5110 3409 3432 3409
q14 279 235 232 232
q15 624 501 523 501
q16 504 437 422 422
q17 2008 1632 1616 1616
q18 8341 7817 7769 7769
q19 1627 1637 1638 1637
q20 2116 1826 1814 1814
q21 6598 6170 6169 6169
q22 561 500 507 500
Total cold run time: 67245 ms
Total hot run time: 59774 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_count [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1933411117
<details>
<summary>ClickBench: <b>Total hot run time: 30.68 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 59c54f323b6f549e66b88e51169775c58c89fca4, data reload: false
query1 0.03 0.02 0.02
query2 0.06 0.02 0.02
query3 0.22 0.07 0.07
query4 1.64 0.09 0.08
query5 0.48 0.48 0.48
query6 1.35 0.66 0.62
query7 0.02 0.01 0.02
query8 0.04 0.02 0.02
query9 0.53 0.44 0.44
query10 0.50 0.48 0.47
query11 0.12 0.10 0.09
query12 0.12 0.10 0.10
query13 0.59 0.58 0.58
query14 0.76 0.78 0.78
query15 0.82 0.79 0.79
query16 0.32 0.33 0.33
query17 0.90 0.88 0.86
query18 0.17 0.17 0.17
query19 1.72 1.67 1.68
query20 0.01 0.02 0.01
query21 15.40 0.62 0.61
query22 3.26 3.37 2.28
query23 17.90 1.04 1.00
query24 2.00 0.38 0.22
query25 0.58 0.06 0.06
query26 0.15 0.14 0.14
query27 0.05 0.06 0.05
query28 12.38 0.84 0.80
query29 12.55 3.35 3.34
query30 0.52 0.48 0.47
query31 2.78 0.36 0.37
query32 3.33 0.48 0.47
query33 3.16 3.06 3.19
query34 15.38 4.49 4.51
query35 4.49 4.50 4.50
query36 1.07 0.96 0.94
query37 0.07 0.05 0.06
query38 0.04 0.03 0.03
query39 0.02 0.02 0.02
query40 0.18 0.15 0.14
query41 0.06 0.01 0.02
query42 0.02 0.02 0.01
query43 0.03 0.02 0.02
Total cold run time: 105.82 s
Total hot run time: 30.68 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_count [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1933412015
TeamCity be ut coverage result:
Function Coverage: 35.86% (8556/23862)
Line Coverage: 27.78% (69458/249999)
Region Coverage: 26.92% (36039/133874)
Branch Coverage: 23.70% (18425/77754)
Coverage Report: http://coverage.selectdb-in.cc/coverage/59c54f323b6f549e66b88e51169775c58c89fca4_59c54f323b6f549e66b88e51169775c58c89fca4/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1929545200
<details>
<summary>TPC-DS: <b>Total hot run time: 174371 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 32a23524fe4a99f1a696776ed3ea5e9a0e78027a, data reload: false
query1 928 344 329 329
query2 6561 1988 1893 1893
query3 6708 205 200 200
query4 31715 22051 21992 21992
query5 4228 374 425 374
query6 269 171 170 170
query7 4603 305 285 285
query8 255 174 173 173
query9 8937 2288 2253 2253
query10 414 243 241 241
query11 18597 15509 15289 15289
query12 131 74 73 73
query13 1619 411 415 411
query14 9763 7049 6937 6937
query15 251 173 189 173
query16 8079 267 247 247
query17 1854 552 500 500
query18 2106 268 268 268
query19 191 142 138 138
query20 85 74 79 74
query21 203 129 112 112
query22 4898 4658 4584 4584
query23 30746 30092 30118 30092
query24 6139 2759 2778 2759
query25 484 339 335 335
query26 706 146 155 146
query27 2013 307 302 302
query28 3585 1864 1839 1839
query29 863 621 582 582
query30 206 134 142 134
query31 906 711 687 687
query32 93 54 54 54
query33 456 221 221 221
query34 816 460 462 460
query35 850 781 763 763
query36 1019 945 918 918
query37 94 56 57 56
query38 3298 3118 3090 3090
query39 1314 1272 1244 1244
query40 181 93 91 91
query41 38 36 34 34
query42 94 89 94 89
query43 483 489 485 485
query44 1040 683 694 683
query45 190 179 174 174
query46 1035 678 648 648
query47 1613 1490 1540 1490
query48 431 359 355 355
query49 620 296 282 282
query50 777 382 372 372
query51 5251 5123 5176 5123
query52 94 87 81 81
query53 328 267 269 267
query54 248 212 211 211
query55 81 77 75 75
query56 227 200 200 200
query57 973 875 934 875
query58 200 174 183 174
query59 2518 2229 2226 2226
query60 240 207 211 207
query61 85 84 86 84
query62 523 347 341 341
query63 297 260 273 260
query64 4253 3564 3674 3564
query65 3275 3217 3225 3217
query66 799 318 300 300
query67 14536 13998 14053 13998
query68 3915 547 546 546
query69 466 325 321 321
query70 1253 1218 1219 1218
query71 319 248 242 242
query72 5782 2837 2665 2665
query73 722 345 335 335
query74 6515 6330 6214 6214
query75 3010 2313 2359 2313
query76 2575 940 887 887
query77 344 228 221 221
query78 9174 8851 8706 8706
query79 2587 512 491 491
query80 1502 360 336 336
query81 504 196 196 196
query82 647 85 78 78
query83 244 118 126 118
query84 248 81 84 81
query85 1679 331 326 326
query86 493 296 289 289
query87 3444 3197 3171 3171
query88 3813 2374 2369 2369
query89 429 349 360 349
query90 1967 166 166 166
query91 154 120 122 120
query92 59 43 43 43
query93 2754 475 463 463
query94 1287 172 172 172
query95 487 368 352 352
query96 590 288 275 275
query97 4237 4125 4115 4115
query98 229 194 189 189
query99 1133 700 722 700
Total cold run time: 268975 ms
Total hot run time: 174371 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](Top-N opt) evicting quering rowsets in prior to correct use_cou… [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30904:
URL: https://github.com/apache/doris/pull/30904#issuecomment-1931183927
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org