You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zzzxl1993 (via GitHub)" <gi...@apache.org> on 2025/12/25 10:41:03 UTC
[PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
zzzxl1993 opened a new pull request, #59363:
URL: https://github.com/apache/doris/pull/59363
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3691387159
<details>
<summary>TPC-DS: <b>Total hot run time: 179334 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5f4c38a01fc91d3e4f6bdb63eeab5698951015ab, data reload: false
query5 4964 585 442 442
query6 348 253 225 225
query7 4226 464 284 284
query8 336 258 247 247
query9 8793 2525 2579 2525
query10 536 379 338 338
query11 15700 15015 14678 14678
query12 180 118 122 118
query13 1259 513 392 392
query14 6231 3031 2737 2737
query14_1 2681 2648 2644 2644
query15 205 196 178 178
query16 915 459 405 405
query17 1144 712 611 611
query18 2544 449 347 347
query19 237 232 205 205
query20 126 118 121 118
query21 227 139 130 130
query22 3902 4056 3844 3844
query23 16899 16318 16207 16207
query23_1 16026 16027 16154 16027
query24 7444 1653 1237 1237
query24_1 1274 1238 1244 1238
query25 607 510 466 466
query26 1250 283 165 165
query27 2732 471 312 312
query28 4445 2167 2131 2131
query29 791 556 465 465
query30 312 235 212 212
query31 819 700 601 601
query32 83 75 71 71
query33 530 325 291 291
query34 885 912 543 543
query35 757 795 706 706
query36 866 918 797 797
query37 135 91 85 85
query38 2997 3056 3018 3018
query39 780 736 745 736
query39_1 696 697 706 697
query40 221 139 121 121
query41 66 62 61 61
query42 115 102 106 102
query43 427 441 402 402
query44 1319 739 736 736
query45 193 187 181 181
query46 870 991 605 605
query47 1693 1724 1645 1645
query48 314 332 242 242
query49 658 432 338 338
query50 659 289 219 219
query51 3818 3831 3945 3831
query52 102 115 101 101
query53 319 348 291 291
query54 279 257 252 252
query55 80 77 71 71
query56 299 329 309 309
query57 1171 1159 1108 1108
query58 271 254 256 254
query59 2353 2490 2362 2362
query60 310 306 282 282
query61 158 158 170 158
query62 727 753 660 660
query63 336 297 296 296
query64 4977 1307 1007 1007
query65 4002 3967 3904 3904
query66 1427 431 311 311
query67 15158 15288 14905 14905
query68 5181 1006 717 717
query69 530 343 310 310
query70 1052 986 968 968
query71 373 293 263 263
query72 6046 4955 5064 4955
query73 704 625 311 311
query74 9006 8816 8643 8643
query75 3197 3162 2794 2794
query76 3879 1150 721 721
query77 520 385 294 294
query78 9427 9610 8982 8982
query79 1148 857 607 607
query80 766 683 587 587
query81 507 272 236 236
query82 225 139 104 104
query83 270 269 286 269
query84 259 116 107 107
query85 901 517 454 454
query86 332 283 299 283
query87 3164 3227 3184 3184
query88 3161 2239 2244 2239
query89 468 436 394 394
query90 1969 153 155 153
query91 180 165 149 149
query92 70 65 68 65
query93 1016 896 548 548
query94 482 291 278 278
query95 587 324 352 324
query96 597 462 207 207
query97 2274 2338 2228 2228
query98 214 196 199 196
query99 1317 1321 1383 1321
Total cold run time: 257180 ms
Total hot run time: 179334 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3691296974
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3698699952
<details>
<summary>ClickBench: <b>Total hot run time: 26.76 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bbc591f032bdb135ba06bb8441135c7fbb42dea8, data reload: false
query1 0.06 0.04 0.04
query2 0.10 0.06 0.05
query3 0.25 0.09 0.09
query4 1.60 0.11 0.11
query5 0.26 0.24 0.24
query6 1.15 0.67 0.64
query7 0.03 0.02 0.02
query8 0.06 0.04 0.05
query9 0.58 0.50 0.49
query10 0.55 0.55 0.54
query11 0.15 0.11 0.12
query12 0.16 0.13 0.14
query13 0.63 0.60 0.59
query14 0.97 0.99 0.97
query15 0.82 0.78 0.81
query16 0.41 0.43 0.40
query17 1.06 1.06 0.97
query18 0.23 0.22 0.21
query19 1.97 1.77 1.87
query20 0.02 0.02 0.01
query21 15.43 0.28 0.14
query22 4.81 0.05 0.06
query23 16.08 0.29 0.10
query24 1.29 0.25 0.25
query25 0.11 0.08 0.08
query26 0.15 0.14 0.14
query27 0.07 0.07 0.08
query28 3.88 1.06 0.89
query29 12.58 3.89 3.12
query30 0.28 0.13 0.13
query31 2.81 0.61 0.38
query32 3.23 0.55 0.46
query33 3.01 3.03 2.99
query34 16.77 5.13 4.42
query35 4.53 4.51 4.52
query36 0.66 0.51 0.48
query37 0.10 0.07 0.07
query38 0.07 0.04 0.03
query39 0.05 0.03 0.03
query40 0.18 0.14 0.13
query41 0.09 0.03 0.03
query42 0.04 0.04 0.03
query43 0.04 0.04 0.03
Total cold run time: 97.32 s
Total hot run time: 26.76 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3695489238
# BE Regression && UT Coverage Report
Increment line coverage `78.12% (50/64)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59363_17e616254da96d2064dac671ca22b14c84b41785_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59363_17e616254da96d2064dac671ca22b14c84b41785_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 72.17% (25038/34694) |
| Line Coverage | 58.87% (262887/446525) |
| Region Coverage | 53.76% (218343/406176) |
| Branch Coverage | 55.27% (93689/169518) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "zzzxl1993 (via GitHub)" <gi...@apache.org>.
zzzxl1993 merged PR #59363:
URL: https://github.com/apache/doris/pull/59363
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3692131398
<details>
<summary>TPC-H: <b>Total hot run time: 35594 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 17e616254da96d2064dac671ca22b14c84b41785, data reload: false
------ Round 1 ----------------------------------
q1 17610 4300 4126 4126
q2 2016 359 229 229
q3 10168 1343 776 776
q4 10203 787 315 315
q5 7507 2186 1950 1950
q6 194 174 137 137
q7 1009 863 723 723
q8 9351 1556 1199 1199
q9 7118 5404 5418 5404
q10 6882 2426 2006 2006
q11 552 330 309 309
q12 737 772 559 559
q13 17773 3686 3056 3056
q14 296 291 269 269
q15 602 513 506 506
q16 699 701 630 630
q17 712 786 623 623
q18 7809 7133 7227 7133
q19 1108 978 625 625
q20 415 367 247 247
q21 4320 3941 3795 3795
q22 1047 1009 977 977
Total cold run time: 108128 ms
Total hot run time: 35594 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4165 4149 4079 4079
q2 329 404 323 323
q3 2127 2684 2275 2275
q4 1369 1783 1306 1306
q5 4234 4803 4689 4689
q6 235 177 139 139
q7 2089 2044 1863 1863
q8 2742 2519 2552 2519
q9 7525 7489 7482 7482
q10 3082 3323 2883 2883
q11 633 545 546 545
q12 766 802 617 617
q13 3565 4251 3207 3207
q14 289 291 282 282
q15 547 511 562 511
q16 717 694 651 651
q17 1228 1396 1410 1396
q18 8039 7697 7687 7687
q19 915 918 917 917
q20 1868 2014 1805 1805
q21 4716 4321 4293 4293
q22 1102 1033 985 985
Total cold run time: 52282 ms
Total hot run time: 50454 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3692155446
<details>
<summary>ClickBench: <b>Total hot run time: 27.14 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 17e616254da96d2064dac671ca22b14c84b41785, data reload: false
query1 0.05 0.05 0.05
query2 0.10 0.06 0.05
query3 0.26 0.09 0.08
query4 1.61 0.11 0.11
query5 0.28 0.28 0.26
query6 1.16 0.64 0.63
query7 0.03 0.03 0.02
query8 0.06 0.04 0.04
query9 0.55 0.48 0.50
query10 0.55 0.56 0.55
query11 0.15 0.11 0.11
query12 0.16 0.12 0.12
query13 0.63 0.61 0.60
query14 0.98 0.98 0.99
query15 0.83 0.80 0.79
query16 0.43 0.39 0.41
query17 1.06 1.07 1.01
query18 0.23 0.22 0.21
query19 1.82 1.89 1.78
query20 0.02 0.01 0.02
query21 15.44 0.28 0.15
query22 4.94 0.06 0.05
query23 16.18 0.28 0.11
query24 2.18 0.26 0.19
query25 0.09 0.11 0.05
query26 0.14 0.14 0.14
query27 0.06 0.05 0.05
query28 4.83 1.23 1.02
query29 12.60 3.99 3.26
query30 0.28 0.15 0.12
query31 2.83 0.61 0.39
query32 3.23 0.55 0.46
query33 2.97 3.08 2.96
query34 16.72 5.15 4.56
query35 4.59 4.59 4.54
query36 0.64 0.50 0.48
query37 0.10 0.07 0.07
query38 0.07 0.04 0.03
query39 0.04 0.03 0.03
query40 0.17 0.15 0.13
query41 0.09 0.04 0.03
query42 0.05 0.04 0.02
query43 0.05 0.03 0.03
Total cold run time: 99.25 s
Total hot run time: 27.14 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3701336976
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "zzzxl1993 (via GitHub)" <gi...@apache.org>.
zzzxl1993 commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3698469945
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3698687838
<details>
<summary>TPC-DS: <b>Total hot run time: 174873 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bbc591f032bdb135ba06bb8441135c7fbb42dea8, data reload: false
query5 4399 589 448 448
query6 330 233 217 217
query7 4205 453 286 286
query8 347 258 241 241
query9 8756 2619 2618 2618
query10 523 381 344 344
query11 15347 15080 14987 14987
query12 176 119 120 119
query13 1262 498 410 410
query14 6301 3011 2757 2757
query14_1 2649 2641 2636 2636
query15 213 200 179 179
query16 985 505 462 462
query17 1108 711 583 583
query18 2518 442 349 349
query19 242 232 198 198
query20 123 120 116 116
query21 221 138 127 127
query22 3938 3918 3850 3850
query23 16028 15516 15269 15269
query23_1 15604 15462 15344 15344
query24 7409 1586 1196 1196
query24_1 1232 1206 1202 1202
query25 570 489 434 434
query26 1241 267 165 165
query27 2765 470 305 305
query28 4504 2207 2186 2186
query29 811 559 468 468
query30 318 243 210 210
query31 808 631 572 572
query32 80 74 67 67
query33 557 333 294 294
query34 911 908 542 542
query35 754 806 700 700
query36 856 835 785 785
query37 127 88 72 72
query38 2730 2787 2666 2666
query39 771 758 725 725
query39_1 726 729 710 710
query40 223 133 115 115
query41 68 62 61 61
query42 104 105 103 103
query43 467 447 395 395
query44 1395 760 751 751
query45 190 176 173 173
query46 848 963 618 618
query47 1389 1482 1386 1386
query48 317 330 242 242
query49 597 420 337 337
query50 631 274 214 214
query51 3734 3781 3900 3781
query52 106 109 99 99
query53 290 326 271 271
query54 279 260 258 258
query55 81 77 79 77
query56 281 311 301 301
query57 1028 985 966 966
query58 272 252 248 248
query59 2110 2124 2195 2124
query60 325 322 339 322
query61 157 163 162 162
query62 382 349 301 301
query63 306 263 280 263
query64 4929 1295 994 994
query65 3759 3752 3703 3703
query66 1446 430 316 316
query67 15022 15723 14740 14740
query68 8224 1008 732 732
query69 492 351 306 306
query70 1020 971 980 971
query71 380 301 285 285
query72 6276 4834 5011 4834
query73 710 630 314 314
query74 8837 8724 8533 8533
query75 2937 2898 2537 2537
query76 3813 1061 662 662
query77 530 372 282 282
query78 9674 10050 9129 9129
query79 1269 926 611 611
query80 647 595 492 492
query81 514 266 237 237
query82 217 148 111 111
query83 268 261 241 241
query84 260 120 96 96
query85 908 523 462 462
query86 390 330 310 310
query87 2871 2848 2787 2787
query88 3250 2284 2298 2284
query89 394 354 327 327
query90 2171 160 151 151
query91 177 174 146 146
query92 82 64 64 64
query93 1151 904 561 561
query94 581 327 304 304
query95 592 335 363 335
query96 582 461 220 220
query97 2334 2375 2298 2298
query98 218 211 196 196
query99 610 575 529 529
Total cold run time: 254654 ms
Total hot run time: 174873 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3691377059
<details>
<summary>TPC-H: <b>Total hot run time: 36675 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5f4c38a01fc91d3e4f6bdb63eeab5698951015ab, data reload: false
------ Round 1 ----------------------------------
q1 17602 4110 4056 4056
q2 1998 359 242 242
q3 10174 1349 763 763
q4 10242 894 326 326
q5 7535 2097 1998 1998
q6 182 169 136 136
q7 1019 856 701 701
q8 9344 1491 1239 1239
q9 7143 5398 5407 5398
q10 6843 2425 1982 1982
q11 542 330 293 293
q12 651 729 586 586
q13 17760 3730 3078 3078
q14 292 295 278 278
q15 587 517 512 512
q16 710 674 626 626
q17 694 813 539 539
q18 8216 7880 7927 7880
q19 1104 1006 658 658
q20 459 371 254 254
q21 4767 4115 4102 4102
q22 1136 1073 1028 1028
Total cold run time: 109000 ms
Total hot run time: 36675 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4322 4245 4268 4245
q2 338 442 339 339
q3 2470 2846 2506 2506
q4 1534 1911 1443 1443
q5 4595 4593 4439 4439
q6 208 166 133 133
q7 2062 1961 1820 1820
q8 2741 2564 2598 2564
q9 7562 7497 7518 7497
q10 3135 3174 2676 2676
q11 615 524 510 510
q12 648 714 544 544
q13 3266 3641 3029 3029
q14 267 274 252 252
q15 541 502 489 489
q16 615 711 619 619
q17 1110 1386 1326 1326
q18 7374 7096 7051 7051
q19 841 796 841 796
q20 1875 1943 1779 1779
q21 4612 4311 4151 4151
q22 1082 1042 988 988
Total cold run time: 51813 ms
Total hot run time: 49196 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3692262020
# BE UT Coverage Report
Increment line coverage `7.81% (5/64)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/17e616254da96d2064dac671ca22b14c84b41785_17e616254da96d2064dac671ca22b14c84b41785/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/17e616254da96d2064dac671ca22b14c84b41785_17e616254da96d2064dac671ca22b14c84b41785/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.40% (18951/35489) |
| Line Coverage | 39.26% (175758/447717) |
| Region Coverage | 33.83% (135986/402011) |
| Branch Coverage | 34.76% (58720/168954) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3701339286
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3701677924
# BE Regression && UT Coverage Report
Increment line coverage `90.38% (47/52)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59363_bbc591f032bdb135ba06bb8441135c7fbb42dea8_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59363_bbc591f032bdb135ba06bb8441135c7fbb42dea8_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 72.16% (25050/34713) |
| Line Coverage | 58.92% (263315/446903) |
| Region Coverage | 53.84% (218918/406587) |
| Branch Coverage | 55.34% (93924/169708) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "zzzxl1993 (via GitHub)" <gi...@apache.org>.
zzzxl1993 commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3692008899
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3692150460
<details>
<summary>TPC-DS: <b>Total hot run time: 179638 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 17e616254da96d2064dac671ca22b14c84b41785, data reload: false
query5 4926 604 454 454
query6 351 225 208 208
query7 4224 479 277 277
query8 303 261 258 258
query9 8759 2491 2510 2491
query10 552 375 313 313
query11 15468 15057 15178 15057
query12 176 118 114 114
query13 1261 505 375 375
query14 5971 3025 2748 2748
query14_1 2694 2665 2647 2647
query15 210 201 174 174
query16 826 433 436 433
query17 1076 675 598 598
query18 2469 433 331 331
query19 230 223 195 195
query20 120 112 113 112
query21 222 144 117 117
query22 4263 4144 4183 4144
query23 17168 16173 15945 15945
query23_1 16012 16102 16071 16071
query24 7368 1645 1214 1214
query24_1 1254 1217 1242 1217
query25 550 478 453 453
query26 1245 267 159 159
query27 2765 456 303 303
query28 4476 2118 2106 2106
query29 822 544 440 440
query30 324 250 217 217
query31 819 724 625 625
query32 80 70 68 68
query33 557 328 283 283
query34 916 905 550 550
query35 771 789 698 698
query36 858 914 829 829
query37 130 93 76 76
query38 3027 3107 2994 2994
query39 776 755 733 733
query39_1 688 705 707 705
query40 229 158 128 128
query41 69 64 62 62
query42 108 114 112 112
query43 436 419 409 409
query44 1303 757 736 736
query45 196 194 179 179
query46 880 982 595 595
query47 1649 1692 1618 1618
query48 309 322 245 245
query49 617 423 345 345
query50 653 298 230 230
query51 3812 3867 3791 3791
query52 104 113 97 97
query53 326 346 289 289
query54 277 248 271 248
query55 79 74 74 74
query56 314 301 302 301
query57 1171 1149 1102 1102
query58 269 254 239 239
query59 2429 2493 2435 2435
query60 314 306 293 293
query61 157 159 161 159
query62 718 753 672 672
query63 326 296 308 296
query64 5064 1346 1107 1107
query65 4035 3960 3935 3935
query66 1480 446 334 334
query67 15150 15245 14868 14868
query68 2846 1041 761 761
query69 498 365 327 327
query70 1049 962 1006 962
query71 319 309 284 284
query72 6507 5121 5236 5121
query73 506 587 300 300
query74 8924 8824 8760 8760
query75 3159 3184 2792 2792
query76 3450 1144 738 738
query77 503 395 287 287
query78 9451 9644 8920 8920
query79 1059 880 631 631
query80 1235 651 569 569
query81 558 272 230 230
query82 425 136 101 101
query83 362 263 241 241
query84 259 116 107 107
query85 950 505 504 504
query86 383 293 290 290
query87 3233 3245 3112 3112
query88 3245 2290 2261 2261
query89 468 430 395 395
query90 1885 158 146 146
query91 172 171 146 146
query92 73 69 69 69
query93 1094 891 558 558
query94 532 299 272 272
query95 588 382 301 301
query96 590 468 211 211
query97 2255 2320 2224 2224
query98 218 195 191 191
query99 1336 1410 1284 1284
Total cold run time: 255721 ms
Total hot run time: 179638 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3691391440
<details>
<summary>ClickBench: <b>Total hot run time: 26.91 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5f4c38a01fc91d3e4f6bdb63eeab5698951015ab, data reload: false
query1 0.05 0.04 0.05
query2 0.09 0.05 0.04
query3 0.25 0.09 0.08
query4 1.60 0.12 0.11
query5 0.27 0.25 0.26
query6 1.18 0.63 0.61
query7 0.03 0.03 0.03
query8 0.06 0.04 0.04
query9 0.56 0.51 0.50
query10 0.55 0.55 0.54
query11 0.15 0.11 0.10
query12 0.14 0.11 0.13
query13 0.62 0.61 0.62
query14 0.99 0.96 1.00
query15 0.82 0.78 0.79
query16 0.39 0.39 0.43
query17 0.97 1.05 0.99
query18 0.23 0.22 0.21
query19 1.93 1.87 1.82
query20 0.02 0.01 0.01
query21 15.46 0.29 0.14
query22 4.76 0.05 0.05
query23 16.08 0.29 0.10
query24 1.43 0.28 0.18
query25 0.08 0.12 0.06
query26 0.14 0.14 0.13
query27 0.07 0.05 0.05
query28 4.07 1.22 1.03
query29 12.65 3.98 3.20
query30 0.27 0.13 0.12
query31 2.81 0.60 0.38
query32 3.22 0.53 0.47
query33 3.04 2.98 3.00
query34 16.95 5.20 4.46
query35 4.53 4.52 4.56
query36 0.68 0.50 0.48
query37 0.11 0.07 0.06
query38 0.07 0.05 0.04
query39 0.04 0.03 0.03
query40 0.18 0.14 0.13
query41 0.09 0.03 0.02
query42 0.05 0.03 0.03
query43 0.04 0.03 0.03
Total cold run time: 97.72 s
Total hot run time: 26.91 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "zzzxl1993 (via GitHub)" <gi...@apache.org>.
zzzxl1993 commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3691297144
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3692394769
# BE Regression && UT Coverage Report
Increment line coverage `78.12% (50/64)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/59363_17e616254da96d2064dac671ca22b14c84b41785_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/59363_17e616254da96d2064dac671ca22b14c84b41785_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 72.17% (25038/34694) |
| Line Coverage | 58.88% (262901/446525) |
| Region Coverage | 53.75% (218326/406176) |
| Branch Coverage | 55.27% (93695/169518) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3698929168
# BE UT Coverage Report
Increment line coverage `9.62% (5/52)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/bbc591f032bdb135ba06bb8441135c7fbb42dea8_bbc591f032bdb135ba06bb8441135c7fbb42dea8/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/bbc591f032bdb135ba06bb8441135c7fbb42dea8_bbc591f032bdb135ba06bb8441135c7fbb42dea8/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.37% (18950/35508) |
| Line Coverage | 39.25% (175869/448095) |
| Region Coverage | 33.81% (136051/402424) |
| Branch Coverage | 34.74% (58767/169146) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [opt](inverted index) optimize S3 operations for inverted index [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #59363:
URL: https://github.com/apache/doris/pull/59363#issuecomment-3698661192
<details>
<summary>TPC-H: <b>Total hot run time: 35753 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bbc591f032bdb135ba06bb8441135c7fbb42dea8, data reload: false
------ Round 1 ----------------------------------
q1 17650 4227 4094 4094
q2 2030 348 240 240
q3 10216 1282 730 730
q4 10220 856 330 330
q5 7788 2121 1951 1951
q6 219 176 137 137
q7 955 808 671 671
q8 9296 1439 1161 1161
q9 6904 5183 5252 5183
q10 6834 1805 1415 1415
q11 525 315 284 284
q12 753 733 607 607
q13 17775 3836 3058 3058
q14 298 308 270 270
q15 594 519 515 515
q16 703 705 623 623
q17 735 783 610 610
q18 7593 7801 8229 7801
q19 1336 1017 660 660
q20 448 383 262 262
q21 4516 4105 4214 4105
q22 1148 1105 1046 1046
Total cold run time: 108536 ms
Total hot run time: 35753 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4377 4241 4333 4241
q2 325 427 327 327
q3 2363 2806 2447 2447
q4 1402 1801 1423 1423
q5 4489 4518 4365 4365
q6 220 172 126 126
q7 2105 1888 1755 1755
q8 2563 2341 2357 2341
q9 7226 7373 7142 7142
q10 2454 2677 2224 2224
q11 545 457 443 443
q12 660 706 548 548
q13 3274 3771 3168 3168
q14 281 289 256 256
q15 532 491 488 488
q16 629 642 621 621
q17 1086 1248 1332 1248
q18 7486 7456 7312 7312
q19 867 834 856 834
q20 1888 1980 1805 1805
q21 4471 4323 4191 4191
q22 1111 1057 990 990
Total cold run time: 50354 ms
Total hot run time: 48295 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org