You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "liaoxin01 (via GitHub)" <gi...@apache.org> on 2026/05/21 08:02:55 UTC
[PR] [improvement](cloud) Enable packed file and empty rowset optimization by default [doris]
liaoxin01 opened a new pull request, #63475:
URL: https://github.com/apache/doris/pull/63475
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: Cloud mode kept packed file small-file merge and empty rowset metadata skipping disabled by default. This change enables enable_packed_file and skip_writing_empty_rowset_metadata by default so new cloud deployments merge small files and avoid writing metadata for empty rowsets without extra configuration.
### Release note
Enable cloud packed file small-file merge and empty rowset metadata skip optimization by default.
### Check List (For Author)
- Test: Manual test
- git diff --check -- be/src/cloud/config.cpp
- Behavior changed: Yes. Cloud mode now enables packed file small-file merge and skips writing empty rowset metadata by default.
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cloud) Enable packed file and empty rowset optimization by default [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #63475:
URL: https://github.com/apache/doris/pull/63475#issuecomment-4506055211
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cloud) Enable packed file and empty rowset optimization by default [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #63475:
URL: https://github.com/apache/doris/pull/63475#issuecomment-4525680074
/review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cloud) Enable packed file and empty rowset optimization by default [doris]
Posted by "gavinchou (via GitHub)" <gi...@apache.org>.
gavinchou merged PR #63475:
URL: https://github.com/apache/doris/pull/63475
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cloud) Enable packed file and empty rowset optimization by default [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63475:
URL: https://github.com/apache/doris/pull/63475#issuecomment-4506048020
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cloud) Enable packed file and empty rowset optimization by default [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63475:
URL: https://github.com/apache/doris/pull/63475#issuecomment-4510694389
<details>
<summary>TPC-H: <b>Total hot run time: 31068 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f63770cd5cb5d1ad8439148fabfc524be058979e, data reload: false
------ Round 1 ----------------------------------
orders Doris NULL NULL 0 0 0 NULL 0 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 17686 3843 3832 3832
q2 q3 10809 1413 808 808
q4 4686 472 345 345
q5 7648 2265 2053 2053
q6 390 185 149 149
q7 935 787 620 620
q8 9379 1776 1650 1650
q9 6754 4856 4897 4856
q10 6470 2059 1775 1775
q11 426 268 247 247
q12 694 428 296 296
q13 18239 3404 2820 2820
q14 259 252 233 233
q15 q16 811 759 704 704
q17 991 886 948 886
q18 6908 5707 5489 5489
q19 1229 1210 1161 1161
q20 544 419 283 283
q21 5978 2687 2544 2544
q22 468 384 317 317
Total cold run time: 101304 ms
Total hot run time: 31068 ms
----- Round 2, with runtime_filter_mode=off -----
orders Doris NULL NULL 150000000 42 6422171781 NULL 22778155 NULL NULL 2023-12-26 18:27:23 2023-12-26 18:42:55 NULL utf-8 NULL NULL
============================================
q1 4657 4597 4700 4597
q2 q3 4863 5159 4666 4666
q4 2176 2170 1395 1395
q5 4757 4742 4619 4619
q6 232 178 141 141
q7 1835 1786 1563 1563
q8 2379 1952 1988 1952
q9 7231 7156 7179 7156
q10 4443 4369 3967 3967
q11 550 373 350 350
q12 707 716 513 513
q13 3019 3403 2778 2778
q14 271 276 250 250
q15 q16 681 698 619 619
q17 1294 1262 1255 1255
q18 7291 6755 6815 6755
q19 1086 1062 1108 1062
q20 2179 2211 1920 1920
q21 5259 4698 4432 4432
q22 528 447 406 406
Total cold run time: 55438 ms
Total hot run time: 50396 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cloud) Enable packed file and empty rowset optimization by default [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63475:
URL: https://github.com/apache/doris/pull/63475#issuecomment-4510919314
# BE UT Coverage Report
Increment line coverage ` ` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/f63770cd5cb5d1ad8439148fabfc524be058979e_f63770cd5cb5d1ad8439148fabfc524be058979e/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/f63770cd5cb5d1ad8439148fabfc524be058979e_f63770cd5cb5d1ad8439148fabfc524be058979e/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 53.64% (20735/38658) |
| Line Coverage | 37.22% (196336/527469) |
| Region Coverage | 33.59% (154077/458736) |
| Branch Coverage | 34.57% (67073/194042) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cloud) Enable packed file and empty rowset optimization by default [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63475:
URL: https://github.com/apache/doris/pull/63475#issuecomment-4510795346
<details>
<summary>TPC-DS: <b>Total hot run time: 169074 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f63770cd5cb5d1ad8439148fabfc524be058979e, data reload: false
query5 4335 673 526 526
query6 324 227 202 202
query7 4267 583 296 296
query8 329 230 224 224
query9 8850 4065 3993 3993
query10 448 360 318 318
query11 5777 2409 2232 2232
query12 186 128 126 126
query13 1280 609 426 426
query14 5982 5392 5105 5105
query14_1 4402 4372 4389 4372
query15 225 203 183 183
query16 1008 459 442 442
query17 1175 739 616 616
query18 2476 501 360 360
query19 221 202 167 167
query20 139 133 131 131
query21 218 142 121 121
query22 13674 13555 13387 13387
query23 17249 16381 16034 16034
query23_1 16154 16182 16090 16090
query24 7780 1793 1291 1291
query24_1 1304 1299 1307 1299
query25 580 501 438 438
query26 1339 344 169 169
query27 2650 562 337 337
query28 4428 1990 1963 1963
query29 1032 605 490 490
query30 316 245 215 215
query31 1114 1059 931 931
query32 89 74 73 73
query33 544 364 291 291
query34 1202 1141 641 641
query35 767 782 671 671
query36 1284 1325 1192 1192
query37 150 107 97 97
query38 3197 3145 3044 3044
query39 919 932 905 905
query39_1 879 868 868 868
query40 233 143 126 126
query41 82 75 62 62
query42 107 111 112 111
query43 331 324 297 297
query44
query45 220 206 193 193
query46 1110 1210 721 721
query47 2280 2278 2112 2112
query48 405 426 297 297
query49 641 492 391 391
query50 1047 352 251 251
query51 4331 4317 4223 4223
query52 106 105 95 95
query53 257 288 203 203
query54 320 274 251 251
query55 98 96 83 83
query56 309 325 297 297
query57 1405 1362 1305 1305
query58 297 279 272 272
query59 1561 1642 1376 1376
query60 327 323 302 302
query61 157 155 156 155
query62 667 626 557 557
query63 240 200 222 200
query64 2442 787 633 633
query65
query66 1764 481 346 346
query67 30329 29892 29785 29785
query68
query69 466 337 300 300
query70 1029 1037 1020 1020
query71 303 282 270 270
query72 3031 2773 2411 2411
query73 831 728 445 445
query74 5051 4886 4734 4734
query75 2644 2574 2246 2246
query76 2281 1155 796 796
query77 409 416 334 334
query78 12186 12107 11623 11623
query79 1454 1021 741 741
query80 642 568 444 444
query81 453 282 238 238
query82 1365 157 123 123
query83 364 277 246 246
query84 255 145 116 116
query85 867 539 461 461
query86 387 331 309 309
query87 3411 3426 3183 3183
query88 3576 2641 2630 2630
query89 444 383 336 336
query90 1935 189 187 187
query91 178 170 139 139
query92 82 80 73 73
query93 1536 1409 872 872
query94 551 350 315 315
query95 667 493 347 347
query96 1017 864 331 331
query97 2675 2670 2568 2568
query98 233 237 223 223
query99 1102 1094 969 969
Total cold run time: 253691 ms
Total hot run time: 169074 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cloud) Enable packed file and empty rowset optimization by default [doris]
Posted by "hello-stephen (via GitHub)" <gi...@apache.org>.
hello-stephen commented on PR #63475:
URL: https://github.com/apache/doris/pull/63475#issuecomment-4512259409
# BE Regression && UT Coverage Report
Increment line coverage `100% (0/0)` :tada:
[Increment coverage report](http://coverage.selectdb-in.cc/coverage/63475_f63770cd5cb5d1ad8439148fabfc524be058979e_merge/increment_report/index.html)
[Complete coverage report](http://coverage.selectdb-in.cc/coverage/63475_f63770cd5cb5d1ad8439148fabfc524be058979e_merge/report/index.html)
| Category | Coverage |
|-------------------|--------------------|
| Function Coverage | 71.95% (27242/37864) |
| Line Coverage | 55.24% (290624/526123) |
| Region Coverage | 52.36% (242532/463165) |
| Branch Coverage | 53.56% (104325/194769) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [improvement](cloud) Enable packed file and empty rowset optimization by default [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #63475:
URL: https://github.com/apache/doris/pull/63475#issuecomment-4509776055
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org