You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "liaoxin01 (via GitHub)" <gi...@apache.org> on 2024/02/04 08:50:30 UTC
[PR] [fix](segcompaction) enable segcompaction by default [doris]
liaoxin01 opened a new pull request, #30810:
URL: https://github.com/apache/doris/pull/30810
## Proposed changes
Issue Number: close #xxx
PR #28901 has been merged, so we enable segment compaction by default.
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1925658936
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 16185b72162df5a789b2309bbf4d6878172888e8 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 14.7 seconds inserted 10000000 Rows, about 680K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1925643385
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1926114685
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1925823201
run pipelinex_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1925641312
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1926232606
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1926225542
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #30810:
URL: https://github.com/apache/doris/pull/30810
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1925728415
run pipeline_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1926114654
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1926129619
run pipelinex_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1925728566
run pipelinex_p0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1926242342
TeamCity be ut coverage result:
Function Coverage: 36.16% (8628/23858)
Line Coverage: 28.22% (70572/250068)
Region Coverage: 27.23% (36428/133780)
Branch Coverage: 24.02% (18660/77688)
Coverage Report: http://coverage.selectdb-in.cc/coverage/7225f0399f4e531fc29d4e5f0070ffb7f1a89f33_7225f0399f4e531fc29d4e5f0070ffb7f1a89f33/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1925641287
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1925656623
<details>
<summary>TPC-DS: <b>Total hot run time: 173605 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 16185b72162df5a789b2309bbf4d6878172888e8, data reload: false
query1 940 337 333 333
query2 6555 2003 1951 1951
query3 6707 206 201 201
query4 31855 21853 21929 21853
query5 4285 434 341 341
query6 258 165 176 165
query7 4606 278 276 276
query8 246 181 183 181
query9 8943 2322 2297 2297
query10 418 201 210 201
query11 17122 15320 15379 15320
query12 139 76 73 73
query13 1630 428 411 411
query14 9426 7085 6760 6760
query15 259 179 186 179
query16 8150 253 240 240
query17 1912 530 503 503
query18 2100 259 262 259
query19 360 139 135 135
query20 81 75 77 75
query21 189 129 121 121
query22 4886 4700 4557 4557
query23 30746 29881 29956 29881
query24 10472 2766 2769 2766
query25 544 329 330 329
query26 719 146 149 146
query27 2155 289 294 289
query28 5862 1858 1861 1858
query29 890 628 595 595
query30 278 132 141 132
query31 925 686 723 686
query32 94 49 47 47
query33 624 216 213 213
query34 838 440 447 440
query35 861 778 775 775
query36 1049 916 918 916
query37 95 65 57 57
query38 3258 3126 3119 3119
query39 1309 1242 1240 1240
query40 183 94 92 92
query41 38 36 33 33
query42 95 92 89 89
query43 523 479 499 479
query44 1048 683 719 683
query45 193 176 168 168
query46 1050 660 639 639
query47 1646 1502 1526 1502
query48 434 346 362 346
query49 1044 285 291 285
query50 765 368 387 368
query51 5255 5110 5206 5110
query52 105 84 88 84
query53 352 271 263 263
query54 279 208 216 208
query55 78 81 78 78
query56 219 197 201 197
query57 938 881 901 881
query58 206 170 176 170
query59 2392 2384 2177 2177
query60 236 206 210 206
query61 81 83 82 82
query62 620 378 369 369
query63 299 262 259 259
query64 4746 3676 3532 3532
query65 3251 3217 3190 3190
query66 821 329 297 297
query67 14343 14127 14040 14040
query68 4151 537 545 537
query69 466 319 307 307
query70 1246 1232 1185 1185
query71 320 241 260 241
query72 5972 2821 2663 2663
query73 709 323 313 313
query74 6626 6243 6250 6243
query75 3020 2353 2331 2331
query76 2558 945 955 945
query77 363 226 220 220
query78 9125 8826 8529 8529
query79 2654 499 490 490
query80 1976 353 331 331
query81 542 192 196 192
query82 779 85 75 75
query83 253 123 119 119
query84 282 83 79 79
query85 2172 330 317 317
query86 488 305 314 305
query87 3372 3199 3176 3176
query88 3827 2328 2311 2311
query89 445 375 365 365
query90 1986 164 163 163
query91 164 119 119 119
query92 54 42 42 42
query93 2638 473 490 473
query94 1301 173 175 173
query95 464 370 362 362
query96 606 273 266 266
query97 4244 4103 4136 4103
query98 207 197 190 190
query99 1149 692 696 692
Total cold run time: 277185 ms
Total hot run time: 173605 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1925650090
<details>
<summary>TPC-H: <b>Total hot run time: 36913 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 16185b72162df5a789b2309bbf4d6878172888e8, data reload: false
------ Round 1 ----------------------------------
q1 17661 4439 4403 4403
q2 2042 143 134 134
q3 10596 908 912 908
q4 4667 731 737 731
q5 7674 2931 2953 2931
q6 182 117 117 117
q7 1118 721 706 706
q8 9316 2017 1980 1980
q9 7213 6363 6368 6363
q10 8085 2405 2412 2405
q11 423 211 203 203
q12 747 270 275 270
q13 18008 3304 3297 3297
q14 270 247 245 245
q15 522 498 489 489
q16 461 392 402 392
q17 926 514 483 483
q18 6728 5968 6027 5968
q19 1573 1346 1344 1344
q20 652 344 319 319
q21 7035 3108 2925 2925
q22 792 307 300 300
Total cold run time: 106691 ms
Total hot run time: 36913 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4476 4407 4413 4407
q2 323 240 242 240
q3 2983 2825 2844 2825
q4 1824 1575 1578 1575
q5 5203 5219 5132 5132
q6 192 114 114 114
q7 2127 1792 1794 1792
q8 3085 3226 3258 3226
q9 8334 8272 8231 8231
q10 6004 3600 3560 3560
q11 550 453 472 453
q12 729 594 571 571
q13 15865 3105 3077 3077
q14 270 263 264 263
q15 535 497 482 482
q16 523 449 462 449
q17 1832 1664 1679 1664
q18 7902 7664 7470 7470
q19 8953 1503 1503 1503
q20 2114 1934 1895 1895
q21 4837 4436 4577 4436
q22 564 467 455 455
Total cold run time: 79225 ms
Total hot run time: 53820 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1925646620
TeamCity be ut coverage result:
Function Coverage: 36.16% (8627/23858)
Line Coverage: 28.22% (70546/249999)
Region Coverage: 27.23% (36405/133678)
Branch Coverage: 24.02% (18648/77620)
Coverage Report: http://coverage.selectdb-in.cc/coverage/16185b72162df5a789b2309bbf4d6878172888e8_16185b72162df5a789b2309bbf4d6878172888e8/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](segcompaction) enable segcompaction by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30810:
URL: https://github.com/apache/doris/pull/30810#issuecomment-1925658044
<details>
<summary>ClickBench: <b>Total hot run time: 31.27 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 16185b72162df5a789b2309bbf4d6878172888e8, data reload: false
query1 0.04 0.03 0.03
query2 0.06 0.03 0.02
query3 0.23 0.06 0.07
query4 1.65 0.09 0.10
query5 0.52 0.52 0.53
query6 1.20 0.64 0.68
query7 0.02 0.01 0.01
query8 0.03 0.03 0.03
query9 0.56 0.50 0.48
query10 0.55 0.56 0.54
query11 0.13 0.08 0.09
query12 0.11 0.08 0.09
query13 0.61 0.61 0.61
query14 0.78 0.80 0.81
query15 0.80 0.77 0.77
query16 0.39 0.38 0.39
query17 1.04 1.03 1.00
query18 0.24 0.23 0.25
query19 1.92 1.75 1.73
query20 0.01 0.01 0.01
query21 15.40 0.61 0.59
query22 3.16 2.92 1.98
query23 17.22 0.91 0.81
query24 2.80 1.47 0.99
query25 0.23 0.20 0.04
query26 0.60 0.13 0.14
query27 0.06 0.05 0.06
query28 11.10 0.86 0.83
query29 12.49 3.26 3.33
query30 0.60 0.55 0.53
query31 2.78 0.35 0.36
query32 3.32 0.48 0.48
query33 3.29 3.23 3.25
query34 15.71 4.32 4.28
query35 4.31 4.30 4.31
query36 1.10 1.06 1.06
query37 0.07 0.05 0.05
query38 0.04 0.03 0.02
query39 0.02 0.01 0.02
query40 0.16 0.13 0.16
query41 0.08 0.01 0.02
query42 0.03 0.01 0.01
query43 0.02 0.02 0.02
Total cold run time: 105.48 s
Total hot run time: 31.27 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org