You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "HHoflittlefish777 (via GitHub)" <gi...@apache.org> on 2024/02/06 10:32:47 UTC
[PR] [branch-2.0](rowset writer) disable check segment when build rowset meta by default [doris]
HHoflittlefish777 opened a new pull request, #30921:
URL: https://github.com/apache/doris/pull/30921
## Proposed changes
(#30857)
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0](rowset writer) disable check segment when build rowset meta by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30921:
URL: https://github.com/apache/doris/pull/30921#issuecomment-1929342989
<details>
<summary>TPC-H: <b>Total hot run time: 50065 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 109aba8f54df9f230d409666c2572b43825de755, data reload: false
------ Round 1 ----------------------------------
q1 17680 4572 4377 4377
q2 2058 146 139 139
q3 10441 2023 1887 1887
q4 10247 1237 1334 1237
q5 8519 3985 3982 3982
q6 229 122 121 121
q7 2036 1619 1586 1586
q8 9268 2738 2734 2734
q9 11038 10574 10549 10549
q10 8640 3549 3521 3521
q11 426 236 246 236
q12 473 297 300 297
q13 18362 3944 4016 3944
q14 346 334 319 319
q15 505 454 453 453
q16 693 604 588 588
q17 1142 989 993 989
q18 7253 6848 6820 6820
q19 1687 1596 1481 1481
q20 517 321 300 300
q21 4517 4165 4118 4118
q22 498 401 387 387
Total cold run time: 116575 ms
Total hot run time: 50065 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4352 4345 4325 4325
q2 320 220 230 220
q3 4178 4138 4128 4128
q4 2776 2760 2753 2753
q5 7281 7218 7238 7218
q6 242 119 118 118
q7 3201 2884 2871 2871
q8 4391 4451 4521 4451
q9 16927 16916 16886 16886
q10 4233 4296 4299 4296
q11 761 704 661 661
q12 1021 852 844 844
q13 4291 3763 3746 3746
q14 457 418 417 417
q15 492 454 465 454
q16 758 715 695 695
q17 3809 3921 3811 3811
q18 8816 8699 8790 8699
q19 1742 1698 1638 1638
q20 2384 2194 2081 2081
q21 8614 8620 8534 8534
q22 1029 971 937 937
Total cold run time: 82075 ms
Total hot run time: 79783 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0](rowset writer) disable check segment when build rowset meta by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30921:
URL: https://github.com/apache/doris/pull/30921#issuecomment-1929221112
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0](rowset writer) disable check segment when build rowset meta by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30921:
URL: https://github.com/apache/doris/pull/30921#issuecomment-1929364474
<details>
<summary>TPC-DS: <b>Total hot run time: 239069 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 109aba8f54df9f230d409666c2572b43825de755, data reload: false
query1 927 390 382 382
query2 6517 2200 2213 2200
query3 6912 198 197 197
query4 21547 18028 17937 17937
query5 19726 6237 6179 6179
query6 275 209 231 209
query7 4149 304 300 300
query8 258 260 227 227
query9 3080 2669 2584 2584
query10 414 279 307 279
query11 11326 10635 10749 10635
query12 127 72 72 72
query13 5652 643 615 615
query14 17680 13316 13225 13225
query15 363 223 239 223
query16 6434 256 261 256
query17 3228 876 866 866
query18 2271 396 398 396
query19 199 144 141 141
query20 76 81 74 74
query21 185 90 91 90
query22 5251 5018 5109 5018
query23 32395 31872 31615 31615
query24 6926 6578 6429 6429
query25 499 432 410 410
query26 526 166 159 159
query27 1880 291 292 291
query28 6072 2206 2195 2195
query29 2909 2860 2899 2860
query30 235 161 158 158
query31 903 736 748 736
query32 64 56 57 56
query33 384 248 252 248
query34 842 458 483 458
query35 1113 932 875 875
query36 1697 1434 1398 1398
query37 85 59 57 57
query38 3073 2957 2969 2957
query39 1377 1312 1327 1312
query40 197 85 94 85
query41 33 35 37 35
query42 83 87 80 80
query43 648 635 583 583
query44 1124 732 725 725
query45 235 229 230 229
query46 1206 961 982 961
query47 1886 2008 1717 1717
query48 985 690 664 664
query49 627 378 352 352
query50 858 586 609 586
query51 5512 5409 5413 5409
query52 77 85 80 80
query53 454 314 316 314
query54 2425 2255 2241 2241
query55 81 88 82 82
query56 218 212 202 202
query57 1180 1086 1065 1065
query58 215 205 213 205
query59 3562 3169 3101 3101
query60 211 202 201 201
query61 83 85 85 85
query62 834 510 515 510
query63 472 333 339 333
query64 2019 1250 1284 1250
query65 3706 3601 3607 3601
query66 817 358 359 358
query67 17398 15607 15426 15426
query68 7091 664 636 636
query69 559 343 352 343
query70 1986 1806 1858 1806
query71 401 319 309 309
query72 4521 3281 3334 3281
query73 716 313 320 313
query74 6260 5946 5840 5840
query75 4345 3454 3535 3454
query76 4525 1180 1203 1180
query77 548 240 249 240
query78 32228 49310 51917 49310
query79 15543 660 627 627
query80 4462 371 376 371
query81 560 234 226 226
query82 1258 105 95 95
query83 327 135 131 131
query84 254 67 70 67
query85 2063 285 281 281
query86 446 373 384 373
query87 3235 3020 3044 3020
query88 7068 2297 2289 2289
query89 439 271 297 271
query90 2532 216 214 214
query91 159 115 119 115
query92 61 50 51 50
query93 6163 593 548 548
query94 1693 202 209 202
query95 1135 1058 1047 1047
query96 645 307 326 307
query97 6543 6297 6392 6297
query98 182 180 177 177
query99 3675 850 933 850
Total cold run time: 346346 ms
Total hot run time: 239069 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0](rowset writer) disable check segment when build rowset meta by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30921:
URL: https://github.com/apache/doris/pull/30921#issuecomment-1929372917
<details>
<summary>ClickBench: <b>Total hot run time: 29.95 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 109aba8f54df9f230d409666c2572b43825de755, data reload: false
query1 0.03 0.02 0.02
query2 0.06 0.02 0.02
query3 0.25 0.04 0.05
query4 1.82 0.07 0.06
query5 0.53 0.52 0.52
query6 1.26 0.63 0.63
query7 0.02 0.01 0.00
query8 0.03 0.02 0.03
query9 0.52 0.48 0.48
query10 0.54 0.54 0.55
query11 0.12 0.09 0.09
query12 0.11 0.09 0.09
query13 0.62 0.60 0.61
query14 0.77 0.78 0.81
query15 0.78 0.76 0.77
query16 0.37 0.37 0.36
query17 1.01 1.00 1.00
query18 0.23 0.23 0.26
query19 1.95 1.82 1.86
query20 0.01 0.01 0.01
query21 15.55 0.54 0.54
query22 1.94 1.90 0.98
query23 16.90 1.02 0.98
query24 6.81 1.18 0.78
query25 1.50 0.12 0.12
query26 0.30 0.14 0.13
query27 0.11 0.11 0.10
query28 7.34 0.72 0.71
query29 12.61 2.38 2.24
query30 0.64 0.57 0.51
query31 2.80 0.37 0.38
query32 3.43 0.51 0.50
query33 3.06 3.08 3.07
query34 15.28 4.76 4.80
query35 4.86 4.84 4.88
query36 1.08 1.03 1.02
query37 0.06 0.04 0.05
query38 0.04 0.02 0.03
query39 0.01 0.01 0.01
query40 0.15 0.13 0.14
query41 0.06 0.01 0.01
query42 0.02 0.01 0.01
query43 0.03 0.02 0.01
Total cold run time: 105.61 s
Total hot run time: 29.95 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0](rowset writer) disable check segment when build rowset meta by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30921:
URL: https://github.com/apache/doris/pull/30921#issuecomment-1929338887
TeamCity be ut coverage result:
Function Coverage: 37.81% (8030/21238)
Line Coverage: 29.48% (65451/222011)
Region Coverage: 28.91% (33628/116300)
Branch Coverage: 24.78% (17260/69648)
Coverage Report: http://coverage.selectdb-in.cc/coverage/109aba8f54df9f230d409666c2572b43825de755_109aba8f54df9f230d409666c2572b43825de755/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0](rowset writer) disable check segment when build rowset meta by default [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #30921:
URL: https://github.com/apache/doris/pull/30921
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0](rowset writer) disable check segment when build rowset meta by default [doris]
Posted by "HHoflittlefish777 (via GitHub)" <gi...@apache.org>.
HHoflittlefish777 commented on PR #30921:
URL: https://github.com/apache/doris/pull/30921#issuecomment-1929221334
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0](rowset writer) disable check segment when build rowset meta by default [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30921:
URL: https://github.com/apache/doris/pull/30921#issuecomment-1929232913
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [branch-2.0](rowset writer) disable check segment when build rowset meta by default [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30921:
URL: https://github.com/apache/doris/pull/30921#issuecomment-1929380823
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 109aba8f54df9f230d409666c2572b43825de755 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 21.8 seconds inserted 10000000 Rows, about 458K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org