You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "liaoxin01 (via GitHub)" <gi...@apache.org> on 2024/03/04 13:31:03 UTC
[PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]
liaoxin01 opened a new pull request, #31758:
URL: https://github.com/apache/doris/pull/31758
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
cherry-pick from: #31532
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976692643
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 31c2eabf77ccf5e7c7229ae6a76a249ba35e368c with default session variables
Stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 20.7 seconds inserted 10000000 Rows, about 483K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976637388
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976645884
<details>
<summary>TPC-H: <b>Total hot run time: 49892 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 31c2eabf77ccf5e7c7229ae6a76a249ba35e368c, data reload: false
------ Round 1 ----------------------------------
q1 17725 4348 4367 4348
q2 2053 155 138 138
q3 10505 1927 1974 1927
q4 10332 1233 1334 1233
q5 8476 3944 3977 3944
q6 232 122 126 122
q7 2048 1604 1572 1572
q8 9254 2709 2723 2709
q9 10958 10539 10601 10539
q10 8639 3544 3515 3515
q11 423 235 247 235
q12 470 301 303 301
q13 18367 3935 4037 3935
q14 344 322 330 322
q15 498 453 454 453
q16 687 590 600 590
q17 1108 948 969 948
q18 7334 6759 6872 6759
q19 1680 1541 1511 1511
q20 494 298 305 298
q21 4477 4129 4095 4095
q22 490 410 398 398
Total cold run time: 116594 ms
Total hot run time: 49892 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4301 4288 4329 4288
q2 310 218 222 218
q3 4161 4119 4150 4119
q4 2721 2723 2752 2723
q5 7200 7146 7217 7146
q6 235 118 119 118
q7 3221 2777 2837 2777
q8 4333 4388 4436 4388
q9 17507 17185 17222 17185
q10 4264 4272 4295 4272
q11 736 664 680 664
q12 1015 847 848 847
q13 7043 3730 3731 3730
q14 465 419 430 419
q15 504 452 447 447
q16 748 695 699 695
q17 3843 3813 3890 3813
q18 8787 8689 8787 8689
q19 1717 1725 1691 1691
q20 2442 2125 2124 2124
q21 8529 8518 8519 8518
q22 1079 996 997 996
Total cold run time: 85161 ms
Total hot run time: 79867 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976686025
<details>
<summary>ClickBench: <b>Total hot run time: 29.7 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 31c2eabf77ccf5e7c7229ae6a76a249ba35e368c, data reload: false
query1 0.02 0.02 0.02
query2 0.06 0.03 0.02
query3 0.24 0.05 0.04
query4 1.84 0.06 0.07
query5 0.53 0.52 0.52
query6 1.24 0.60 0.60
query7 0.01 0.01 0.01
query8 0.03 0.02 0.02
query9 0.52 0.49 0.46
query10 0.54 0.54 0.54
query11 0.11 0.08 0.08
query12 0.11 0.08 0.09
query13 0.62 0.62 0.61
query14 0.79 0.79 0.79
query15 0.80 0.75 0.76
query16 0.37 0.40 0.36
query17 1.01 1.02 0.98
query18 0.22 0.27 0.22
query19 1.92 1.78 1.83
query20 0.02 0.01 0.00
query21 15.47 0.55 0.53
query22 2.00 2.56 0.81
query23 17.31 1.06 0.92
query24 4.85 1.63 1.12
query25 0.38 0.12 0.04
query26 0.59 0.13 0.14
query27 0.04 0.04 0.03
query28 7.86 0.76 0.69
query29 12.62 2.26 2.12
query30 0.58 0.58 0.52
query31 2.81 0.40 0.38
query32 3.41 0.52 0.49
query33 3.07 3.05 3.05
query34 15.25 4.85 4.83
query35 4.85 4.86 4.83
query36 1.05 1.01 1.02
query37 0.05 0.05 0.05
query38 0.04 0.02 0.02
query39 0.01 0.01 0.01
query40 0.16 0.14 0.14
query41 0.07 0.01 0.02
query42 0.02 0.01 0.01
query43 0.02 0.02 0.02
Total cold run time: 103.51 s
Total hot run time: 29.7 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976588433
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976674909
<details>
<summary>TPC-DS: <b>Total hot run time: 201665 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 31c2eabf77ccf5e7c7229ae6a76a249ba35e368c, data reload: false
query1 920 389 377 377
query2 6523 2201 2126 2126
query3 7378 253 219 219
query4 23202 18307 17821 17821
query5 19721 6495 6497 6495
query6 411 216 224 216
query7 5248 300 297 297
query8 246 249 229 229
query9 3083 2659 2588 2588
query10 477 278 290 278
query11 11385 10589 10665 10589
query12 119 76 64 64
query13 5623 628 604 604
query14 19132 13720 13729 13720
query15 358 235 231 231
query16 6418 266 257 257
query17 1334 1522 842 842
query18 2307 427 404 404
query19 224 139 147 139
query20 68 73 74 73
query21 183 99 87 87
query22 5138 5027 5043 5027
query23 32446 31988 31824 31824
query24 6747 6450 6481 6450
query25 515 425 409 409
query26 518 163 155 155
query27 1684 289 294 289
query28 6009 2204 2177 2177
query29 2810 2698 2685 2685
query30 236 164 157 157
query31 882 710 739 710
query32 64 58 54 54
query33 385 250 248 248
query34 854 466 484 466
query35 1103 955 916 916
query36 1542 1769 1563 1563
query37 86 60 58 58
query38 3085 2926 2982 2926
query39 1365 1319 1315 1315
query40 195 93 96 93
query41 35 32 32 32
query42 82 85 84 84
query43 570 547 559 547
query44 1115 706 710 706
query45 239 224 227 224
query46 1221 943 987 943
query47 1782 1818 1866 1818
query48 999 669 660 660
query49 609 378 359 359
query50 845 611 608 608
query51 4779 4693 4622 4622
query52 91 79 67 67
query53 444 323 313 313
query54 2629 2448 2482 2448
query55 87 72 78 72
query56 212 196 200 196
query57 1114 1132 1241 1132
query58 213 180 190 180
query59 3621 3166 3206 3166
query60 231 214 203 203
query61 84 82 85 82
query62 816 577 485 485
query63 472 345 345 345
query64 2505 1518 1282 1282
query65 3656 3566 3564 3564
query66 758 361 363 361
query67 18038 15347 15950 15347
query68 8171 628 645 628
query69 583 354 356 354
query70 1895 1835 1536 1536
query71 403 302 305 302
query72 6430 3421 3465 3421
query73 715 322 310 310
query74 6419 5842 5848 5842
query75 4577 3710 3630 3630
query76 4638 1172 1221 1172
query77 562 240 252 240
query78 13360 12439 11477 11477
query79 10340 643 640 640
query80 910 397 388 388
query81 502 229 231 229
query82 1136 101 92 92
query83 158 132 130 130
query84 260 70 66 66
query85 753 295 284 284
query86 392 371 419 371
query87 3250 2993 3021 2993
query88 4951 2329 2318 2318
query89 442 312 294 294
query90 1854 199 198 198
query91 146 114 114 114
query92 53 49 49 49
query93 5911 561 561 561
query94 744 197 195 195
query95 1092 1048 1052 1048
query96 647 324 333 324
query97 6471 6368 6431 6368
query98 187 179 181 179
query99 2928 928 888 888
Total cold run time: 318151 ms
Total hot run time: 201665 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #31758:
URL: https://github.com/apache/doris/pull/31758
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976637368
TeamCity be ut coverage result:
Function Coverage: 37.82% (8040/21256)
Line Coverage: 29.50% (65572/222281)
Region Coverage: 28.96% (33736/116482)
Branch Coverage: 24.82% (17319/69776)
Coverage Report: http://coverage.selectdb-in.cc/coverage/31c2eabf77ccf5e7c7229ae6a76a249ba35e368c_31c2eabf77ccf5e7c7229ae6a76a249ba35e368c/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976637273
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976588551
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow (#31532) [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31758:
URL: https://github.com/apache/doris/pull/31758#issuecomment-1976600915
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org