You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "liaoxin01 (via GitHub)" <gi...@apache.org> on 2024/02/28 09:04:00 UTC
[PR] [fix](load) Fix using uint32 for tablet id overflow [doris]
liaoxin01 opened a new pull request, #31532:
URL: https://github.com/apache/doris/pull/31532
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
[INTERNAL_ERROR][INTERNAL_ERROR]close wait failed coz rpc error. VNodeChannel[4310428608-10003], load_id=c478455147aeb7f-847eb9b9965435a7, txn_id=42764708, node=10.86.1.83:8060, add batch req success but status isn't ok, err: [INTERNAL_ERROR]PStatus: (10.86.1.83)[INTERNAL_ERROR]unknown tablet to append data, tablet=15461331
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968747262
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968527099
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968578306
<details>
<summary>TPC-DS: <b>Total hot run time: 169506 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c68475eebda09b3c32794775d1cc9036db9e76eb, data reload: false
query1 941 337 335 335
query2 6521 1746 1658 1658
query3 6720 210 210 210
query4 23408 20590 20626 20590
query5 4223 371 373 371
query6 258 170 160 160
query7 4618 301 303 301
query8 274 194 196 194
query9 8412 2260 2243 2243
query10 417 211 218 211
query11 14675 14164 14180 14164
query12 132 83 82 82
query13 1653 426 437 426
query14 8277 6637 6532 6532
query15 197 178 186 178
query16 7089 262 248 248
query17 1043 532 502 502
query18 1894 271 265 265
query19 193 146 144 144
query20 87 81 86 81
query21 198 132 117 117
query22 4789 4668 4547 4547
query23 30571 29821 29951 29821
query24 11824 3131 3119 3119
query25 654 356 345 345
query26 1782 152 158 152
query27 3018 317 324 317
query28 7008 1812 1804 1804
query29 1159 563 566 563
query30 284 132 139 132
query31 869 685 706 685
query32 94 58 56 56
query33 725 220 219 219
query34 1046 482 494 482
query35 836 732 748 732
query36 926 881 849 849
query37 134 61 61 61
query38 3152 3007 2960 2960
query39 1318 1290 1261 1261
query40 280 101 104 101
query41 38 36 37 36
query42 103 98 101 98
query43 453 425 427 425
query44 1055 708 728 708
query45 197 183 172 172
query46 1036 786 792 786
query47 1543 1453 1495 1453
query48 434 346 343 343
query49 1123 285 296 285
query50 783 383 388 383
query51 4377 4341 4260 4260
query52 97 93 89 89
query53 330 266 264 264
query54 299 226 224 224
query55 87 80 86 80
query56 224 197 194 194
query57 959 883 872 872
query58 217 201 186 186
query59 2260 2085 2101 2085
query60 242 215 214 214
query61 90 84 82 82
query62 607 367 360 360
query63 280 263 251 251
query64 6578 3047 3109 3047
query65 3221 3209 3224 3209
query66 1443 327 310 310
query67 14490 14371 14130 14130
query68 5004 558 572 558
query69 535 364 378 364
query70 1276 1225 1206 1206
query71 411 255 268 255
query72 6230 2785 2656 2656
query73 707 320 334 320
query74 6304 5960 5959 5959
query75 3075 2455 2410 2410
query76 2789 1025 1141 1025
query77 352 227 223 223
query78 9068 8496 8492 8492
query79 1125 531 523 523
query80 813 357 341 341
query81 465 194 193 193
query82 1305 84 84 84
query83 205 120 121 120
query84 278 80 82 80
query85 1189 340 330 330
query86 337 297 303 297
query87 3349 3109 3091 3091
query88 2793 2332 2323 2323
query89 388 335 324 324
query90 2037 164 169 164
query91 151 119 122 119
query92 56 47 51 47
query93 1031 521 510 510
query94 1113 180 177 177
query95 437 332 348 332
query96 573 270 265 265
query97 4248 4160 4151 4151
query98 218 202 186 186
query99 1043 617 691 617
Total cold run time: 263170 ms
Total hot run time: 169506 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968587860
<details>
<summary>ClickBench: <b>Total hot run time: 32.1 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c68475eebda09b3c32794775d1cc9036db9e76eb, data reload: false
query1 0.04 0.03 0.03
query2 0.05 0.02 0.02
query3 0.23 0.06 0.06
query4 1.68 0.10 0.10
query5 0.52 0.52 0.52
query6 1.25 0.66 0.69
query7 0.01 0.01 0.02
query8 0.03 0.02 0.02
query9 0.55 0.53 0.53
query10 0.56 0.56 0.58
query11 0.13 0.10 0.09
query12 0.12 0.10 0.10
query13 0.63 0.61 0.61
query14 0.79 0.79 0.82
query15 0.84 0.82 0.80
query16 0.37 0.37 0.39
query17 0.99 1.00 0.96
query18 0.27 0.25 0.25
query19 1.77 1.74 1.73
query20 0.01 0.01 0.02
query21 15.41 0.62 0.62
query22 2.58 3.35 2.45
query23 17.64 1.06 1.04
query24 2.16 0.24 0.12
query25 0.23 0.12 0.05
query26 0.16 0.15 0.15
query27 0.05 0.04 0.04
query28 12.81 0.83 0.83
query29 12.60 3.24 3.31
query30 0.57 0.54 0.54
query31 2.79 0.35 0.35
query32 3.36 0.48 0.48
query33 3.16 3.22 3.21
query34 15.36 4.73 4.73
query35 4.74 4.71 4.71
query36 1.13 1.07 1.07
query37 0.06 0.04 0.04
query38 0.04 0.03 0.02
query39 0.02 0.01 0.01
query40 0.18 0.13 0.13
query41 0.07 0.02 0.02
query42 0.02 0.02 0.01
query43 0.03 0.02 0.02
Total cold run time: 106.01 s
Total hot run time: 32.1 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring merged PR #31532:
URL: https://github.com/apache/doris/pull/31532
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968559373
<details>
<summary>TPC-H: <b>Total hot run time: 38019 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c68475eebda09b3c32794775d1cc9036db9e76eb, data reload: false
------ Round 1 ----------------------------------
q1 18133 4183 4120 4120
q2 2603 146 140 140
q3 11000 983 966 966
q4 5041 971 998 971
q5 7973 3000 3075 3000
q6 185 126 122 122
q7 1294 788 777 777
q8 9638 1968 2072 1968
q9 7223 6267 6263 6263
q10 8178 2481 2489 2481
q11 421 202 216 202
q12 798 318 309 309
q13 17973 3263 3207 3207
q14 285 262 251 251
q15 530 499 487 487
q16 470 418 412 412
q17 941 892 886 886
q18 6577 6126 5811 5811
q19 1569 1510 1511 1510
q20 549 287 279 279
q21 6781 3583 3577 3577
q22 801 283 280 280
Total cold run time: 108963 ms
Total hot run time: 38019 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4008 3983 4007 3983
q2 300 215 211 211
q3 3001 2927 2949 2927
q4 1868 1813 1805 1805
q5 5084 5126 5125 5125
q6 200 114 114 114
q7 2130 1699 1717 1699
q8 3105 3182 3197 3182
q9 8258 8237 8267 8237
q10 6213 3624 3628 3624
q11 517 433 437 433
q12 713 517 521 517
q13 3826 3069 3061 3061
q14 276 242 256 242
q15 532 492 487 487
q16 502 466 476 466
q17 1700 1678 1677 1677
q18 7839 7535 7391 7391
q19 1630 1643 1621 1621
q20 2094 1910 1907 1907
q21 4757 4640 4593 4593
q22 513 455 462 455
Total cold run time: 59066 ms
Total hot run time: 53757 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968543992
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968582939
TeamCity be ut coverage result:
Function Coverage: 35.69% (8548/23952)
Line Coverage: 27.49% (69386/252411)
Region Coverage: 26.65% (35974/135007)
Branch Coverage: 23.46% (18389/78394)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c68475eebda09b3c32794775d1cc9036db9e76eb_c68475eebda09b3c32794775d1cc9036db9e76eb/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968527487
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968593620
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit c68475eebda09b3c32794775d1cc9036db9e76eb with default session variables
Stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc: 60 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 16.8 seconds inserted 10000000 Rows, about 595K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](load) Fix using uint32 for tablet id overflow [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31532:
URL: https://github.com/apache/doris/pull/31532#issuecomment-1968747199
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org