You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "kaijchen (via GitHub)" <gi...@apache.org> on 2024/03/04 17:09:38 UTC
[PR] [fix](move-memtable) free resources when fail to open stream [doris]
kaijchen opened a new pull request, #31762:
URL: https://github.com/apache/doris/pull/31762
## Proposed changes
Free resources when error occurs in load stream open.
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](move-memtable) free resources when fail to open stream [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31762:
URL: https://github.com/apache/doris/pull/31762#issuecomment-1978366886
TeamCity be ut coverage result:
Function Coverage: 35.43% (8557/24152)
Line Coverage: 27.19% (69436/255406)
Region Coverage: 26.41% (36032/136426)
Branch Coverage: 23.25% (18412/79202)
Coverage Report: http://coverage.selectdb-in.cc/coverage/146dfa5fd082d15296da623432cf71954b6f7aa9_146dfa5fd082d15296da623432cf71954b6f7aa9/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](move-memtable) free resources when fail to open stream [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 merged PR #31762:
URL: https://github.com/apache/doris/pull/31762
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](move-memtable) free resources when fail to open stream [doris]
Posted by "kaijchen (via GitHub)" <gi...@apache.org>.
kaijchen commented on PR #31762:
URL: https://github.com/apache/doris/pull/31762#issuecomment-1977072041
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](move-memtable) free resources when fail to open stream [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31762:
URL: https://github.com/apache/doris/pull/31762#issuecomment-1977152332
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 146dfa5fd082d15296da623432cf71954b6f7aa9 with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select: 18.1 seconds inserted 10000000 Rows, about 552K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](move-memtable) free resources when fail to open stream [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31762:
URL: https://github.com/apache/doris/pull/31762#issuecomment-1977876366
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](move-memtable) free resources when fail to open stream [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31762:
URL: https://github.com/apache/doris/pull/31762#issuecomment-1977138540
<details>
<summary>TPC-DS: <b>Total hot run time: 177845 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 146dfa5fd082d15296da623432cf71954b6f7aa9, data reload: false
query1 940 370 345 345
query2 7351 2191 2049 2049
query3 6698 221 220 220
query4 25830 20998 20979 20979
query5 4307 2703 476 476
query6 279 184 174 174
query7 4635 309 307 307
query8 236 179 179 179
query9 8487 2169 2172 2169
query10 450 236 246 236
query11 15028 14371 14445 14371
query12 137 90 85 85
query13 1632 424 432 424
query14 9085 7013 7014 7013
query15 246 198 192 192
query16 7239 271 283 271
query17 962 617 577 577
query18 1927 294 286 286
query19 203 160 160 160
query20 97 85 93 85
query21 202 132 127 127
query22 4688 4452 4411 4411
query23 31517 30972 30457 30457
query24 12244 3027 3043 3027
query25 704 399 398 398
query26 1931 161 173 161
query27 3003 375 375 375
query28 6597 1827 1799 1799
query29 1328 624 608 608
query30 312 151 147 147
query31 940 722 726 722
query32 104 65 61 61
query33 763 272 265 265
query34 1008 467 502 467
query35 927 824 792 792
query36 940 826 814 814
query37 288 70 70 70
query38 3282 3155 3140 3140
query39 1427 1355 1363 1355
query40 298 130 142 130
query41 61 57 57 57
query42 112 112 109 109
query43 452 403 395 395
query44 1078 732 711 711
query45 204 199 193 193
query46 1070 798 760 760
query47 1622 1556 1555 1555
query48 413 351 344 344
query49 1183 329 328 328
query50 785 380 375 375
query51 6668 6536 6573 6536
query52 105 94 100 94
query53 355 291 286 286
query54 339 229 241 229
query55 93 85 80 80
query56 231 238 227 227
query57 1101 990 1004 990
query58 249 205 216 205
query59 2587 2361 2370 2361
query60 255 243 238 238
query61 95 92 91 91
query62 658 418 396 396
query63 309 274 281 274
query64 6253 3283 3134 3134
query65 3018 3063 2999 2999
query66 1475 332 323 323
query67 14875 14903 14394 14394
query68 7759 588 616 588
query69 651 390 407 390
query70 1239 1146 1119 1119
query71 612 300 273 273
query72 8660 2813 2608 2608
query73 801 327 341 327
query74 7198 6850 6987 6850
query75 4648 2673 2691 2673
query76 5836 1113 1194 1113
query77 850 263 262 262
query78 10324 9670 9498 9498
query79 8479 527 527 527
query80 1125 426 417 417
query81 562 221 219 219
query82 744 97 95 95
query83 206 152 155 152
query84 296 79 87 79
query85 1645 350 331 331
query86 514 277 304 277
query87 3417 3148 3186 3148
query88 4137 2401 2375 2375
query89 576 380 378 378
query90 2113 180 179 179
query91 165 129 130 129
query92 67 55 50 50
query93 6941 534 508 508
query94 1331 188 193 188
query95 447 349 362 349
query96 599 264 265 264
query97 3993 3878 3869 3869
query98 228 221 219 219
query99 1203 785 783 783
Total cold run time: 301284 ms
Total hot run time: 177845 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](move-memtable) free resources when fail to open stream [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31762:
URL: https://github.com/apache/doris/pull/31762#issuecomment-1977118490
<details>
<summary>TPC-H: <b>Total hot run time: 38127 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 146dfa5fd082d15296da623432cf71954b6f7aa9, data reload: false
------ Round 1 ----------------------------------
q1 18534 4659 4181 4181
q2 2263 172 142 142
q3 11590 963 975 963
q4 4970 968 961 961
q5 8258 2804 2979 2804
q6 192 122 125 122
q7 1304 839 811 811
q8 9496 2094 2078 2078
q9 7380 6494 6480 6480
q10 8239 2536 2584 2536
q11 425 221 222 221
q12 802 322 317 317
q13 17972 2956 2969 2956
q14 288 255 248 248
q15 492 448 442 442
q16 490 385 401 385
q17 955 899 857 857
q18 6784 5988 5866 5866
q19 1591 1523 1532 1523
q20 559 289 284 284
q21 7582 3710 3667 3667
q22 791 290 283 283
Total cold run time: 110957 ms
Total hot run time: 38127 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4010 3988 4035 3988
q2 317 219 221 219
q3 2962 2937 2888 2888
q4 1856 1808 1822 1808
q5 5208 5250 5202 5202
q6 201 114 114 114
q7 2310 1858 1845 1845
q8 3190 3252 3277 3252
q9 8523 8488 8527 8488
q10 6099 3728 3737 3728
q11 526 436 447 436
q12 691 545 529 529
q13 13558 2806 2791 2791
q14 272 257 267 257
q15 484 437 434 434
q16 455 420 416 416
q17 1688 1675 1668 1668
q18 7756 7502 7275 7275
q19 1611 1608 1615 1608
q20 1925 1722 1764 1722
q21 5004 4816 4795 4795
q22 483 484 464 464
Total cold run time: 69129 ms
Total hot run time: 53927 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](move-memtable) free resources when fail to open stream [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31762:
URL: https://github.com/apache/doris/pull/31762#issuecomment-1977088587
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](move-memtable) free resources when fail to open stream [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31762:
URL: https://github.com/apache/doris/pull/31762#issuecomment-1977876285
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](move-memtable) free resources when fail to open stream [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31762:
URL: https://github.com/apache/doris/pull/31762#issuecomment-1977147346
<details>
<summary>ClickBench: <b>Total hot run time: 30.93 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 146dfa5fd082d15296da623432cf71954b6f7aa9, data reload: false
query1 0.04 0.03 0.03
query2 0.07 0.03 0.03
query3 0.23 0.06 0.06
query4 1.64 0.10 0.09
query5 0.50 0.49 0.50
query6 1.23 0.66 0.69
query7 0.02 0.02 0.01
query8 0.05 0.03 0.03
query9 0.56 0.52 0.51
query10 0.57 0.57 0.57
query11 0.13 0.10 0.10
query12 0.12 0.10 0.10
query13 0.58 0.57 0.56
query14 0.77 0.74 0.75
query15 0.83 0.81 0.81
query16 0.37 0.37 0.37
query17 0.95 1.01 0.96
query18 0.25 0.25 0.25
query19 1.82 1.71 1.71
query20 0.02 0.01 0.02
query21 15.43 0.61 0.59
query22 3.37 4.49 2.56
query23 17.43 1.13 0.97
query24 2.30 0.27 0.21
query25 0.16 0.07 0.04
query26 0.18 0.13 0.14
query27 0.03 0.03 0.03
query28 12.47 0.84 0.87
query29 12.60 3.51 3.34
query30 0.66 0.57 0.55
query31 2.80 0.33 0.34
query32 3.36 0.45 0.43
query33 2.97 2.91 2.91
query34 15.51 4.32 4.31
query35 4.33 4.30 4.30
query36 1.09 1.03 1.01
query37 0.07 0.05 0.05
query38 0.05 0.03 0.03
query39 0.02 0.02 0.02
query40 0.18 0.13 0.14
query41 0.08 0.03 0.02
query42 0.03 0.02 0.01
query43 0.03 0.03 0.03
Total cold run time: 105.9 s
Total hot run time: 30.93 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](move-memtable) free resources when fail to open stream [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31762:
URL: https://github.com/apache/doris/pull/31762#issuecomment-1977070698
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org