You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "liaoxin01 (via GitHub)" <gi...@apache.org> on 2024/02/06 10:31:47 UTC
[PR] [Fix](load) fix load channel leak when load exception occurs (#30915) [doris]
liaoxin01 opened a new pull request, #30920:
URL: https://github.com/apache/doris/pull/30920
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](load) fix load channel leak when load exception occurs (#30915) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30920:
URL: https://github.com/apache/doris/pull/30920#issuecomment-1929219386
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](load) fix load channel leak when load exception occurs (#30915) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30920:
URL: https://github.com/apache/doris/pull/30920#issuecomment-1929316570
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
```
Load test result on commit 536a9ee2e609c979b58656769551d6d36e78a03e with default session variables
Stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select: 21.5 seconds inserted 10000000 Rows, about 465K ops/s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](load) fix load channel leak when load exception occurs (#30915) [doris]
Posted by "xiaokang (via GitHub)" <gi...@apache.org>.
xiaokang merged PR #30920:
URL: https://github.com/apache/doris/pull/30920
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](load) fix load channel leak when load exception occurs (#30915) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30920:
URL: https://github.com/apache/doris/pull/30920#issuecomment-1929264016
TeamCity be ut coverage result:
Function Coverage: 37.83% (8035/21238)
Line Coverage: 29.50% (65503/222013)
Region Coverage: 28.94% (33651/116296)
Branch Coverage: 24.79% (17268/69644)
Coverage Report: http://coverage.selectdb-in.cc/coverage/536a9ee2e609c979b58656769551d6d36e78a03e_536a9ee2e609c979b58656769551d6d36e78a03e/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](load) fix load channel leak when load exception occurs (#30915) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30920:
URL: https://github.com/apache/doris/pull/30920#issuecomment-1929300852
<details>
<summary>TPC-DS: <b>Total hot run time: 237243 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 536a9ee2e609c979b58656769551d6d36e78a03e, data reload: false
query1 1116 394 389 389
query2 7661 2285 2040 2040
query3 6915 210 206 206
query4 21946 18814 18834 18814
query5 20047 6353 6332 6332
query6 729 217 223 217
query7 5437 300 307 300
query8 250 281 255 255
query9 3115 2644 2620 2620
query10 426 280 285 280
query11 11367 10673 10696 10673
query12 120 74 74 74
query13 5627 637 599 599
query14 18801 13110 13355 13110
query15 384 233 228 228
query16 6417 254 262 254
query17 3089 867 871 867
query18 2261 418 396 396
query19 202 141 141 141
query20 78 73 77 73
query21 181 92 97 92
query22 5234 5147 5105 5105
query23 32502 31942 31914 31914
query24 6744 6480 6562 6480
query25 512 423 396 396
query26 477 162 154 154
query27 1703 290 282 282
query28 6133 2237 2198 2198
query29 2937 2799 2707 2707
query30 249 163 161 161
query31 914 724 747 724
query32 56 56 57 56
query33 387 252 240 240
query34 873 459 473 459
query35 1112 932 898 898
query36 1788 1476 1510 1476
query37 91 57 62 57
query38 3038 2873 2938 2873
query39 1368 1314 1326 1314
query40 198 95 90 90
query41 35 33 31 31
query42 81 84 87 84
query43 631 642 545 545
query44 1122 711 702 702
query45 238 226 225 225
query46 1225 973 966 966
query47 2050 1841 1843 1841
query48 986 676 649 649
query49 611 372 351 351
query50 851 588 595 588
query51 5592 5438 5458 5438
query52 81 79 74 74
query53 438 313 309 309
query54 2451 2245 2242 2242
query55 76 77 77 77
query56 215 196 190 190
query57 1137 1090 1136 1090
query58 205 199 210 199
query59 3560 3167 3139 3139
query60 197 185 197 185
query61 82 80 77 77
query62 827 503 505 503
query63 471 327 331 327
query64 1916 1254 1270 1254
query65 3686 3615 3658 3615
query66 808 367 361 361
query67 15750 15449 16071 15449
query68 8926 648 657 648
query69 558 331 348 331
query70 2127 1801 1698 1698
query71 411 304 303 303
query72 4536 3318 3283 3283
query73 721 306 316 306
query74 6310 5882 5816 5816
query75 4900 3419 3454 3419
query76 5616 1185 1215 1185
query77 929 260 245 245
query78 31970 52446 46404 46404
query79 14633 616 643 616
query80 4491 371 378 371
query81 569 227 232 227
query82 567 100 94 94
query83 424 130 140 130
query84 257 68 70 68
query85 2404 284 270 270
query86 444 376 388 376
query87 3267 2976 2972 2972
query88 6240 2291 2274 2274
query89 375 280 271 271
query90 2497 211 209 209
query91 163 120 132 120
query92 58 50 47 47
query93 3907 591 592 591
query94 1730 204 196 196
query95 1117 1047 1052 1047
query96 648 339 330 330
query97 6451 6297 6277 6277
query98 194 177 170 170
query99 3854 896 894 894
Total cold run time: 349101 ms
Total hot run time: 237243 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](load) fix load channel leak when load exception occurs (#30915) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30920:
URL: https://github.com/apache/doris/pull/30920#issuecomment-1929275469
<details>
<summary>TPC-H: <b>Total hot run time: 49706 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 536a9ee2e609c979b58656769551d6d36e78a03e, data reload: false
------ Round 1 ----------------------------------
q1 17828 4359 4370 4359
q2 2024 151 136 136
q3 10492 1843 1916 1843
q4 10327 1233 1302 1233
q5 8409 3949 3921 3921
q6 223 124 122 122
q7 2019 1583 1590 1583
q8 9546 2700 2706 2700
q9 13172 10781 10386 10386
q10 8647 3507 3481 3481
q11 405 234 247 234
q12 475 293 307 293
q13 18356 3955 4010 3955
q14 351 313 335 313
q15 509 446 454 446
q16 707 590 587 587
q17 1121 981 934 934
q18 7294 6895 6973 6895
q19 1692 1557 1526 1526
q20 508 315 289 289
q21 4440 4169 4080 4080
q22 502 390 403 390
Total cold run time: 119047 ms
Total hot run time: 49706 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4340 4325 4270 4270
q2 317 222 219 219
q3 4190 4158 4138 4138
q4 2747 2739 2753 2739
q5 7225 7157 7155 7155
q6 235 119 119 119
q7 3238 2888 2834 2834
q8 4323 4476 4459 4459
q9 17106 16969 16923 16923
q10 4225 4278 4279 4278
q11 789 689 704 689
q12 1024 861 855 855
q13 7030 3728 3747 3728
q14 456 406 422 406
q15 486 458 454 454
q16 764 710 700 700
q17 3819 3811 3857 3811
q18 8755 8761 8773 8761
q19 1714 1635 1693 1635
q20 2424 2151 2154 2151
q21 8619 8527 8581 8527
q22 1086 957 991 957
Total cold run time: 84912 ms
Total hot run time: 79808 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](load) fix load channel leak when load exception occurs (#30915) [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30920:
URL: https://github.com/apache/doris/pull/30920#issuecomment-1929231615
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](load) fix load channel leak when load exception occurs (#30915) [doris]
Posted by "liaoxin01 (via GitHub)" <gi...@apache.org>.
liaoxin01 commented on PR #30920:
URL: https://github.com/apache/doris/pull/30920#issuecomment-1929220065
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](load) fix load channel leak when load exception occurs (#30915) [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #30920:
URL: https://github.com/apache/doris/pull/30920#issuecomment-1929311581
<details>
<summary>ClickBench: <b>Total hot run time: 30.95 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 536a9ee2e609c979b58656769551d6d36e78a03e, data reload: false
query1 0.02 0.02 0.02
query2 0.06 0.03 0.02
query3 0.24 0.05 0.04
query4 1.81 0.07 0.07
query5 0.52 0.52 0.53
query6 1.28 0.62 0.67
query7 0.02 0.02 0.01
query8 0.04 0.02 0.02
query9 0.53 0.48 0.48
query10 0.54 0.55 0.54
query11 0.11 0.09 0.08
query12 0.11 0.09 0.09
query13 0.61 0.61 0.60
query14 0.78 0.77 0.81
query15 0.78 0.77 0.76
query16 0.40 0.37 0.37
query17 1.01 1.04 1.02
query18 0.23 0.26 0.24
query19 1.92 1.80 1.90
query20 0.01 0.01 0.00
query21 15.49 0.57 0.56
query22 1.93 2.12 1.38
query23 17.46 0.97 1.10
query24 6.68 1.34 1.37
query25 1.45 0.12 0.11
query26 0.31 0.15 0.14
query27 0.11 0.11 0.10
query28 6.65 0.74 0.71
query29 12.72 2.25 2.27
query30 0.57 0.49 0.47
query31 2.83 0.39 0.37
query32 3.41 0.50 0.50
query33 3.11 3.09 3.09
query34 15.25 4.80 4.80
query35 4.84 4.82 4.85
query36 1.06 1.02 1.03
query37 0.06 0.05 0.05
query38 0.04 0.02 0.02
query39 0.02 0.01 0.02
query40 0.15 0.14 0.14
query41 0.07 0.01 0.01
query42 0.02 0.02 0.01
query43 0.02 0.01 0.02
Total cold run time: 105.27 s
Total hot run time: 30.95 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org