You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "yiguolei (via GitHub)" <gi...@apache.org> on 2024/06/01 15:06:18 UTC
[PR] [enhancement](oom) add exception in olap data convertor when memory is not enough to prevent oom [doris]
yiguolei opened a new pull request, #35761:
URL: https://github.com/apache/doris/pull/35761
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](oom) add exception in olap data convertor when memory is not enough to prevent oom [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35761:
URL: https://github.com/apache/doris/pull/35761#issuecomment-2143506165
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](oom) add exception in olap data convertor when memory is not enough to prevent oom [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #35761:
URL: https://github.com/apache/doris/pull/35761#issuecomment-2143481407
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](oom) add exception in olap data convertor when memory is not enough to prevent oom [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35761:
URL: https://github.com/apache/doris/pull/35761#issuecomment-2143672085
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](oom) add exception in olap data convertor when memory is not enough to prevent oom [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #35761:
URL: https://github.com/apache/doris/pull/35761#issuecomment-2143504637
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](oom) add exception in olap data convertor when memory is not enough to prevent oom [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei merged PR #35761:
URL: https://github.com/apache/doris/pull/35761
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](oom) add exception in olap data convertor when memory is not enough to prevent oom [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #35761:
URL: https://github.com/apache/doris/pull/35761#issuecomment-2143479980
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](oom) add exception in olap data convertor when memory is not enough to prevent oom [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35761:
URL: https://github.com/apache/doris/pull/35761#issuecomment-2143482418
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](oom) add exception in olap data convertor when memory is not enough to prevent oom [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #35761:
URL: https://github.com/apache/doris/pull/35761#issuecomment-2143525428
<details>
<summary>TPC-H: <b>Total hot run time: 40053 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 18911c11ad13005076a2731864a6632a728609b7, data reload: false
------ Round 1 ----------------------------------
q1 17673 4397 4397 4397
q2 2029 194 199 194
q3 10487 1287 1183 1183
q4 10140 904 834 834
q5 7470 2715 2655 2655
q6 228 132 134 132
q7 961 595 600 595
q8 9214 2100 2126 2100
q9 9189 6526 6471 6471
q10 9005 3684 3736 3684
q11 465 244 240 240
q12 424 230 218 218
q13 17760 2977 2979 2977
q14 269 220 231 220
q15 515 478 467 467
q16 522 383 374 374
q17 992 695 661 661
q18 8166 7575 7369 7369
q19 5522 1508 1436 1436
q20 654 315 310 310
q21 4942 3206 3307 3206
q22 384 330 340 330
Total cold run time: 117011 ms
Total hot run time: 40053 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4355 4265 4217 4217
q2 387 264 270 264
q3 3021 2796 2735 2735
q4 1889 1616 1636 1616
q5 5273 5330 5319 5319
q6 210 125 124 124
q7 2161 1737 1753 1737
q8 3220 3339 3315 3315
q9 8354 8334 8334 8334
q10 3864 3616 3726 3616
q11 588 483 492 483
q12 751 637 609 609
q13 16264 2960 2978 2960
q14 292 267 277 267
q15 516 498 479 479
q16 469 410 420 410
q17 1784 1499 1480 1480
q18 7615 7595 7422 7422
q19 1707 1639 1640 1639
q20 1976 1785 1789 1785
q21 4926 4832 4883 4832
q22 627 564 545 545
Total cold run time: 70249 ms
Total hot run time: 54188 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](oom) add exception in olap data convertor when memory is not enough to prevent oom [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #35761:
URL: https://github.com/apache/doris/pull/35761#issuecomment-2143529001
<details>
<summary>TPC-DS: <b>Total hot run time: 172623 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 18911c11ad13005076a2731864a6632a728609b7, data reload: false
query1 928 389 377 377
query2 6458 2518 2342 2342
query3 6650 201 201 201
query4 19747 17295 17227 17227
query5 4148 445 453 445
query6 258 158 161 158
query7 4597 309 300 300
query8 310 290 285 285
query9 8416 2368 2344 2344
query10 446 292 275 275
query11 10549 10180 9985 9985
query12 128 87 86 86
query13 1630 368 384 368
query14 10304 7553 6983 6983
query15 268 188 188 188
query16 8006 257 256 256
query17 1850 523 525 523
query18 2056 269 269 269
query19 194 147 157 147
query20 92 94 86 86
query21 206 133 124 124
query22 4182 4023 4010 4010
query23 33668 33164 32966 32966
query24 10409 2880 2914 2880
query25 590 349 357 349
query26 703 155 156 155
query27 2265 323 329 323
query28 5903 2070 2074 2070
query29 883 604 601 601
query30 307 148 153 148
query31 953 733 727 727
query32 89 51 52 51
query33 662 279 270 270
query34 872 489 476 476
query35 751 612 601 601
query36 1071 925 913 913
query37 132 72 70 70
query38 2864 2730 2707 2707
query39 866 790 787 787
query40 215 124 123 123
query41 58 52 51 51
query42 119 94 97 94
query43 577 557 536 536
query44 1072 745 753 745
query45 202 174 171 171
query46 1067 699 717 699
query47 1834 1774 1787 1774
query48 382 304 308 304
query49 1049 429 441 429
query50 776 400 386 386
query51 6790 6751 6877 6751
query52 104 90 91 90
query53 350 290 286 286
query54 1001 446 439 439
query55 74 71 70 70
query56 293 254 247 247
query57 1109 1079 1071 1071
query58 257 236 235 235
query59 3454 3221 3042 3042
query60 297 265 264 264
query61 89 90 90 90
query62 659 483 433 433
query63 316 285 284 284
query64 8626 2202 1732 1732
query65 3165 3125 3185 3125
query66 825 341 337 337
query67 15351 14926 14978 14926
query68 4587 548 549 548
query69 472 304 311 304
query70 1113 1084 1132 1084
query71 352 283 282 282
query72 7113 5521 6139 5521
query73 739 327 326 326
query74 5957 5481 5501 5481
query75 3325 2681 2674 2674
query76 2390 855 902 855
query77 489 302 292 292
query78 10441 9769 9668 9668
query79 2971 520 514 514
query80 2267 482 504 482
query81 597 220 221 220
query82 903 103 103 103
query83 288 174 172 172
query84 276 96 89 89
query85 1945 328 317 317
query86 488 318 316 316
query87 3270 3059 3080 3059
query88 4209 2451 2458 2451
query89 482 394 390 390
query90 1792 187 187 187
query91 136 110 107 107
query92 60 49 52 49
query93 3446 527 509 509
query94 1232 195 191 191
query95 397 317 324 317
query96 600 276 271 271
query97 3204 2980 3051 2980
query98 251 220 215 215
query99 1285 887 842 842
Total cold run time: 271599 ms
Total hot run time: 172623 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](oom) add exception in olap data convertor when memory is not enough to prevent oom [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35761:
URL: https://github.com/apache/doris/pull/35761#issuecomment-2143672081
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](oom) add exception in olap data convertor when memory is not enough to prevent oom [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on PR #35761:
URL: https://github.com/apache/doris/pull/35761#issuecomment-2143480056
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](oom) add exception in olap data convertor when memory is not enough to prevent oom [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35761:
URL: https://github.com/apache/doris/pull/35761#issuecomment-2143482069
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](oom) add exception in olap data convertor when memory is not enough to prevent oom [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #35761:
URL: https://github.com/apache/doris/pull/35761#issuecomment-2143531589
TeamCity be ut coverage result:
Function Coverage: 36.27% (9235/25460)
Line Coverage: 27.61% (75714/274204)
Region Coverage: 26.83% (39206/146148)
Branch Coverage: 23.55% (19860/84344)
Coverage Report: http://coverage.selectdb-in.cc/coverage/18911c11ad13005076a2731864a6632a728609b7_18911c11ad13005076a2731864a6632a728609b7/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [enhancement](oom) add exception in olap data convertor when memory is not enough to prevent oom [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #35761:
URL: https://github.com/apache/doris/pull/35761#issuecomment-2143530525
<details>
<summary>ClickBench: <b>Total hot run time: 30.63 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 18911c11ad13005076a2731864a6632a728609b7, data reload: false
query1 0.04 0.04 0.04
query2 0.07 0.04 0.04
query3 0.23 0.06 0.06
query4 1.66 0.10 0.11
query5 0.51 0.52 0.50
query6 1.12 0.75 0.73
query7 0.02 0.01 0.01
query8 0.05 0.04 0.04
query9 0.54 0.49 0.49
query10 0.54 0.54 0.55
query11 0.15 0.10 0.11
query12 0.14 0.11 0.11
query13 0.59 0.59 0.60
query14 0.75 0.77 0.80
query15 0.81 0.81 0.82
query16 0.36 0.35 0.37
query17 0.94 1.04 0.97
query18 0.22 0.24 0.23
query19 1.86 1.72 1.74
query20 0.01 0.01 0.01
query21 15.49 0.70 0.70
query22 4.73 7.14 1.86
query23 18.32 1.32 1.24
query24 1.93 0.22 0.23
query25 0.14 0.09 0.09
query26 0.26 0.18 0.17
query27 0.08 0.07 0.08
query28 13.26 1.02 0.99
query29 12.84 3.39 3.37
query30 0.23 0.06 0.07
query31 2.86 0.38 0.38
query32 3.29 0.47 0.47
query33 2.87 2.92 2.95
query34 17.12 4.38 4.43
query35 4.52 4.48 4.60
query36 0.68 0.48 0.45
query37 0.18 0.15 0.15
query38 0.16 0.14 0.15
query39 0.05 0.03 0.04
query40 0.17 0.15 0.14
query41 0.09 0.05 0.04
query42 0.05 0.04 0.04
query43 0.05 0.03 0.04
Total cold run time: 109.98 s
Total hot run time: 30.63 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org