You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "dataroaring (via GitHub)" <gi...@apache.org> on 2024/06/08 11:46:49 UTC
[PR] [fix](pipeline) failure if memory is not enough [doris]
dataroaring opened a new pull request, #36071:
URL: https://github.com/apache/doris/pull/36071
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) failure if memory is not enough [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36071:
URL: https://github.com/apache/doris/pull/36071#issuecomment-2156002393
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website).
See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) failure if memory is not enough [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36071:
URL: https://github.com/apache/doris/pull/36071#issuecomment-2156058503
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) failure if memory is not enough [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36071:
URL: https://github.com/apache/doris/pull/36071#issuecomment-2156022362
<details>
<summary>ClickBench: <b>Total hot run time: 31.02 s</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4164c189e67a46c3db340f469db8754f830a07c3, data reload: false
query1 0.04 0.03 0.03
query2 0.08 0.04 0.04
query3 0.23 0.05 0.05
query4 1.68 0.06 0.07
query5 0.50 0.51 0.49
query6 1.12 0.72 0.73
query7 0.02 0.01 0.01
query8 0.06 0.04 0.04
query9 0.54 0.50 0.49
query10 0.54 0.53 0.54
query11 0.15 0.11 0.11
query12 0.15 0.13 0.13
query13 0.58 0.58 0.60
query14 0.77 0.77 0.76
query15 0.83 0.82 0.82
query16 0.36 0.35 0.37
query17 1.03 0.95 1.00
query18 0.22 0.25 0.24
query19 1.75 1.74 1.79
query20 0.01 0.01 0.01
query21 15.41 0.66 0.65
query22 4.81 6.20 2.40
query23 18.29 1.45 1.28
query24 2.12 0.23 0.22
query25 0.16 0.08 0.08
query26 0.26 0.18 0.17
query27 0.09 0.10 0.08
query28 13.28 1.02 0.98
query29 12.62 3.26 3.27
query30 0.26 0.08 0.07
query31 2.83 0.38 0.38
query32 3.27 0.47 0.46
query33 2.85 3.01 2.92
query34 17.07 4.42 4.40
query35 4.48 4.47 4.51
query36 0.66 0.46 0.46
query37 0.19 0.15 0.15
query38 0.15 0.14 0.14
query39 0.04 0.04 0.04
query40 0.19 0.14 0.16
query41 0.09 0.04 0.04
query42 0.06 0.04 0.04
query43 0.04 0.04 0.04
Total cold run time: 109.88 s
Total hot run time: 31.02 s
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) failure if memory is not enough [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring closed pull request #36071: [fix](pipeline) failure if memory is not enough
URL: https://github.com/apache/doris/pull/36071
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) failure if memory is not enough [doris]
Posted by "dataroaring (via GitHub)" <gi...@apache.org>.
dataroaring commented on PR #36071:
URL: https://github.com/apache/doris/pull/36071#issuecomment-2156002397
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) failure if memory is not enough [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36071:
URL: https://github.com/apache/doris/pull/36071#issuecomment-2156014501
TeamCity be ut coverage result:
Function Coverage: 36.44% (8975/24630)
Line Coverage: 27.98% (73351/262193)
Region Coverage: 27.37% (38023/138899)
Branch Coverage: 23.99% (19303/80456)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4164c189e67a46c3db340f469db8754f830a07c3_4164c189e67a46c3db340f469db8754f830a07c3/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) failure if memory is not enough [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on code in PR #36071:
URL: https://github.com/apache/doris/pull/36071#discussion_r1634041669
##########
be/src/vec/exprs/vexpr.cpp:
##########
@@ -343,7 +343,7 @@ Status VExpr::create_tree_from_thrift(const std::vector<TExprNode>& nodes, int*
// create root expr
int root_children = nodes[*node_idx].num_children;
VExprSPtr root;
- RETURN_IF_ERROR(create_expr(nodes[*node_idx], root));
Review Comment:
Catch exception in `FoldConstantExecutor::fold_constant_vexpr`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) failure if memory is not enough [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36071:
URL: https://github.com/apache/doris/pull/36071#issuecomment-2156020731
<details>
<summary>TPC-DS: <b>Total hot run time: 171414 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4164c189e67a46c3db340f469db8754f830a07c3, data reload: false
query1 920 379 374 374
query2 6457 2346 2223 2223
query3 6638 208 210 208
query4 20905 17529 17286 17286
query5 3574 472 466 466
query6 241 157 156 156
query7 4574 301 302 301
query8 328 290 280 280
query9 8469 2391 2385 2385
query10 563 309 285 285
query11 10535 9964 10047 9964
query12 117 93 84 84
query13 1654 373 379 373
query14 10380 7724 7664 7664
query15 246 192 184 184
query16 7931 269 271 269
query17 1900 551 532 532
query18 1941 282 280 280
query19 203 163 161 161
query20 89 83 82 82
query21 204 130 125 125
query22 4560 4153 4169 4153
query23 34065 33655 33676 33655
query24 10799 2874 2971 2874
query25 564 404 369 369
query26 707 158 153 153
query27 2289 325 320 320
query28 5988 2135 2101 2101
query29 868 649 610 610
query30 251 155 151 151
query31 1015 778 760 760
query32 93 51 58 51
query33 689 283 271 271
query34 898 480 482 480
query35 735 672 656 656
query36 1139 960 970 960
query37 155 80 69 69
query38 2912 2819 2818 2818
query39 928 843 836 836
query40 202 123 131 123
query41 56 51 52 51
query42 113 106 99 99
query43 588 565 542 542
query44 1169 721 738 721
query45 192 160 168 160
query46 1083 711 699 699
query47 1867 1754 1776 1754
query48 360 297 290 290
query49 833 395 416 395
query50 762 400 383 383
query51 6802 6714 6624 6624
query52 100 91 91 91
query53 352 288 291 288
query54 893 445 443 443
query55 73 74 74 74
query56 273 245 262 245
query57 1118 1064 1087 1064
query58 235 252 241 241
query59 3383 3178 3176 3176
query60 282 267 265 265
query61 97 82 84 82
query62 612 440 473 440
query63 315 289 297 289
query64 8459 2233 1734 1734
query65 3159 3071 3097 3071
query66 742 330 320 320
query67 15349 15082 14890 14890
query68 4593 544 553 544
query69 623 359 324 324
query70 1170 1135 1137 1135
query71 435 277 282 277
query72 7903 2716 2596 2596
query73 748 315 321 315
query74 5979 5508 5563 5508
query75 4083 2662 2659 2659
query76 2955 927 948 927
query77 646 295 294 294
query78 10557 9763 9823 9763
query79 2593 529 513 513
query80 933 469 511 469
query81 538 219 218 218
query82 736 101 100 100
query83 347 170 164 164
query84 256 85 84 84
query85 1717 310 261 261
query86 465 347 313 313
query87 3318 3099 3084 3084
query88 4297 2318 2312 2312
query89 476 375 382 375
query90 1848 191 196 191
query91 127 97 142 97
query92 61 48 49 48
query93 2035 511 510 510
query94 1151 186 183 183
query95 396 308 311 308
query96 600 262 277 262
query97 3178 3024 3079 3024
query98 222 200 199 199
query99 1166 862 834 834
Total cold run time: 272145 ms
Total hot run time: 171414 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) failure if memory is not enough [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #36071:
URL: https://github.com/apache/doris/pull/36071#issuecomment-2156004023
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) failure if memory is not enough [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #36071:
URL: https://github.com/apache/doris/pull/36071#issuecomment-2156016914
<details>
<summary>TPC-H: <b>Total hot run time: 39631 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4164c189e67a46c3db340f469db8754f830a07c3, data reload: false
------ Round 1 ----------------------------------
q1 17633 4291 4261 4261
q2 2026 193 186 186
q3 10466 1112 1125 1112
q4 10195 687 722 687
q5 7480 2775 2681 2681
q6 218 135 136 135
q7 964 614 596 596
q8 9214 2054 2073 2054
q9 9120 6524 6479 6479
q10 8973 3695 3696 3695
q11 460 242 239 239
q12 546 235 230 230
q13 18568 2991 2963 2963
q14 277 221 233 221
q15 515 467 481 467
q16 485 395 377 377
q17 957 680 709 680
q18 8030 7372 7291 7291
q19 4075 1430 1499 1430
q20 634 303 310 303
q21 5238 3241 3216 3216
q22 388 348 328 328
Total cold run time: 116462 ms
Total hot run time: 39631 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4346 4254 4345 4254
q2 364 266 263 263
q3 2961 2772 2942 2772
q4 1959 1729 1684 1684
q5 5589 5505 5531 5505
q6 224 138 130 130
q7 2180 1843 1837 1837
q8 3273 3402 3419 3402
q9 8659 8722 8786 8722
q10 4103 3867 3783 3783
q11 592 483 476 476
q12 789 610 616 610
q13 15889 3198 3168 3168
q14 294 269 289 269
q15 532 497 482 482
q16 500 461 426 426
q17 1810 1539 1488 1488
q18 8095 7904 7757 7757
q19 1825 1639 1674 1639
q20 2144 1837 1864 1837
q21 5049 4845 5049 4845
q22 615 580 558 558
Total cold run time: 71792 ms
Total hot run time: 55907 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [fix](pipeline) failure if memory is not enough [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on code in PR #36071:
URL: https://github.com/apache/doris/pull/36071#discussion_r1634039248
##########
be/src/pipeline/pipeline_task.cpp:
##########
@@ -302,7 +302,7 @@ Status PipelineTask::execute(bool* eos) {
}
// The status must be runnable
if (!_opened && !_fragment_context->is_canceled()) {
- RETURN_IF_ERROR(_open());
Review Comment:
Exception is already caught by caller
##########
be/src/pipeline/pipeline_fragment_context.cpp:
##########
@@ -337,13 +337,13 @@ Status PipelineFragmentContext::prepare(const doris::TPipelineFragmentParams& re
for (PipelinePtr& pipeline : _pipelines) {
SCOPED_TIMER(_prepare_all_pipelines_timer);
pipeline->children().clear();
- RETURN_IF_ERROR(pipeline->prepare(_runtime_state.get()));
+ RETURN_IF_ERROR_OR_CATCH_EXCEPTION(pipeline->prepare(_runtime_state.get()));
Review Comment:
Maybe we should catch exception by `prepare`'s caller
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org