You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "Gabriel39 (via GitHub)" <gi...@apache.org> on 2024/03/01 03:02:16 UTC
[PR] [pipelineX](bug) cancel pipeline tasks if timeout [doris]
Gabriel39 opened a new pull request, #31635:
URL: https://github.com/apache/doris/pull/31635
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) cancel pipeline tasks if timeout [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31635:
URL: https://github.com/apache/doris/pull/31635#issuecomment-1972385395
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) cancel pipeline tasks if timeout [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31635:
URL: https://github.com/apache/doris/pull/31635#issuecomment-1972488080
TeamCity be ut coverage result:
Function Coverage: 35.67% (8550/23969)
Line Coverage: 27.47% (69388/252636)
Region Coverage: 26.61% (35998/135261)
Branch Coverage: 23.43% (18395/78514)
Coverage Report: http://coverage.selectdb-in.cc/coverage/99c4d391690d183dc9196759e708adbfe2d15c60_99c4d391690d183dc9196759e708adbfe2d15c60/report/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) cancel pipeline tasks if timeout [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31635:
URL: https://github.com/apache/doris/pull/31635#issuecomment-1972433581
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) cancel pipeline tasks if timeout [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #31635:
URL: https://github.com/apache/doris/pull/31635#issuecomment-1972400062
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) cancel pipeline tasks if timeout [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 merged PR #31635:
URL: https://github.com/apache/doris/pull/31635
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) cancel pipeline tasks if timeout [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on code in PR #31635:
URL: https://github.com/apache/doris/pull/31635#discussion_r1508438501
##########
be/src/runtime/plan_fragment_executor.cpp:
##########
@@ -113,7 +113,7 @@ PlanFragmentExecutor::~PlanFragmentExecutor() {
}
Status PlanFragmentExecutor::prepare(const TExecPlanFragmentParams& request) {
Review Comment:
warning: function 'prepare' has cognitive complexity of 62 (threshold 50) [readability-function-cognitive-complexity]
```cpp
Status PlanFragmentExecutor::prepare(const TExecPlanFragmentParams& request) {
^
```
<details>
<summary>Additional context</summary>
**be/src/runtime/plan_fragment_executor.cpp:115:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
if (request.__isset.query_options && request.query_options.__isset.execution_timeout) {
^
```
**be/src/runtime/plan_fragment_executor.cpp:115:** +1
```cpp
if (request.__isset.query_options && request.query_options.__isset.execution_timeout) {
^
```
**be/src/runtime/plan_fragment_executor.cpp:135:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
if (request.__isset.backend_id) {
^
```
**be/src/runtime/plan_fragment_executor.cpp:138:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
if (request.__isset.import_label) {
^
```
**be/src/runtime/plan_fragment_executor.cpp:141:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
if (request.__isset.db_name) {
^
```
**be/src/runtime/plan_fragment_executor.cpp:144:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
if (request.__isset.load_job_id) {
^
```
**be/src/runtime/plan_fragment_executor.cpp:147:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
if (request.__isset.wal_id) {
^
```
**be/src/runtime/plan_fragment_executor.cpp:150:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
if (request.__isset.content_length) {
^
```
**be/src/runtime/plan_fragment_executor.cpp:154:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
if (request.query_options.__isset.is_report_success) {
^
```
**be/src/runtime/plan_fragment_executor.cpp:159:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
if (request.is_simplified_param) {
^
```
**be/src/runtime/plan_fragment_executor.cpp:161:** +1, nesting level increased to 1
```cpp
} else {
^
```
**be/src/runtime/plan_fragment_executor.cpp:163:** +2, including nesting penalty of 1, nesting level increased to 2
```cpp
RETURN_IF_ERROR(
^
```
**be/src/common/status.h:541:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do { \
^
```
**be/src/runtime/plan_fragment_executor.cpp:163:** +3, including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR(
^
```
**be/src/common/status.h:543:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/runtime/plan_fragment_executor.cpp:170:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
RETURN_IF_ERROR_OR_CATCH_EXCEPTION(ExecNode::create_tree(
^
```
**be/src/common/exception.h:105:** expanded from macro 'RETURN_IF_ERROR_OR_CATCH_EXCEPTION'
```cpp
do { \
^
```
**be/src/runtime/plan_fragment_executor.cpp:170:** +2, including nesting penalty of 1, nesting level increased to 2
```cpp
RETURN_IF_ERROR_OR_CATCH_EXCEPTION(ExecNode::create_tree(
^
```
**be/src/common/exception.h:111:** expanded from macro 'RETURN_IF_ERROR_OR_CATCH_EXCEPTION'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/runtime/plan_fragment_executor.cpp:170:** +2, including nesting penalty of 1, nesting level increased to 2
```cpp
RETURN_IF_ERROR_OR_CATCH_EXCEPTION(ExecNode::create_tree(
^
```
**be/src/common/exception.h:115:** expanded from macro 'RETURN_IF_ERROR_OR_CATCH_EXCEPTION'
```cpp
} catch (const doris::Exception& e) { \
^
```
**be/src/runtime/plan_fragment_executor.cpp:170:** +3, including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR_OR_CATCH_EXCEPTION(ExecNode::create_tree(
^
```
**be/src/common/exception.h:116:** expanded from macro 'RETURN_IF_ERROR_OR_CATCH_EXCEPTION'
```cpp
if (e.code() == doris::ErrorCode::MEM_ALLOC_FAILED) { \
^
```
**be/src/runtime/plan_fragment_executor.cpp:184:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
RETURN_IF_ERROR(_plan->prepare(_runtime_state.get()));
^
```
**be/src/common/status.h:541:** expanded from macro 'RETURN_IF_ERROR'
```cpp
do { \
^
```
**be/src/runtime/plan_fragment_executor.cpp:184:** +2, including nesting penalty of 1, nesting level increased to 2
```cpp
RETURN_IF_ERROR(_plan->prepare(_runtime_state.get()));
^
```
**be/src/common/status.h:543:** expanded from macro 'RETURN_IF_ERROR'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/runtime/plan_fragment_executor.cpp:192:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
for (int i = 0; i < scan_nodes.size(); ++i) {
^
```
**be/src/runtime/plan_fragment_executor.cpp:195:** +2, including nesting penalty of 1, nesting level increased to 2
```cpp
if (typeid(*node) == typeid(vectorized::NewOlapScanNode) ||
^
```
**be/src/runtime/plan_fragment_executor.cpp:199:** +1
```cpp
typeid(*node) == typeid(vectorized::NewJdbcScanNode) ||
^
```
**be/src/runtime/plan_fragment_executor.cpp:205:** +1, nesting level increased to 2
```cpp
} else {
^
```
**be/src/runtime/plan_fragment_executor.cpp:222:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
if (request.fragment.__isset.output_sink) {
^
```
**be/src/runtime/plan_fragment_executor.cpp:223:** +2, including nesting penalty of 1, nesting level increased to 2
```cpp
RETURN_IF_ERROR_OR_CATCH_EXCEPTION(
^
```
**be/src/common/exception.h:105:** expanded from macro 'RETURN_IF_ERROR_OR_CATCH_EXCEPTION'
```cpp
do { \
^
```
**be/src/runtime/plan_fragment_executor.cpp:223:** +3, including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR_OR_CATCH_EXCEPTION(
^
```
**be/src/common/exception.h:111:** expanded from macro 'RETURN_IF_ERROR_OR_CATCH_EXCEPTION'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/runtime/plan_fragment_executor.cpp:223:** +3, including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR_OR_CATCH_EXCEPTION(
^
```
**be/src/common/exception.h:115:** expanded from macro 'RETURN_IF_ERROR_OR_CATCH_EXCEPTION'
```cpp
} catch (const doris::Exception& e) { \
^
```
**be/src/runtime/plan_fragment_executor.cpp:223:** +4, including nesting penalty of 3, nesting level increased to 4
```cpp
RETURN_IF_ERROR_OR_CATCH_EXCEPTION(
^
```
**be/src/common/exception.h:116:** expanded from macro 'RETURN_IF_ERROR_OR_CATCH_EXCEPTION'
```cpp
if (e.code() == doris::ErrorCode::MEM_ALLOC_FAILED) { \
^
```
**be/src/runtime/plan_fragment_executor.cpp:227:** +2, including nesting penalty of 1, nesting level increased to 2
```cpp
RETURN_IF_ERROR_OR_CATCH_EXCEPTION(_sink->prepare(runtime_state()));
^
```
**be/src/common/exception.h:105:** expanded from macro 'RETURN_IF_ERROR_OR_CATCH_EXCEPTION'
```cpp
do { \
^
```
**be/src/runtime/plan_fragment_executor.cpp:227:** +3, including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR_OR_CATCH_EXCEPTION(_sink->prepare(runtime_state()));
^
```
**be/src/common/exception.h:111:** expanded from macro 'RETURN_IF_ERROR_OR_CATCH_EXCEPTION'
```cpp
if (UNLIKELY(!_status_.ok())) { \
^
```
**be/src/runtime/plan_fragment_executor.cpp:227:** +3, including nesting penalty of 2, nesting level increased to 3
```cpp
RETURN_IF_ERROR_OR_CATCH_EXCEPTION(_sink->prepare(runtime_state()));
^
```
**be/src/common/exception.h:115:** expanded from macro 'RETURN_IF_ERROR_OR_CATCH_EXCEPTION'
```cpp
} catch (const doris::Exception& e) { \
^
```
**be/src/runtime/plan_fragment_executor.cpp:227:** +4, including nesting penalty of 3, nesting level increased to 4
```cpp
RETURN_IF_ERROR_OR_CATCH_EXCEPTION(_sink->prepare(runtime_state()));
^
```
**be/src/common/exception.h:116:** expanded from macro 'RETURN_IF_ERROR_OR_CATCH_EXCEPTION'
```cpp
if (e.code() == doris::ErrorCode::MEM_ALLOC_FAILED) { \
^
```
**be/src/runtime/plan_fragment_executor.cpp:229:** +2, including nesting penalty of 1, nesting level increased to 2
```cpp
if (sink_profile != nullptr) {
^
```
**be/src/runtime/plan_fragment_executor.cpp:232:** +1, nesting level increased to 1
```cpp
} else {
^
```
**be/src/runtime/plan_fragment_executor.cpp:238:** +1, including nesting penalty of 0, nesting level increased to 1
```cpp
if (sink_profile != nullptr) {
^
```
**be/src/runtime/plan_fragment_executor.cpp:240:** +1, nesting level increased to 1
```cpp
} else {
^
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) cancel pipeline tasks if timeout [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31635:
URL: https://github.com/apache/doris/pull/31635#issuecomment-1972393568
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) cancel pipeline tasks if timeout [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #31635:
URL: https://github.com/apache/doris/pull/31635#issuecomment-1972455847
<details>
<summary>TPC-H: <b>Total hot run time: 38124 ms</b></summary>
```
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 99c4d391690d183dc9196759e708adbfe2d15c60, data reload: false
------ Round 1 ----------------------------------
q1 17663 3999 4071 3999
q2 2048 151 141 141
q3 10554 946 924 924
q4 4668 942 946 942
q5 7630 2879 3035 2879
q6 181 125 122 122
q7 1314 840 818 818
q8 9488 2100 2090 2090
q9 7378 6547 6489 6489
q10 8212 2559 2565 2559
q11 418 219 223 219
q12 781 320 328 320
q13 18012 3015 2996 2996
q14 272 276 267 267
q15 470 437 429 429
q16 488 390 401 390
q17 944 901 867 867
q18 6667 5889 5903 5889
q19 1555 1531 1526 1526
q20 546 288 306 288
q21 7523 3758 3677 3677
q22 826 298 293 293
Total cold run time: 107638 ms
Total hot run time: 38124 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4020 4004 4010 4004
q2 320 223 225 223
q3 2982 2930 2923 2923
q4 1855 1867 1792 1792
q5 5224 5263 5228 5228
q6 204 118 123 118
q7 2264 1819 1815 1815
q8 3207 3279 3306 3279
q9 8570 8508 8552 8508
q10 6153 3712 3721 3712
q11 527 440 441 440
q12 694 535 527 527
q13 8650 2813 2780 2780
q14 270 252 255 252
q15 478 445 432 432
q16 441 412 420 412
q17 1704 1669 1664 1664
q18 7872 7440 7398 7398
q19 1656 1605 1611 1605
q20 1923 1694 1711 1694
q21 4945 4823 4717 4717
q22 551 495 478 478
Total cold run time: 64510 ms
Total hot run time: 54001 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) cancel pipeline tasks if timeout [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31635:
URL: https://github.com/apache/doris/pull/31635#issuecomment-1972433391
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) cancel pipeline tasks if timeout [doris]
Posted by "yiguolei (via GitHub)" <gi...@apache.org>.
yiguolei commented on code in PR #31635:
URL: https://github.com/apache/doris/pull/31635#discussion_r1508430777
##########
be/src/pipeline/pipeline_fragment_context.cpp:
##########
@@ -214,6 +225,9 @@ Status PipelineFragmentContext::prepare(const doris::TPipelineFragmentParams& re
if (_prepared) {
return Status::InternalError("Already prepared");
}
+ if (request.__isset.query_options) {
+ _timeout = request.query_options.execution_timeout;
Review Comment:
__isset.execution_timeout
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [pipelineX](bug) cancel pipeline tasks if timeout [doris]
Posted by "Gabriel39 (via GitHub)" <gi...@apache.org>.
Gabriel39 commented on PR #31635:
URL: https://github.com/apache/doris/pull/31635#issuecomment-1972385533
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org