You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "wangbo (via GitHub)" <gi...@apache.org> on 2023/11/27 10:56:48 UTC
[PR] [Fix](executor)Make blockscheduler first stop then delete [doris]
wangbo opened a new pull request, #27645:
URL: https://github.com/apache/doris/pull/27645
## Proposed changes
Whether ```first stop BlockScheduler then stop TaskScheduler``` or ```first stop TaskScheduler then stop BlockScheduler``` does not matter, because this just stop active thread.
But the stop of a scheduler must precede the release of resources.
Currently BlockScheduler's stop is after resource delete, this may cause NPE.
because the PipelineTask maybe is deleted before BlockScheduler stop active thread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1827702935
<details>
<summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
```
Tpch sf100 test result on commit 7a67f1decaac95a6688f40c93d2356f455bbcb29, data reload: false
run tpch-sf100 query with default conf and session variables
q1 4911 4620 4637 4620
q2 360 163 160 160
q3 2033 1935 1959 1935
q4 1392 1271 1234 1234
q5 3949 3952 3994 3952
q6 259 135 130 130
q7 1420 871 894 871
q8 2791 2790 2806 2790
q9 9971 9676 9590 9590
q10 3446 3519 3514 3514
q11 381 252 249 249
q12 440 292 302 292
q13 4554 3833 3802 3802
q14 311 286 290 286
q15 582 531 518 518
q16 665 597 580 580
q17 1138 951 921 921
q18 7850 7428 7536 7428
q19 1659 1673 1680 1673
q20 525 299 309 299
q21 4489 4029 4081 4029
q22 478 382 380 380
Total cold run time: 53604 ms
Total hot run time: 49253 ms
run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 4631 4619 4572 4572
q2 353 243 258 243
q3 4039 4016 4018 4016
q4 2724 2717 2731 2717
q5 9519 9607 9595 9595
q6 250 124 127 124
q7 3062 2514 2504 2504
q8 4420 4431 4402 4402
q9 13237 13013 13141 13013
q10 4032 4184 4148 4148
q11 803 753 648 648
q12 956 823 838 823
q13 4324 3572 3550 3550
q14 377 359 340 340
q15 577 521 530 521
q16 741 683 695 683
q17 3871 3893 3840 3840
q18 9654 9174 9131 9131
q19 1831 1799 1757 1757
q20 2377 2091 2076 2076
q21 8802 8573 8583 8573
q22 962 826 778 778
Total cold run time: 81542 ms
Total hot run time: 78054 ms
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]
Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1828950963
This PR is unfinished. Lifecycle of `_global_block_scheduler` is not controlled explicitly. We need another refactor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1827615059
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]
Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1827719749
(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.44 seconds
stream load tsv: 562 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 31 seconds loaded 2358488459 Bytes, about 72 MB/s
stream load orc: 67 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17098106534 Bytes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]
Posted by "wangbo (via GitHub)" <gi...@apache.org>.
wangbo commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1827604808
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]
Posted by "wangbo (via GitHub)" <gi...@apache.org>.
wangbo merged PR #27645:
URL: https://github.com/apache/doris/pull/27645
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1828501871
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1827617198
clang-tidy review says "All clean, LGTM! :+1:"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]
Posted by "wangbo (via GitHub)" <gi...@apache.org>.
wangbo commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1827607784
run buildall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1828501893
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org