You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "wangbo (via GitHub)" <gi...@apache.org> on 2023/11/27 10:56:48 UTC

[PR] [Fix](executor)Make blockscheduler first stop then delete [doris]

wangbo opened a new pull request, #27645:
URL: https://github.com/apache/doris/pull/27645

   ## Proposed changes
   Whether ```first stop BlockScheduler then stop TaskScheduler``` or ```first stop TaskScheduler then stop BlockScheduler``` does not matter, because this just stop active thread.
   But the stop of a scheduler must precede the release of resources.
   Currently BlockScheduler's stop is after resource delete, this may cause NPE.
   because the PipelineTask maybe is deleted before BlockScheduler stop active thread.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1827702935

   
   <details>
   <summary>TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'</summary>
   
   ```
   Tpch sf100 test result on commit 7a67f1decaac95a6688f40c93d2356f455bbcb29, data reload: false
   
   run tpch-sf100 query with default conf and session variables
   q1	4911	4620	4637	4620
   q2	360	163	160	160
   q3	2033	1935	1959	1935
   q4	1392	1271	1234	1234
   q5	3949	3952	3994	3952
   q6	259	135	130	130
   q7	1420	871	894	871
   q8	2791	2790	2806	2790
   q9	9971	9676	9590	9590
   q10	3446	3519	3514	3514
   q11	381	252	249	249
   q12	440	292	302	292
   q13	4554	3833	3802	3802
   q14	311	286	290	286
   q15	582	531	518	518
   q16	665	597	580	580
   q17	1138	951	921	921
   q18	7850	7428	7536	7428
   q19	1659	1673	1680	1673
   q20	525	299	309	299
   q21	4489	4029	4081	4029
   q22	478	382	380	380
   Total cold run time: 53604 ms
   Total hot run time: 49253 ms
   
   run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
   q1	4631	4619	4572	4572
   q2	353	243	258	243
   q3	4039	4016	4018	4016
   q4	2724	2717	2731	2717
   q5	9519	9607	9595	9595
   q6	250	124	127	124
   q7	3062	2514	2504	2504
   q8	4420	4431	4402	4402
   q9	13237	13013	13141	13013
   q10	4032	4184	4148	4148
   q11	803	753	648	648
   q12	956	823	838	823
   q13	4324	3572	3550	3550
   q14	377	359	340	340
   q15	577	521	530	521
   q16	741	683	695	683
   q17	3871	3893	3840	3840
   q18	9654	9174	9131	9131
   q19	1831	1799	1757	1757
   q20	2377	2091	2076	2076
   q21	8802	8573	8583	8573
   q22	962	826	778	778
   Total cold run time: 81542 ms
   Total hot run time: 78054 ms
   ```
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]

Posted by "zhiqiang-hhhh (via GitHub)" <gi...@apache.org>.
zhiqiang-hhhh commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1828950963

   This PR is unfinished. Lifecycle of `_global_block_scheduler` is not controlled explicitly. We need another refactor.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1827615059

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]

Posted by "doris-robot (via GitHub)" <gi...@apache.org>.
doris-robot commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1827719749

   (From new machine)TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 45.44 seconds
    stream load tsv:          562 seconds loaded 74807831229 Bytes, about 126 MB/s
    stream load json:         31 seconds loaded 2358488459 Bytes, about 72 MB/s
    stream load orc:          67 seconds loaded 1101869774 Bytes, about 15 MB/s
    stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
    insert into select:          28.7 seconds inserted 10000000 Rows, about 348K ops/s
    storage size: 17098106534 Bytes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]

Posted by "wangbo (via GitHub)" <gi...@apache.org>.
wangbo commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1827604808

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]

Posted by "wangbo (via GitHub)" <gi...@apache.org>.
wangbo merged PR #27645:
URL: https://github.com/apache/doris/pull/27645


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1828501871

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1827617198

   clang-tidy review says "All clean, LGTM! :+1:"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]

Posted by "wangbo (via GitHub)" <gi...@apache.org>.
wangbo commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1827607784

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org


Re: [PR] [Fix](executor)Make blockscheduler first stop then delete [doris]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27645:
URL: https://github.com/apache/doris/pull/27645#issuecomment-1828501893

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org