You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@dolphinscheduler.apache.org by "tangzhi1318@163.com" <ta...@163.com> on 2021/08/02 09:02:39 UTC
About the source code
Hello everyone.
After MasterBaseTaskExecThread submit task to db, the dispatch task will be carried out, but it will be put into the TaskPriorityQueue queue. But the WorkerManagerThread thread takes specific tasks from the workerExecuteQueue queue for execution. How are the two related?
tangzhi1318@163.com
Re:About the source code
Posted by 黄智远 <hz...@acmsmu.cn>.
This explanation also solved the trouble confusing me quite a while! Thanks!
===================
Stay Hungry, Stay Foolish
Martin Huang
hzy@acmsmu.cn
Sent with a Spark
===================
在 2021年8月2日 +0800 PM6:17，Calvin Kirs <ac...@163.com>，写道：
> Worker and Master communicate through Netty, and both queues are local queues.
> WorkerManagerThread gets TaskExecuteThread from workerExecuteQueue and executes it.
> TaskExecuteThread.run() performs AbstractTask.handle() processing, and after processing, calls NettyRemoteChannel in TaskCallbackService.sendResult() to send a responseCommand message to the master
>
>
> Best Wishes！
> CalvinKirs, Apache DolphinScheduler PMC
>
>
> On 08/2/2021 17:02，tangzhi1318@163.com<ta...@163.com> wrote：
>
> Hello everyone.
> After MasterBaseTaskExecThread submit task to db, the dispatch task will be carried out, but it will be put into the TaskPriorityQueue queue. But the WorkerManagerThread thread takes specific tasks from the workerExecuteQueue queue for execution. How are the two related?
>
>
> tangzhi1318@163.com
Re: Re: About the source code
Posted by "tangzhi1318@163.com" <ta...@163.com>.
First of all, thank you for your reply. But I still have some questions: 1. Does dispatchTask finally call MasterBaseTaskExecThread.dispatchTask or TaskPriorityQueueConsumer.dispatch? 2. When did the TaskComsumer (TaskPriorityQueueConsumer) in the picture start? Could not find its related code.
Best Wishes！
tangzhi1318@163.com
From: CalvinKirs
Date: 2021-08-02 18:32
To: dev@dolphinscheduler.apache.org
Subject: Re: About the source code
You can look at the official website:[1]
It might be easier to understand.
If you encounter any problems, you can always email to chat.
[1] https://dolphinscheduler.apache.org/en-us/docs/1.3.6/user_doc/quick-start.html
Best Wishes！
CalvinKirs, Apache DolphinScheduler PMC
On 08/2/2021 18:27，tangzhi1318@163.com<ta...@163.com> wrote：
get it,thx.
tangzhi1318@163.com
From: Calvin Kirs
Date: 2021-08-02 18:17
To: dev@dolphinscheduler.apache.org
Subject: Re:About the source code
Worker and Master communicate through Netty, and both queues are local queues.
WorkerManagerThread gets TaskExecuteThread from workerExecuteQueue and executes it.
TaskExecuteThread.run() performs AbstractTask.handle() processing, and after processing, calls NettyRemoteChannel in TaskCallbackService.sendResult() to send a responseCommand message to the master
Best Wishes！
CalvinKirs, Apache DolphinScheduler PMC
On 08/2/2021 17:02，tangzhi1318@163.com<ta...@163.com> wrote：
Hello everyone.
After MasterBaseTaskExecThread submit task to db, the dispatch task will be carried out, but it will be put into the TaskPriorityQueue queue. But the WorkerManagerThread thread takes specific tasks from the workerExecuteQueue queue for execution. How are the two related?
tangzhi1318@163.com
Re: About the source code
Posted by CalvinKirs <ac...@163.com>.
You can look at the official website:[1]
It might be easier to understand.
If you encounter any problems, you can always email to chat.
[1] https://dolphinscheduler.apache.org/en-us/docs/1.3.6/user_doc/quick-start.html
Best Wishes！
CalvinKirs, Apache DolphinScheduler PMC
On 08/2/2021 18:27，tangzhi1318@163.com<ta...@163.com> wrote：
get it,thx.
tangzhi1318@163.com
From: Calvin Kirs
Date: 2021-08-02 18:17
To: dev@dolphinscheduler.apache.org
Subject: Re:About the source code
Worker and Master communicate through Netty, and both queues are local queues.
WorkerManagerThread gets TaskExecuteThread from workerExecuteQueue and executes it.
TaskExecuteThread.run() performs AbstractTask.handle() processing, and after processing, calls NettyRemoteChannel in TaskCallbackService.sendResult() to send a responseCommand message to the master
Best Wishes！
CalvinKirs, Apache DolphinScheduler PMC
On 08/2/2021 17:02，tangzhi1318@163.com<ta...@163.com> wrote：
Hello everyone.
After MasterBaseTaskExecThread submit task to db, the dispatch task will be carried out, but it will be put into the TaskPriorityQueue queue. But the WorkerManagerThread thread takes specific tasks from the workerExecuteQueue queue for execution. How are the two related?
tangzhi1318@163.com
Re: Re:About the source code
Posted by "tangzhi1318@163.com" <ta...@163.com>.
get it,thx.
tangzhi1318@163.com
From: Calvin Kirs
Date: 2021-08-02 18:17
To: dev@dolphinscheduler.apache.org
Subject: Re:About the source code
Worker and Master communicate through Netty, and both queues are local queues.
WorkerManagerThread gets TaskExecuteThread from workerExecuteQueue and executes it.
TaskExecuteThread.run() performs AbstractTask.handle() processing, and after processing, calls NettyRemoteChannel in TaskCallbackService.sendResult() to send a responseCommand message to the master
Best Wishes！
CalvinKirs, Apache DolphinScheduler PMC
On 08/2/2021 17:02，tangzhi1318@163.com<ta...@163.com> wrote：
Hello everyone.
After MasterBaseTaskExecThread submit task to db, the dispatch task will be carried out, but it will be put into the TaskPriorityQueue queue. But the WorkerManagerThread thread takes specific tasks from the workerExecuteQueue queue for execution. How are the two related?
tangzhi1318@163.com
Re:About the source code
Posted by Calvin Kirs <ac...@163.com>.
Worker and Master communicate through Netty, and both queues are local queues.
WorkerManagerThread gets TaskExecuteThread from workerExecuteQueue and executes it.
TaskExecuteThread.run() performs AbstractTask.handle() processing, and after processing, calls NettyRemoteChannel in TaskCallbackService.sendResult() to send a responseCommand message to the master
Best Wishes！
CalvinKirs, Apache DolphinScheduler PMC
On 08/2/2021 17:02，tangzhi1318@163.com<ta...@163.com> wrote：
Hello everyone.
After MasterBaseTaskExecThread submit task to db, the dispatch task will be carried out, but it will be put into the TaskPriorityQueue queue. But the WorkerManagerThread thread takes specific tasks from the workerExecuteQueue queue for execution. How are the two related?
tangzhi1318@163.com