You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by ki...@apache.org on 2021/11/08 01:47:09 UTC

[dolphinscheduler-website] branch master updated: update 2.0 architecture (#506)

This is an automated email from the ASF dual-hosted git repository.

kirs pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler-website.git


The following commit(s) were added to refs/heads/master by this push:
     new 8f8e61c  update 2.0 architecture (#506)
8f8e61c is described below

commit 8f8e61cda8afe6c1dcc6470411d5b61b63ca63bb
Author: OS <29...@users.noreply.github.com>
AuthorDate: Mon Nov 8 09:47:03 2021 +0800

    update 2.0 architecture (#506)
---
 docs/en-us/2.0.0/user_doc/architecture/design.md | 20 ++++++++++----------
 docs/zh-cn/2.0.0/user_doc/architecture/design.md |  7 ++-----
 2 files changed, 12 insertions(+), 15 deletions(-)

diff --git a/docs/en-us/2.0.0/user_doc/architecture/design.md b/docs/en-us/2.0.0/user_doc/architecture/design.md
index 7c69078..79ec65b 100644
--- a/docs/en-us/2.0.0/user_doc/architecture/design.md
+++ b/docs/en-us/2.0.0/user_doc/architecture/design.md
@@ -45,7 +45,7 @@ Before explaining the architecture of the scheduling system, let's first underst
 
 #### 2.2 Start process activity diagram
 <p align="center">
-  <img src="/img/process-start-flow-1.3.0.png" alt="Start process activity diagram"  width="70%" />
+  <img src="/img/master-process-2.0-en.png" alt="Start process activity diagram"  width="70%" />
   <p align="center">
         <em>Start process activity diagram</em>
   </p>
@@ -104,6 +104,7 @@ Before explaining the architecture of the scheduling system, let's first underst
 ###### Centralized thinking
 
 The centralized design concept is relatively simple. The nodes in the distributed cluster are divided into roles according to roles, which are roughly divided into two roles:
+
 <p align="center">
    <img src="https://analysys.github.io/easyscheduler_docs_cn/images/master_slave.png" alt="master-slave character"  width="50%" />
  </p>
@@ -130,21 +131,20 @@ Problems in centralized thought design:
 - In fact, truly decentralized distributed systems are rare. Instead, dynamic centralized distributed systems are constantly pouring out. Under this architecture, the managers in the cluster are dynamically selected, rather than preset, and when the cluster fails, the nodes of the cluster will automatically hold "meetings" to elect new "managers" To preside over the work. The most typical case is Etcd implemented by ZooKeeper and Go language.
 
 
--The decentralization of DolphinScheduler is that the Master/Worker is registered in Zookeeper to realize the non-centralization of the Master cluster and the Worker cluster. The sharding mechanism is used to fairly distribute the workflow for execution on the master, and tasks are sent to the workers for execution through different sending strategies. Specific task
+- The decentralization of DolphinScheduler is that the Master/Worker is registered in Zookeeper to realize the non-centralization of the Master cluster and the Worker cluster. The sharding mechanism is used to fairly distribute the workflow for execution on the master, and tasks are sent to the workers for execution through different sending strategies. Specific task
 
 ##### Second, the master execution process
 
 1. DolphinScheduler uses the sharding algorithm to modulate the command and assigns it according to the sort id of the master. The master converts the received command into a workflow instance, and uses the thread pool to process the workflow instance
 
+2. DolphinScheduler's process of workflow:
 
-2. Dolphinscheduler's process of workflow:
-
-  -Start the workflow through UI or API calls, and persist a command to the database
-  -The Master scans the Command table through the sharding algorithm, generates a workflow instance ProcessInstance, and deletes the Command data at the same time
-  -The Master uses the thread pool to run WorkflowExecuteThread to execute the process of the workflow instance, including building DAG, creating task instance TaskInstance, and sending TaskInstance to worker through netty
-  -After the worker receives the task, it modifies the task status and returns the execution information to the Master
-  -The Master receives the task information, persists it to the database, and stores the state change event in the EventExecuteService event queue
-  -EventExecuteService calls WorkflowExecuteThread according to the event queue to submit subsequent tasks and modify workflow status
+  - Start the workflow through UI or API calls, and persist a command to the database
+  - The Master scans the Command table through the sharding algorithm, generates a workflow instance ProcessInstance, and deletes the Command data at the same time
+  - The Master uses the thread pool to run WorkflowExecuteThread to execute the process of the workflow instance, including building DAG, creating task instance TaskInstance, and sending TaskInstance to worker through netty
+  - After the worker receives the task, it modifies the task status and returns the execution information to the Master
+  - The Master receives the task information, persists it to the database, and stores the state change event in the EventExecuteService event queue
+  - EventExecuteService calls WorkflowExecuteThread according to the event queue to submit subsequent tasks and modify workflow status
 
 ##### Three、Insufficient thread loop waiting problem
 
diff --git a/docs/zh-cn/2.0.0/user_doc/architecture/design.md b/docs/zh-cn/2.0.0/user_doc/architecture/design.md
index 81b0c30..9ac86a4 100644
--- a/docs/zh-cn/2.0.0/user_doc/architecture/design.md
+++ b/docs/zh-cn/2.0.0/user_doc/architecture/design.md
@@ -45,7 +45,7 @@
 
 #### 2.2 启动流程活动图
 <p align="center">
-  <img src="/img/process-start-flow-1.3.0.png" alt="启动流程活动图"  width="70%" />
+  <img src="/img/master-process-2.0-zh_cn.png" alt="Start process activity diagram"  width="70%" />
   <p align="center">
         <em>启动流程活动图</em>
   </p>
@@ -141,7 +141,7 @@
 1. DolphinScheduler使用分片算法将command取模,根据master的排序id分配,master将拿到的command转换成工作流实例,使用线程池处理工作流实例
 
 
-2. dolphinscheduler对工作流的处理流程:
+2. DolphinScheduler对工作流的处理流程:
 
   - 通过UI或者API调用,启动工作流,持久化一条command到数据库中
   - Master通过分片算法,扫描Command表,生成工作流实例ProcessInstance,同时删除Command数据
@@ -150,9 +150,6 @@
   - Master收到任务信息,持久化到数据库,并且将状态变化事件存入EventExecuteService事件队列
   - EventExecuteService根据事件队列调用WorkflowExecuteThread进行后续任务的提交和工作流状态的修改
 
- <p align="center">
-   <img src="/img/master-process-2.0-zh_cn.png" alt="master执行流程"  width="50%" />
- </p>
 
 ##### 三、容错设计
 容错分为服务宕机容错和任务重试,服务宕机容错又分为Master容错和Worker容错两种情况