You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by le...@apache.org on 2021/10/22 03:42:03 UTC
[dolphinscheduler-website] branch master updated: Add switch task
(#457)
This is an automated email from the ASF dual-hosted git repository.
leonbao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler-website.git
The following commit(s) were added to refs/heads/master by this push:
new 6df548c Add switch task (#457)
6df548c is described below
commit 6df548cf94228ee0a8bcb95bad085f7e5e8e530b
Author: Jiajie Zhong <zh...@hotmail.com>
AuthorDate: Fri Oct 22 11:41:56 2021 +0800
Add switch task (#457)
* Add switch task
* Change order
---
docs/en-us/dev/user_doc/guide/task/conditions.md | 4 +++
docs/en-us/dev/user_doc/guide/task/switch.md | 37 +++++++++++++++++++
docs/zh-cn/dev/user_doc/guide/task/conditions.md | 4 +++
docs/zh-cn/dev/user_doc/guide/task/switch.md | 37 +++++++++++++++++++
docs/zh-cn/dev/user_doc/switch_node.md | 44 -----------------------
img/switch.png | Bin 0 -> 885 bytes
img/switch_configure.jpg | Bin 0 -> 142764 bytes
site_config/docsdev.js | 16 ++++++---
8 files changed, 94 insertions(+), 48 deletions(-)
diff --git a/docs/en-us/dev/user_doc/guide/task/conditions.md b/docs/en-us/dev/user_doc/guide/task/conditions.md
index 80b0d82..5f63b4e 100644
--- a/docs/en-us/dev/user_doc/guide/task/conditions.md
+++ b/docs/en-us/dev/user_doc/guide/task/conditions.md
@@ -22,3 +22,7 @@ Drag in the toolbar<img src="/img/conditions.png" width="20"/>The task node to t
- Upstream condition selection: one or more upstream tasks can be selected for conditions
- Add the upstream dependency: Use the first parameter to choose task name, and the second parameter for status of the upsteam task.
- Upstream task relationship: we use `and` and `or` operators to handle complex relationship of upstream when multiple upstream tasks for Conditions task
+
+## Related task
+
+[switch](switch.md): [Condition](conditions.md)task mainly executes the corresponding branch based on the execution status (success, failure) of the upstream node. The [Switch](switch.md) task mainly executes the corresponding branch based on the value of the [global parameter](../parameter/global.md) and the judgment expression result written by the user.
diff --git a/docs/en-us/dev/user_doc/guide/task/switch.md b/docs/en-us/dev/user_doc/guide/task/switch.md
new file mode 100644
index 0000000..7dc71d5
--- /dev/null
+++ b/docs/en-us/dev/user_doc/guide/task/switch.md
@@ -0,0 +1,37 @@
+# Switch
+
+Switch is a conditional judgment node, which branch should be executes according to the value of [global variable](../parameter/global.md) and the expression result written by the user.
+
+## Create
+
+Drag the <img src="/img/switch.png" width="20"/> in the tool bar to create task. **Note** After the switch task is created, you must configure it downstream to make parameter `Branch flow` work.
+
+## Parameter
+
+- Node name: The node name in a workflow definition is unique.
+- Run flag: Identifies whether this node can be scheduled normally, if it does not need to be executed, you can turn on the prohibition switch.
+- Descriptive information: describe the function of the node.
+- Task priority: When the number of worker threads is insufficient, they are executed in order from high to low, and when the priority is the same, they are executed according to the first-in first-out principle.
+- Worker grouping: Tasks are assigned to the machines of the worker group to execute. If Default is selected, a worker machine will be randomly selected for execution.
+- Number of failed retry attempts: The number of times the task failed to be resubmitted. It supports drop-down and hand-filling.
+- Failed retry interval: The time interval for resubmitting the task after a failed task. It supports drop-down and hand-filling.
+- Timeout alarm: Check the timeout alarm and timeout failure. When the task exceeds the "timeout period", an alarm email will be sent and the task execution will fail.
+- condition: You can configure multiple conditions for the switch task. When the conditions are true, the configured branch will be executed. You can configure multiple different conditions to satisfy different businesses.
+- Branch flow: The default branch flow, when all the conditions are false, it will execute this branch flow.
+
+## Detail
+
+Here we have three tasks, the dependencies are `A -> B -> [C, D]`, and task_a is a shell task and task_b is a switch task
+
+- In task A, a global variable named `id` is defined through [global variable](../parameter/global.md), and the declaration method is `${setValue(id=1)}`
+- Task B adds conditions and uses global variables declared upstream to achieve conditional judgment (note that global variables must exist when the switch is running, which means that switch task can use global variables that are not directly upstream). We want workflow execute task C when id = 1 else run task D
+ - Configure task C to run when the global variable `id=1`. Then edit `${id} == 1` in the condition of task B, select `C` as branch flow
+ - For other tasks, select `D` as branch flow
+
+Switch task configuration is as follows
+
+
+
+## Related Task
+
+[condition](conditions.md)：[Condition](conditions.md)task mainly executes the corresponding branch based on the execution status (success, failure) of the upstream node. The [Switch](switch.md) task mainly executes the corresponding branch based on the value of the [global parameter](../parameter/global.md) and the judgment expression result written by the user.
\ No newline at end of file
diff --git a/docs/zh-cn/dev/user_doc/guide/task/conditions.md b/docs/zh-cn/dev/user_doc/guide/task/conditions.md
index 2dcb7a1..80b0b4b 100644
--- a/docs/zh-cn/dev/user_doc/guide/task/conditions.md
+++ b/docs/zh-cn/dev/user_doc/guide/task/conditions.md
@@ -24,3 +24,7 @@ Conditions是一个条件节点，根据上游任务运行状态，判断应该
- 上游任务关系选择：当有多个上游任务时，可以通过`且`以及`或`操作符实现任务的复杂关系。
<!-- TODO 缺少condintions任务的截图，分别需要一个简单任务和一个复杂任务 -->
+
+## 相关任务
+
+[switch](switch.md)：Condition节点主要依据上游节点的执行状态（成功、失败）执行对应分支。[Switch](switch.md)节点主要依据全局变量的值和用户所编写的表达式判断结果执行对应分支
diff --git a/docs/zh-cn/dev/user_doc/guide/task/switch.md b/docs/zh-cn/dev/user_doc/guide/task/switch.md
new file mode 100644
index 0000000..e4e315b
--- /dev/null
+++ b/docs/zh-cn/dev/user_doc/guide/task/switch.md
@@ -0,0 +1,37 @@
+# Switch
+
+Switch是一个条件判断节点，依据[全局变量](../parameter/global.md)的值和用户所编写的表达式判断结果执行对应分支。
+
+## 创建任务
+
+拖动工具栏中的<img src="/img/switch.png" width="20"/>任务节点到画板中即能完成任务创建，**注意**switch任务创建后，要先配置上下游，才能配置任务分支的参数
+
+## 任务参数
+
+- 节点名称：一个工作流定义中的节点名称是唯一的。
+- 运行标志：标识这个节点是否能正常调度,如果不需要执行，可以打开禁止执行开关。
+- 描述信息：描述该节点的功能。
+- 任务优先级：worker线程数不足时，根据优先级从高到低依次执行，优先级一样时根据先进先出原则执行。
+- Worker分组：任务分配给worker组的机器机执行，选择Default，会随机选择一台worker机执行。
+- 失败重试次数：任务失败重新提交的次数，支持下拉和手填。
+- 失败重试间隔：任务失败重新提交任务的时间间隔，支持下拉和手填。
+- 超时告警：勾选超时告警、超时失败，当任务超过"超时时长"后，会发送告警邮件并且任务执行失败.
+- 条件：可以为switch任务配置多个条件，当条件满足时，就会执行指定的分支，可以配置多个不同的条件来满足不同的业务
+- 分支流转：默认的流转内容，当**条件**中的内容为全部不符合要求时，则运行**分支流转**中指定的分支
+
+## 详细流程
+
+假设我们三个任务，其依赖关系是 `A -> B -> [C, D]` 其中task_a是shell任务，task_b是switch任务
+
+- 任务A中通过[全局变量](../parameter/global.md)定义了名为`id`的全局变量，声明方式为`${setValue(id=1)}`
+- 任务B增加条件，使用上游声明的全局变量实现条件判断（注意switch运行时存在的全局变量就行，意味着可以是非直接上游产生的全局变量）。下面我们想要实现当id为1时，运行任务C，其他运行任务D
+ - 配置当全局变量`id=1`时，运行任务C。则在任务B的条件中编辑`${id} == 1`，分支流转选择`C`
+ - 对于其他任务，在分支流转中选择`D`
+
+最终switch任务的配置如下
+
+
+
+## 相关任务
+
+[condition](conditions.md)：[Condition](conditions.md)节点主要依据上游节点的执行状态（成功、失败）执行对应分支。Switch节点主要依据全局变量的值和用户所编写的表达式判断结果执行对应分支
diff --git a/docs/zh-cn/dev/user_doc/switch_node.md b/docs/zh-cn/dev/user_doc/switch_node.md
index 4b80359..b5cdb64 100644
--- a/docs/zh-cn/dev/user_doc/switch_node.md
+++ b/docs/zh-cn/dev/user_doc/switch_node.md
@@ -2,32 +2,8 @@
Dolphinscheduler中目前包含条件判断的节点有两个，condition节点和switch节点。Condition节点主要依据上游节点的执行状态（成功、失败）执行对应分支。Switch节点主要依据全局变量的值和用户所编写的表达式判断结果执行对应分支。本文所讲述的是switch节点相关内容。
-
-
-#### ***\*用户使用手册\****
-
-1.首先我们画一个简单的工作流定义：
-
-<img src="/img/switchNode/image-20210901144536115.png" alt="image-20210901144536115" style="zoom: 67%;" />
-
-2.其中sql1节点会产生一个全局变量（有关于全局变量的请参考Global-Params），名称为id：
-
-<img src="/img/switchNode/image-20210901144649439.png" alt="image-20210901144649439" style="zoom:67%;" />
-
-3. 用户在switch节点的“条件”框内定义表达式，当然这个表达式需要用到上游节点产生的全局变量，而上游节点不只是相邻的上游节点，可以是有依赖关系的任何一个上游节点。变量的引用仍然使用${变量名}的方式。
-
-<img src="/img/switchNode/image-20210901144752457.png" alt="image-20210901144752457" style="zoom:67%;" />
-
-您可以编写多个条件，对应执行不同的分支。如果只定义了条件，没有定义“分支流转”的部分，那当满足该条件后，工作流定义的该分支便结束了。上图中突出的“分支流转”，也就是定义了sql4的部分，是当所有的条件都不满足之后选择的分支。当然这部分也可以不选择任何下游节点，同样上面的条件都不满足后该分支便会结束。
-
-4. 下游有三个分支，我们简单的工作流定义中每个分支只包含了一个节点。实际上，sql1定义的全局变量id的值为12，最终经过switch节点运行之后，sql3、sql4会被抛弃运行，sql2被执行。
-
-
-
#### ***\*开发文档\****
-
-
1. 用户定义的表达式和分支流转的信息存在了taskdefinition中的taskParams中，当switch被执行到时，会被格式化为SwitchParameters。
2. SwitchTaskExecThread从上到下（用户在页面上定义的表达式顺序）处理switch中定义的表达式，从varPool中获取变量的值，通过js解析表达式，如果表达式返回true，则停止检查，并且记录该表达式的顺序，这里我们记录为resultConditionLocation。SwitchTaskExecThread的任务便结束了。
@@ -37,23 +13,3 @@ Dolphinscheduler中目前包含条件判断的节点有两个，condition节点
4. DagHelper.parsePostNodes中如果发现当前节点（刚刚运行完成功的节点）是switch节点的话，会获取resultConditionLocation，将SwitchParameters中除了resultConditionLocation以外的其他分支全部skip掉。这样留下来的就只有需要执行的分支了。
以上便是switch工作的流程。
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/img/switch.png b/img/switch.png
new file mode 100644
index 0000000..02717ea
Binary files /dev/null and b/img/switch.png differ
diff --git a/img/switch_configure.jpg b/img/switch_configure.jpg
new file mode 100644
index 0000000..75bbd9c
Binary files /dev/null and b/img/switch_configure.jpg differ
diff --git a/site_config/docsdev.js b/site_config/docsdev.js
index b0b1189..c75c3b1 100644
--- a/site_config/docsdev.js
+++ b/site_config/docsdev.js
@@ -97,12 +97,16 @@ export default {
link: '/en-us/docs/dev/user_doc/guide/task/datax.html',
},
{
+ title: 'Pigeon',
+ link: '/en-us/docs/dev/user_doc/guide/task/pigeon.html',
+ },
+ {
title: 'Conditions',
link: '/en-us/docs/dev/user_doc/guide/task/conditions.html',
},
{
- title: 'Pigeon',
- link: '/en-us/docs/dev/user_doc/guide/task/pigeon.html',
+ title: 'Switch',
+ link: '/en-us/docs/dev/user_doc/guide/task/switch.html',
},
],
},
@@ -357,12 +361,16 @@ export default {
link: '/zh-cn/docs/dev/user_doc/guide/task/datax.html',
},
{
+ title: 'Pigeon',
+ link: '/zh-cn/docs/dev/user_doc/guide/task/pigeon.html',
+ },
+ {
title: 'Conditions',
link: '/zh-cn/docs/dev/user_doc/guide/task/conditions.html',
},
{
- title: 'Pigeon',
- link: '/zh-cn/docs/dev/user_doc/guide/task/pigeon.html',
+ title: 'Switch',
+ link: '/zh-cn/docs/dev/user_doc/guide/task/switch.html',
},
],
},