You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by gi...@apache.org on 2021/01/17 14:24:35 UTC
[incubator-dolphinscheduler-website] branch asf-site updated:
Automated deployment: cd9e3bc749928226c8c2ad0161621915860d077f
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-dolphinscheduler-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new d80c600 Automated deployment: cd9e3bc749928226c8c2ad0161621915860d077f
d80c600 is described below
commit d80c600259015662654bd5a252748ff975f275c3
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Sun Jan 17 14:24:24 2021 +0000
Automated deployment: cd9e3bc749928226c8c2ad0161621915860d077f
---
en-us/docs/1.3.4/user_doc/task-structure.html | 3282 +++++++++++++++++++++++++
en-us/docs/1.3.4/user_doc/task-structure.json | 6 +
2 files changed, 3288 insertions(+)
diff --git a/en-us/docs/1.3.4/user_doc/task-structure.html b/en-us/docs/1.3.4/user_doc/task-structure.html
new file mode 100644
index 0000000..d38f127
--- /dev/null
+++ b/en-us/docs/1.3.4/user_doc/task-structure.html
@@ -0,0 +1,3282 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+ <meta charset="UTF-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
+ <meta name="keywords" content="task-structure" />
+ <meta name="description" content="task-structure" />
+ <title>task-structure</title>
+ <link rel="shortcut icon" href="/img/favicon.ico"/>
+ <link rel="stylesheet" href="/build/documentation.md.css" />
+</head>
+<body>
+ <div id="root"><div class="md2html documentation-page" data-reactroot=""><header class="header-container header-container-normal"><div class="header-body"><a href="/en-us/index.html"><img class="logo" src="/img/hlogo_colorful.svg"/></a><div class="search search-normal"><span class="icon-search"></span></div><span class="language-switch language-switch-normal">中</span><div class="header-menu"><img class="header-menu-toggle" src="/img/system/menu_gray.png"/><div><ul class="ant-menu black [...]
+<p>All tasks created in Dolphinscheduler are saved in the t_ds_process_definition table.</p>
+<p>The following shows the 't_ds_process_definition' table structure:</p>
+<table>
+<thead>
+<tr>
+<th>No.</th>
+<th>field</th>
+<th>type</th>
+<th>description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>1</td>
+<td>id</td>
+<td>int(11)</td>
+<td>primary key</td>
+</tr>
+<tr>
+<td>2</td>
+<td>name</td>
+<td>varchar(255)</td>
+<td>process definition name</td>
+</tr>
+<tr>
+<td>3</td>
+<td>version</td>
+<td>int(11)</td>
+<td>process definition version</td>
+</tr>
+<tr>
+<td>4</td>
+<td>release_state</td>
+<td>tinyint(4)</td>
+<td>release status of process definition: 0 not online, 1 online</td>
+</tr>
+<tr>
+<td>5</td>
+<td>project_id</td>
+<td>int(11)</td>
+<td>project id</td>
+</tr>
+<tr>
+<td>6</td>
+<td>user_id</td>
+<td>int(11)</td>
+<td>user id of the process definition</td>
+</tr>
+<tr>
+<td>7</td>
+<td>process_definition_json</td>
+<td>longtext</td>
+<td>process definition JSON</td>
+</tr>
+<tr>
+<td>8</td>
+<td>description</td>
+<td>text</td>
+<td>process definition description</td>
+</tr>
+<tr>
+<td>9</td>
+<td>global_params</td>
+<td>text</td>
+<td>global parameters</td>
+</tr>
+<tr>
+<td>10</td>
+<td>flag</td>
+<td>tinyint(4)</td>
+<td>specify whether the process is available: 0 is not available, 1 is available</td>
+</tr>
+<tr>
+<td>11</td>
+<td>locations</td>
+<td>text</td>
+<td>node location infomation</td>
+</tr>
+<tr>
+<td>12</td>
+<td>connects</td>
+<td>text</td>
+<td>node connectivity info</td>
+</tr>
+<tr>
+<td>13</td>
+<td>receivers</td>
+<td>text</td>
+<td>receivers</td>
+</tr>
+<tr>
+<td>14</td>
+<td>receivers_cc</td>
+<td>text</td>
+<td>CC receivers</td>
+</tr>
+<tr>
+<td>15</td>
+<td>create_time</td>
+<td>datetime</td>
+<td>create time</td>
+</tr>
+<tr>
+<td>16</td>
+<td>timeout</td>
+<td>int(11)</td>
+<td>timeout</td>
+</tr>
+<tr>
+<td>17</td>
+<td>tenant_id</td>
+<td>int(11)</td>
+<td>tenant id</td>
+</tr>
+<tr>
+<td>18</td>
+<td>update_time</td>
+<td>datetime</td>
+<td>update time</td>
+</tr>
+<tr>
+<td>19</td>
+<td>modify_by</td>
+<td>varchar(36)</td>
+<td>specifics of the user that made the modification</td>
+</tr>
+<tr>
+<td>20</td>
+<td>resource_ids</td>
+<td>varchar(255)</td>
+<td>resource ids</td>
+</tr>
+</tbody>
+</table>
+<p>The 'process_definition_json' field is the core field, which defines the task information in the DAG diagram, and it is stored in JSON format.</p>
+<p>The following table describes the common data structure.</p>
+<table>
+<thead>
+<tr>
+<th>No.</th>
+<th>field</th>
+<th>type</th>
+<th>description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>1</td>
+<td>globalParams</td>
+<td>Array</td>
+<td>global parameters</td>
+</tr>
+<tr>
+<td>2</td>
+<td>tasks</td>
+<td>Array</td>
+<td>task collections in the process [for the structure of each type, please refer to the following sections]</td>
+</tr>
+<tr>
+<td>3</td>
+<td>tenantId</td>
+<td>int</td>
+<td>tenant ID</td>
+</tr>
+<tr>
+<td>4</td>
+<td>timeout</td>
+<td>int</td>
+<td>timeout</td>
+</tr>
+</tbody>
+</table>
+<p>Data example:</p>
+<pre><code class="language-bash">{
+ <span class="hljs-string">"globalParams"</span>:[
+ {
+ <span class="hljs-string">"prop"</span>:<span class="hljs-string">"golbal_bizdate"</span>,
+ <span class="hljs-string">"direct"</span>:<span class="hljs-string">"IN"</span>,
+ <span class="hljs-string">"type"</span>:<span class="hljs-string">"VARCHAR"</span>,
+ <span class="hljs-string">"value"</span>:<span class="hljs-string">"<span class="hljs-variable">${system.biz.date}</span>"</span>
+ }
+ ],
+ <span class="hljs-string">"tasks"</span>:Array[1],
+ <span class="hljs-string">"tenantId"</span>:0,
+ <span class="hljs-string">"timeout"</span>:0
+}
+</code></pre>
+<h1>The Detailed Explanation of The Storage Structure of Each Task Type</h1>
+<h2>Shell Nodes</h2>
+<p><strong>The node data structure is as follows:</strong></p>
+<table>
+<thead>
+<tr>
+<th>No.</th>
+<th>parameter name</th>
+<th></th>
+<th>type</th>
+<th>description</th>
+<th>notes</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>1</td>
+<td>id</td>
+<td></td>
+<td>String</td>
+<td>task Id</td>
+<td></td>
+</tr>
+<tr>
+<td>2</td>
+<td>type</td>
+<td></td>
+<td>String</td>
+<td>task type</td>
+<td>SHELL</td>
+</tr>
+<tr>
+<td>3</td>
+<td>name</td>
+<td></td>
+<td>String</td>
+<td>task name</td>
+<td></td>
+</tr>
+<tr>
+<td>4</td>
+<td>params</td>
+<td></td>
+<td>Object</td>
+<td>customized parameters</td>
+<td>Json format</td>
+</tr>
+<tr>
+<td>5</td>
+<td></td>
+<td>rawScript</td>
+<td>String</td>
+<td>Shell script</td>
+<td></td>
+</tr>
+<tr>
+<td>6</td>
+<td></td>
+<td>localParams</td>
+<td>Array</td>
+<td>customized local parameters</td>
+<td></td>
+</tr>
+<tr>
+<td>7</td>
+<td></td>
+<td>resourceList</td>
+<td>Array</td>
+<td>resource files</td>
+<td></td>
+</tr>
+<tr>
+<td>8</td>
+<td>description</td>
+<td></td>
+<td>String</td>
+<td>description</td>
+<td></td>
+</tr>
+<tr>
+<td>9</td>
+<td>runFlag</td>
+<td></td>
+<td>String</td>
+<td>execution flag</td>
+<td></td>
+</tr>
+<tr>
+<td>10</td>
+<td>conditionResult</td>
+<td></td>
+<td>Object</td>
+<td>condition branch</td>
+<td></td>
+</tr>
+<tr>
+<td>11</td>
+<td></td>
+<td>successNode</td>
+<td>Array</td>
+<td>jump to node if success</td>
+<td></td>
+</tr>
+<tr>
+<td>12</td>
+<td></td>
+<td>failedNode</td>
+<td>Array</td>
+<td>jump to node if failure</td>
+<td></td>
+</tr>
+<tr>
+<td>13</td>
+<td>dependence</td>
+<td></td>
+<td>Object</td>
+<td>task dependency</td>
+<td>mutual exclusion with params</td>
+</tr>
+<tr>
+<td>14</td>
+<td>maxRetryTimes</td>
+<td></td>
+<td>String</td>
+<td>max retry times</td>
+<td></td>
+</tr>
+<tr>
+<td>15</td>
+<td>retryInterval</td>
+<td></td>
+<td>String</td>
+<td>retry interval</td>
+<td></td>
+</tr>
+<tr>
+<td>16</td>
+<td>timeout</td>
+<td></td>
+<td>Object</td>
+<td>timeout</td>
+<td></td>
+</tr>
+<tr>
+<td>17</td>
+<td>taskInstancePriority</td>
+<td></td>
+<td>String</td>
+<td>task priority</td>
+<td></td>
+</tr>
+<tr>
+<td>18</td>
+<td>workerGroup</td>
+<td></td>
+<td>String</td>
+<td>Worker group</td>
+<td></td>
+</tr>
+<tr>
+<td>19</td>
+<td>preTasks</td>
+<td></td>
+<td>Array</td>
+<td>preposition tasks</td>
+<td></td>
+</tr>
+</tbody>
+</table>
+<p><strong>Node data example:</strong></p>
+<pre><code class="language-bash">{
+ <span class="hljs-string">"type"</span>:<span class="hljs-string">"SHELL"</span>,
+ <span class="hljs-string">"id"</span>:<span class="hljs-string">"tasks-80760"</span>,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"Shell Task"</span>,
+ <span class="hljs-string">"params"</span>:{
+ <span class="hljs-string">"resourceList"</span>:[
+ {
+ <span class="hljs-string">"id"</span>:3,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"run.sh"</span>,
+ <span class="hljs-string">"res"</span>:<span class="hljs-string">"run.sh"</span>
+ }
+ ],
+ <span class="hljs-string">"localParams"</span>:[
+
+ ],
+ <span class="hljs-string">"rawScript"</span>:<span class="hljs-string">"echo "</span>This is a shell script<span class="hljs-string">""</span>
+ },
+ <span class="hljs-string">"description"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"runFlag"</span>:<span class="hljs-string">"NORMAL"</span>,
+ <span class="hljs-string">"conditionResult"</span>:{
+ <span class="hljs-string">"successNode"</span>:[
+ <span class="hljs-string">""</span>
+ ],
+ <span class="hljs-string">"failedNode"</span>:[
+ <span class="hljs-string">""</span>
+ ]
+ },
+ <span class="hljs-string">"dependence"</span>:{
+
+ },
+ <span class="hljs-string">"maxRetryTimes"</span>:<span class="hljs-string">"0"</span>,
+ <span class="hljs-string">"retryInterval"</span>:<span class="hljs-string">"1"</span>,
+ <span class="hljs-string">"timeout"</span>:{
+ <span class="hljs-string">"strategy"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"interval"</span>:null,
+ <span class="hljs-string">"enable"</span>:<span class="hljs-literal">false</span>
+ },
+ <span class="hljs-string">"taskInstancePriority"</span>:<span class="hljs-string">"MEDIUM"</span>,
+ <span class="hljs-string">"workerGroup"</span>:<span class="hljs-string">"default"</span>,
+ <span class="hljs-string">"preTasks"</span>:[
+
+ ]
+}
+
+</code></pre>
+<h2>SQL Node</h2>
+<p>Perform data query and update operations on the specified datasource through SQL.</p>
+<p><strong>The node data structure is as follows:</strong></p>
+<table>
+<thead>
+<tr>
+<th>No.</th>
+<th>parameter name</th>
+<th></th>
+<th>type</th>
+<th>description</th>
+<th>note</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>1</td>
+<td>id</td>
+<td></td>
+<td>String</td>
+<td>task id</td>
+<td></td>
+</tr>
+<tr>
+<td>2</td>
+<td>type</td>
+<td></td>
+<td>String</td>
+<td>task type</td>
+<td>SQL</td>
+</tr>
+<tr>
+<td>3</td>
+<td>name</td>
+<td></td>
+<td>String</td>
+<td>task name</td>
+<td></td>
+</tr>
+<tr>
+<td>4</td>
+<td>params</td>
+<td></td>
+<td>Object</td>
+<td>customized parameters</td>
+<td>Json format</td>
+</tr>
+<tr>
+<td>5</td>
+<td></td>
+<td>type</td>
+<td>String</td>
+<td>database type</td>
+<td></td>
+</tr>
+<tr>
+<td>6</td>
+<td></td>
+<td>datasource</td>
+<td>Int</td>
+<td>datasource id</td>
+<td></td>
+</tr>
+<tr>
+<td>7</td>
+<td></td>
+<td>sql</td>
+<td>String</td>
+<td>query SQL statement</td>
+<td></td>
+</tr>
+<tr>
+<td>8</td>
+<td></td>
+<td>udfs</td>
+<td>String</td>
+<td>udf functions</td>
+<td>specify UDF function ids, separate by comma</td>
+</tr>
+<tr>
+<td>9</td>
+<td></td>
+<td>sqlType</td>
+<td>String</td>
+<td>SQL node type</td>
+<td>0 for query and 1 for none-query SQL</td>
+</tr>
+<tr>
+<td>10</td>
+<td></td>
+<td>title</td>
+<td>String</td>
+<td>mail title</td>
+<td></td>
+</tr>
+<tr>
+<td>11</td>
+<td></td>
+<td>receivers</td>
+<td>String</td>
+<td>receivers</td>
+<td></td>
+</tr>
+<tr>
+<td>12</td>
+<td></td>
+<td>receiversCc</td>
+<td>String</td>
+<td>CC receivers</td>
+<td></td>
+</tr>
+<tr>
+<td>13</td>
+<td></td>
+<td>showType</td>
+<td>String</td>
+<td>display type of mail</td>
+<td>optionals: TABLE or ATTACHMENT</td>
+</tr>
+<tr>
+<td>14</td>
+<td></td>
+<td>connParams</td>
+<td>String</td>
+<td>connect parameters</td>
+<td></td>
+</tr>
+<tr>
+<td>15</td>
+<td></td>
+<td>preStatements</td>
+<td>Array</td>
+<td>preposition SQL statements</td>
+<td></td>
+</tr>
+<tr>
+<td>16</td>
+<td></td>
+<td>postStatements</td>
+<td>Array</td>
+<td>postposition SQL statements</td>
+<td></td>
+</tr>
+<tr>
+<td>17</td>
+<td></td>
+<td>localParams</td>
+<td>Array</td>
+<td>customized parameters</td>
+<td></td>
+</tr>
+<tr>
+<td>18</td>
+<td>description</td>
+<td></td>
+<td>String</td>
+<td>description</td>
+<td></td>
+</tr>
+<tr>
+<td>19</td>
+<td>runFlag</td>
+<td></td>
+<td>String</td>
+<td>execution flag</td>
+<td></td>
+</tr>
+<tr>
+<td>20</td>
+<td>conditionResult</td>
+<td></td>
+<td>Object</td>
+<td>condition branch</td>
+<td></td>
+</tr>
+<tr>
+<td>21</td>
+<td></td>
+<td>successNode</td>
+<td>Array</td>
+<td>jump to node if success</td>
+<td></td>
+</tr>
+<tr>
+<td>22</td>
+<td></td>
+<td>failedNode</td>
+<td>Array</td>
+<td>jump to node if failure</td>
+<td></td>
+</tr>
+<tr>
+<td>23</td>
+<td>dependence</td>
+<td></td>
+<td>Object</td>
+<td>task dependency</td>
+<td>mutual exclusion with params</td>
+</tr>
+<tr>
+<td>24</td>
+<td>maxRetryTimes</td>
+<td></td>
+<td>String</td>
+<td>max retry times</td>
+<td></td>
+</tr>
+<tr>
+<td>25</td>
+<td>retryInterval</td>
+<td></td>
+<td>String</td>
+<td>retry interval</td>
+<td></td>
+</tr>
+<tr>
+<td>26</td>
+<td>timeout</td>
+<td></td>
+<td>Object</td>
+<td>timeout</td>
+<td></td>
+</tr>
+<tr>
+<td>27</td>
+<td>taskInstancePriority</td>
+<td></td>
+<td>String</td>
+<td>task priority</td>
+<td></td>
+</tr>
+<tr>
+<td>28</td>
+<td>workerGroup</td>
+<td></td>
+<td>String</td>
+<td>Worker group</td>
+<td></td>
+</tr>
+<tr>
+<td>29</td>
+<td>preTasks</td>
+<td></td>
+<td>Array</td>
+<td>preposition tasks</td>
+<td></td>
+</tr>
+</tbody>
+</table>
+<p><strong>Node data example:</strong></p>
+<pre><code class="language-bash">{
+ <span class="hljs-string">"type"</span>:<span class="hljs-string">"SQL"</span>,
+ <span class="hljs-string">"id"</span>:<span class="hljs-string">"tasks-95648"</span>,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"SqlTask-Query"</span>,
+ <span class="hljs-string">"params"</span>:{
+ <span class="hljs-string">"type"</span>:<span class="hljs-string">"MYSQL"</span>,
+ <span class="hljs-string">"datasource"</span>:1,
+ <span class="hljs-string">"sql"</span>:<span class="hljs-string">"select id , namge , age from emp where id = <span class="hljs-variable">${id}</span>"</span>,
+ <span class="hljs-string">"udfs"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"sqlType"</span>:<span class="hljs-string">"0"</span>,
+ <span class="hljs-string">"title"</span>:<span class="hljs-string">"xxxx@xxx.com"</span>,
+ <span class="hljs-string">"receivers"</span>:<span class="hljs-string">"xxxx@xxx.com"</span>,
+ <span class="hljs-string">"receiversCc"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"showType"</span>:<span class="hljs-string">"TABLE"</span>,
+ <span class="hljs-string">"localParams"</span>:[
+ {
+ <span class="hljs-string">"prop"</span>:<span class="hljs-string">"id"</span>,
+ <span class="hljs-string">"direct"</span>:<span class="hljs-string">"IN"</span>,
+ <span class="hljs-string">"type"</span>:<span class="hljs-string">"INTEGER"</span>,
+ <span class="hljs-string">"value"</span>:<span class="hljs-string">"1"</span>
+ }
+ ],
+ <span class="hljs-string">"connParams"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"preStatements"</span>:[
+ <span class="hljs-string">"insert into emp ( id,name ) value (1,'Li' )"</span>
+ ],
+ <span class="hljs-string">"postStatements"</span>:[
+
+ ]
+ },
+ <span class="hljs-string">"description"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"runFlag"</span>:<span class="hljs-string">"NORMAL"</span>,
+ <span class="hljs-string">"conditionResult"</span>:{
+ <span class="hljs-string">"successNode"</span>:[
+ <span class="hljs-string">""</span>
+ ],
+ <span class="hljs-string">"failedNode"</span>:[
+ <span class="hljs-string">""</span>
+ ]
+ },
+ <span class="hljs-string">"dependence"</span>:{
+
+ },
+ <span class="hljs-string">"maxRetryTimes"</span>:<span class="hljs-string">"0"</span>,
+ <span class="hljs-string">"retryInterval"</span>:<span class="hljs-string">"1"</span>,
+ <span class="hljs-string">"timeout"</span>:{
+ <span class="hljs-string">"strategy"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"interval"</span>:null,
+ <span class="hljs-string">"enable"</span>:<span class="hljs-literal">false</span>
+ },
+ <span class="hljs-string">"taskInstancePriority"</span>:<span class="hljs-string">"MEDIUM"</span>,
+ <span class="hljs-string">"workerGroup"</span>:<span class="hljs-string">"default"</span>,
+ <span class="hljs-string">"preTasks"</span>:[
+
+ ]
+}
+</code></pre>
+<h2>PROCEDURE [stored procedures] Node</h2>
+<p><strong>The node data structure is as follows:</strong>
+<strong>Node data example:</strong></p>
+<h2>SPARK Node</h2>
+<p><strong>The node data structure is as follows:</strong></p>
+<table>
+<thead>
+<tr>
+<th>No.</th>
+<th>parameter name</th>
+<th></th>
+<th>type</th>
+<th>description</th>
+<th>notes</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>1</td>
+<td>id</td>
+<td></td>
+<td>String</td>
+<td>task Id</td>
+<td></td>
+</tr>
+<tr>
+<td>2</td>
+<td>type</td>
+<td></td>
+<td>String</td>
+<td>task type</td>
+<td>SPARK</td>
+</tr>
+<tr>
+<td>3</td>
+<td>name</td>
+<td></td>
+<td>String</td>
+<td>task name</td>
+<td></td>
+</tr>
+<tr>
+<td>4</td>
+<td>params</td>
+<td></td>
+<td>Object</td>
+<td>customized parameters</td>
+<td>Json format</td>
+</tr>
+<tr>
+<td>5</td>
+<td></td>
+<td>mainClass</td>
+<td>String</td>
+<td>main class</td>
+<td></td>
+</tr>
+<tr>
+<td>6</td>
+<td></td>
+<td>mainArgs</td>
+<td>String</td>
+<td>execution arguments</td>
+<td></td>
+</tr>
+<tr>
+<td>7</td>
+<td></td>
+<td>others</td>
+<td>String</td>
+<td>other arguments</td>
+<td></td>
+</tr>
+<tr>
+<td>8</td>
+<td></td>
+<td>mainJar</td>
+<td>Object</td>
+<td>application jar package</td>
+<td></td>
+</tr>
+<tr>
+<td>9</td>
+<td></td>
+<td>deployMode</td>
+<td>String</td>
+<td>deployment mode</td>
+<td>local,client,cluster</td>
+</tr>
+<tr>
+<td>10</td>
+<td></td>
+<td>driverCores</td>
+<td>String</td>
+<td>driver cores</td>
+<td></td>
+</tr>
+<tr>
+<td>11</td>
+<td></td>
+<td>driverMemory</td>
+<td>String</td>
+<td>driver memory</td>
+<td></td>
+</tr>
+<tr>
+<td>12</td>
+<td></td>
+<td>numExecutors</td>
+<td>String</td>
+<td>executor count</td>
+<td></td>
+</tr>
+<tr>
+<td>13</td>
+<td></td>
+<td>executorMemory</td>
+<td>String</td>
+<td>executor memory</td>
+<td></td>
+</tr>
+<tr>
+<td>14</td>
+<td></td>
+<td>executorCores</td>
+<td>String</td>
+<td>executor cores</td>
+<td></td>
+</tr>
+<tr>
+<td>15</td>
+<td></td>
+<td>programType</td>
+<td>String</td>
+<td>program type</td>
+<td>JAVA,SCALA,PYTHON</td>
+</tr>
+<tr>
+<td>16</td>
+<td></td>
+<td>sparkVersion</td>
+<td>String</td>
+<td>Spark version</td>
+<td>SPARK1 , SPARK2</td>
+</tr>
+<tr>
+<td>17</td>
+<td></td>
+<td>localParams</td>
+<td>Array</td>
+<td>customized local parameters</td>
+<td></td>
+</tr>
+<tr>
+<td>18</td>
+<td></td>
+<td>resourceList</td>
+<td>Array</td>
+<td>resource files</td>
+<td></td>
+</tr>
+<tr>
+<td>19</td>
+<td>description</td>
+<td></td>
+<td>String</td>
+<td>description</td>
+<td></td>
+</tr>
+<tr>
+<td>20</td>
+<td>runFlag</td>
+<td></td>
+<td>String</td>
+<td>execution flag</td>
+<td></td>
+</tr>
+<tr>
+<td>21</td>
+<td>conditionResult</td>
+<td></td>
+<td>Object</td>
+<td>condition branch</td>
+<td></td>
+</tr>
+<tr>
+<td>22</td>
+<td></td>
+<td>successNode</td>
+<td>Array</td>
+<td>jump to node if success</td>
+<td></td>
+</tr>
+<tr>
+<td>23</td>
+<td></td>
+<td>failedNode</td>
+<td>Array</td>
+<td>jump to node if failure</td>
+<td></td>
+</tr>
+<tr>
+<td>24</td>
+<td>dependence</td>
+<td></td>
+<td>Object</td>
+<td>task dependency</td>
+<td>mutual exclusion with params</td>
+</tr>
+<tr>
+<td>25</td>
+<td>maxRetryTimes</td>
+<td></td>
+<td>String</td>
+<td>max retry times</td>
+<td></td>
+</tr>
+<tr>
+<td>26</td>
+<td>retryInterval</td>
+<td></td>
+<td>String</td>
+<td>retry interval</td>
+<td></td>
+</tr>
+<tr>
+<td>27</td>
+<td>timeout</td>
+<td></td>
+<td>Object</td>
+<td>timeout</td>
+<td></td>
+</tr>
+<tr>
+<td>28</td>
+<td>taskInstancePriority</td>
+<td></td>
+<td>String</td>
+<td>task priority</td>
+<td></td>
+</tr>
+<tr>
+<td>29</td>
+<td>workerGroup</td>
+<td></td>
+<td>String</td>
+<td>Worker group</td>
+<td></td>
+</tr>
+<tr>
+<td>30</td>
+<td>preTasks</td>
+<td></td>
+<td>Array</td>
+<td>preposition tasks</td>
+<td></td>
+</tr>
+</tbody>
+</table>
+<p><strong>Node data example:</strong></p>
+<pre><code class="language-bash">{
+ <span class="hljs-string">"type"</span>:<span class="hljs-string">"SPARK"</span>,
+ <span class="hljs-string">"id"</span>:<span class="hljs-string">"tasks-87430"</span>,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"SparkTask"</span>,
+ <span class="hljs-string">"params"</span>:{
+ <span class="hljs-string">"mainClass"</span>:<span class="hljs-string">"org.apache.spark.examples.SparkPi"</span>,
+ <span class="hljs-string">"mainJar"</span>:{
+ <span class="hljs-string">"id"</span>:4
+ },
+ <span class="hljs-string">"deployMode"</span>:<span class="hljs-string">"cluster"</span>,
+ <span class="hljs-string">"resourceList"</span>:[
+ {
+ <span class="hljs-string">"id"</span>:3,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"run.sh"</span>,
+ <span class="hljs-string">"res"</span>:<span class="hljs-string">"run.sh"</span>
+ }
+ ],
+ <span class="hljs-string">"localParams"</span>:[
+
+ ],
+ <span class="hljs-string">"driverCores"</span>:1,
+ <span class="hljs-string">"driverMemory"</span>:<span class="hljs-string">"512M"</span>,
+ <span class="hljs-string">"numExecutors"</span>:2,
+ <span class="hljs-string">"executorMemory"</span>:<span class="hljs-string">"2G"</span>,
+ <span class="hljs-string">"executorCores"</span>:2,
+ <span class="hljs-string">"mainArgs"</span>:<span class="hljs-string">"10"</span>,
+ <span class="hljs-string">"others"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"programType"</span>:<span class="hljs-string">"SCALA"</span>,
+ <span class="hljs-string">"sparkVersion"</span>:<span class="hljs-string">"SPARK2"</span>
+ },
+ <span class="hljs-string">"description"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"runFlag"</span>:<span class="hljs-string">"NORMAL"</span>,
+ <span class="hljs-string">"conditionResult"</span>:{
+ <span class="hljs-string">"successNode"</span>:[
+ <span class="hljs-string">""</span>
+ ],
+ <span class="hljs-string">"failedNode"</span>:[
+ <span class="hljs-string">""</span>
+ ]
+ },
+ <span class="hljs-string">"dependence"</span>:{
+
+ },
+ <span class="hljs-string">"maxRetryTimes"</span>:<span class="hljs-string">"0"</span>,
+ <span class="hljs-string">"retryInterval"</span>:<span class="hljs-string">"1"</span>,
+ <span class="hljs-string">"timeout"</span>:{
+ <span class="hljs-string">"strategy"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"interval"</span>:null,
+ <span class="hljs-string">"enable"</span>:<span class="hljs-literal">false</span>
+ },
+ <span class="hljs-string">"taskInstancePriority"</span>:<span class="hljs-string">"MEDIUM"</span>,
+ <span class="hljs-string">"workerGroup"</span>:<span class="hljs-string">"default"</span>,
+ <span class="hljs-string">"preTasks"</span>:[
+
+ ]
+}
+</code></pre>
+<h2>MapReduce(MR) Node</h2>
+<p><strong>The node data structure is as follows:</strong></p>
+<table>
+<thead>
+<tr>
+<th>No.</th>
+<th>parameter name</th>
+<th></th>
+<th>type</th>
+<th>description</th>
+<th>notes</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>1</td>
+<td>id</td>
+<td></td>
+<td>String</td>
+<td>task Id</td>
+<td></td>
+</tr>
+<tr>
+<td>2</td>
+<td>type</td>
+<td></td>
+<td>String</td>
+<td>task type</td>
+<td>MR</td>
+</tr>
+<tr>
+<td>3</td>
+<td>name</td>
+<td></td>
+<td>String</td>
+<td>task name</td>
+<td></td>
+</tr>
+<tr>
+<td>4</td>
+<td>params</td>
+<td></td>
+<td>Object</td>
+<td>customized parameters</td>
+<td>Json format</td>
+</tr>
+<tr>
+<td>5</td>
+<td></td>
+<td>mainClass</td>
+<td>String</td>
+<td>main class</td>
+<td></td>
+</tr>
+<tr>
+<td>6</td>
+<td></td>
+<td>mainArgs</td>
+<td>String</td>
+<td>execution arguments</td>
+<td></td>
+</tr>
+<tr>
+<td>7</td>
+<td></td>
+<td>others</td>
+<td>String</td>
+<td>other arguments</td>
+<td></td>
+</tr>
+<tr>
+<td>8</td>
+<td></td>
+<td>mainJar</td>
+<td>Object</td>
+<td>application jar package</td>
+<td></td>
+</tr>
+<tr>
+<td>9</td>
+<td></td>
+<td>programType</td>
+<td>String</td>
+<td>program type</td>
+<td>JAVA,PYTHON</td>
+</tr>
+<tr>
+<td>10</td>
+<td></td>
+<td>localParams</td>
+<td>Array</td>
+<td>customized local parameters</td>
+<td></td>
+</tr>
+<tr>
+<td>11</td>
+<td></td>
+<td>resourceList</td>
+<td>Array</td>
+<td>resource files</td>
+<td></td>
+</tr>
+<tr>
+<td>12</td>
+<td>description</td>
+<td></td>
+<td>String</td>
+<td>description</td>
+<td></td>
+</tr>
+<tr>
+<td>13</td>
+<td>runFlag</td>
+<td></td>
+<td>String</td>
+<td>execution flag</td>
+<td></td>
+</tr>
+<tr>
+<td>14</td>
+<td>conditionResult</td>
+<td></td>
+<td>Object</td>
+<td>condition branch</td>
+<td></td>
+</tr>
+<tr>
+<td>15</td>
+<td></td>
+<td>successNode</td>
+<td>Array</td>
+<td>jump to node if success</td>
+<td></td>
+</tr>
+<tr>
+<td>16</td>
+<td></td>
+<td>failedNode</td>
+<td>Array</td>
+<td>jump to node if failure</td>
+<td></td>
+</tr>
+<tr>
+<td>17</td>
+<td>dependence</td>
+<td></td>
+<td>Object</td>
+<td>task dependency</td>
+<td>mutual exclusion with params</td>
+</tr>
+<tr>
+<td>18</td>
+<td>maxRetryTimes</td>
+<td></td>
+<td>String</td>
+<td>max retry times</td>
+<td></td>
+</tr>
+<tr>
+<td>19</td>
+<td>retryInterval</td>
+<td></td>
+<td>String</td>
+<td>retry interval</td>
+<td></td>
+</tr>
+<tr>
+<td>20</td>
+<td>timeout</td>
+<td></td>
+<td>Object</td>
+<td>timeout</td>
+<td></td>
+</tr>
+<tr>
+<td>21</td>
+<td>taskInstancePriority</td>
+<td></td>
+<td>String</td>
+<td>task priority</td>
+<td></td>
+</tr>
+<tr>
+<td>22</td>
+<td>workerGroup</td>
+<td></td>
+<td>String</td>
+<td>Worker group</td>
+<td></td>
+</tr>
+<tr>
+<td>23</td>
+<td>preTasks</td>
+<td></td>
+<td>Array</td>
+<td>preposition tasks</td>
+<td></td>
+</tr>
+</tbody>
+</table>
+<p><strong>Node data example:</strong></p>
+<pre><code class="language-bash">{
+ <span class="hljs-string">"type"</span>:<span class="hljs-string">"MR"</span>,
+ <span class="hljs-string">"id"</span>:<span class="hljs-string">"tasks-28997"</span>,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"MRTask"</span>,
+ <span class="hljs-string">"params"</span>:{
+ <span class="hljs-string">"mainClass"</span>:<span class="hljs-string">"wordcount"</span>,
+ <span class="hljs-string">"mainJar"</span>:{
+ <span class="hljs-string">"id"</span>:5
+ },
+ <span class="hljs-string">"resourceList"</span>:[
+ {
+ <span class="hljs-string">"id"</span>:3,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"run.sh"</span>,
+ <span class="hljs-string">"res"</span>:<span class="hljs-string">"run.sh"</span>
+ }
+ ],
+ <span class="hljs-string">"localParams"</span>:[
+
+ ],
+ <span class="hljs-string">"mainArgs"</span>:<span class="hljs-string">"/tmp/wordcount/input /tmp/wordcount/output/"</span>,
+ <span class="hljs-string">"others"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"programType"</span>:<span class="hljs-string">"JAVA"</span>
+ },
+ <span class="hljs-string">"description"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"runFlag"</span>:<span class="hljs-string">"NORMAL"</span>,
+ <span class="hljs-string">"conditionResult"</span>:{
+ <span class="hljs-string">"successNode"</span>:[
+ <span class="hljs-string">""</span>
+ ],
+ <span class="hljs-string">"failedNode"</span>:[
+ <span class="hljs-string">""</span>
+ ]
+ },
+ <span class="hljs-string">"dependence"</span>:{
+
+ },
+ <span class="hljs-string">"maxRetryTimes"</span>:<span class="hljs-string">"0"</span>,
+ <span class="hljs-string">"retryInterval"</span>:<span class="hljs-string">"1"</span>,
+ <span class="hljs-string">"timeout"</span>:{
+ <span class="hljs-string">"strategy"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"interval"</span>:null,
+ <span class="hljs-string">"enable"</span>:<span class="hljs-literal">false</span>
+ },
+ <span class="hljs-string">"taskInstancePriority"</span>:<span class="hljs-string">"MEDIUM"</span>,
+ <span class="hljs-string">"workerGroup"</span>:<span class="hljs-string">"default"</span>,
+ <span class="hljs-string">"preTasks"</span>:[
+
+ ]
+}
+</code></pre>
+<h2>Python Node</h2>
+<p><strong>The node data structure is as follows:</strong></p>
+<table>
+<thead>
+<tr>
+<th>No.</th>
+<th>parameter name</th>
+<th></th>
+<th>type</th>
+<th>description</th>
+<th>notes</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>1</td>
+<td>id</td>
+<td></td>
+<td>String</td>
+<td>task Id</td>
+<td></td>
+</tr>
+<tr>
+<td>2</td>
+<td>type</td>
+<td></td>
+<td>String</td>
+<td>task type</td>
+<td>PYTHON</td>
+</tr>
+<tr>
+<td>3</td>
+<td>name</td>
+<td></td>
+<td>String</td>
+<td>task name</td>
+<td></td>
+</tr>
+<tr>
+<td>4</td>
+<td>params</td>
+<td></td>
+<td>Object</td>
+<td>customized parameters</td>
+<td>Json format</td>
+</tr>
+<tr>
+<td>5</td>
+<td></td>
+<td>rawScript</td>
+<td>String</td>
+<td>Python script</td>
+<td></td>
+</tr>
+<tr>
+<td>6</td>
+<td></td>
+<td>localParams</td>
+<td>Array</td>
+<td>customized local parameters</td>
+<td></td>
+</tr>
+<tr>
+<td>7</td>
+<td></td>
+<td>resourceList</td>
+<td>Array</td>
+<td>resource files</td>
+<td></td>
+</tr>
+<tr>
+<td>8</td>
+<td>description</td>
+<td></td>
+<td>String</td>
+<td>description</td>
+<td></td>
+</tr>
+<tr>
+<td>9</td>
+<td>runFlag</td>
+<td></td>
+<td>String</td>
+<td>execution flag</td>
+<td></td>
+</tr>
+<tr>
+<td>10</td>
+<td>conditionResult</td>
+<td></td>
+<td>Object</td>
+<td>condition branch</td>
+<td></td>
+</tr>
+<tr>
+<td>11</td>
+<td></td>
+<td>successNode</td>
+<td>Array</td>
+<td>jump to node if success</td>
+<td></td>
+</tr>
+<tr>
+<td>12</td>
+<td></td>
+<td>failedNode</td>
+<td>Array</td>
+<td>jump to node if failure</td>
+<td></td>
+</tr>
+<tr>
+<td>13</td>
+<td>dependence</td>
+<td></td>
+<td>Object</td>
+<td>task dependency</td>
+<td>mutual exclusion with params</td>
+</tr>
+<tr>
+<td>14</td>
+<td>maxRetryTimes</td>
+<td></td>
+<td>String</td>
+<td>max retry times</td>
+<td></td>
+</tr>
+<tr>
+<td>15</td>
+<td>retryInterval</td>
+<td></td>
+<td>String</td>
+<td>retry interval</td>
+<td></td>
+</tr>
+<tr>
+<td>16</td>
+<td>timeout</td>
+<td></td>
+<td>Object</td>
+<td>timeout</td>
+<td></td>
+</tr>
+<tr>
+<td>17</td>
+<td>taskInstancePriority</td>
+<td></td>
+<td>String</td>
+<td>task priority</td>
+<td></td>
+</tr>
+<tr>
+<td>18</td>
+<td>workerGroup</td>
+<td></td>
+<td>String</td>
+<td>Worker group</td>
+<td></td>
+</tr>
+<tr>
+<td>19</td>
+<td>preTasks</td>
+<td></td>
+<td>Array</td>
+<td>preposition tasks</td>
+<td></td>
+</tr>
+</tbody>
+</table>
+<p><strong>Node data example:</strong></p>
+<pre><code class="language-bash">{
+ <span class="hljs-string">"type"</span>:<span class="hljs-string">"PYTHON"</span>,
+ <span class="hljs-string">"id"</span>:<span class="hljs-string">"tasks-5463"</span>,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"Python Task"</span>,
+ <span class="hljs-string">"params"</span>:{
+ <span class="hljs-string">"resourceList"</span>:[
+ {
+ <span class="hljs-string">"id"</span>:3,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"run.sh"</span>,
+ <span class="hljs-string">"res"</span>:<span class="hljs-string">"run.sh"</span>
+ }
+ ],
+ <span class="hljs-string">"localParams"</span>:[
+
+ ],
+ <span class="hljs-string">"rawScript"</span>:<span class="hljs-string">"print("</span>This is a python script<span class="hljs-string">")"</span>
+ },
+ <span class="hljs-string">"description"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"runFlag"</span>:<span class="hljs-string">"NORMAL"</span>,
+ <span class="hljs-string">"conditionResult"</span>:{
+ <span class="hljs-string">"successNode"</span>:[
+ <span class="hljs-string">""</span>
+ ],
+ <span class="hljs-string">"failedNode"</span>:[
+ <span class="hljs-string">""</span>
+ ]
+ },
+ <span class="hljs-string">"dependence"</span>:{
+
+ },
+ <span class="hljs-string">"maxRetryTimes"</span>:<span class="hljs-string">"0"</span>,
+ <span class="hljs-string">"retryInterval"</span>:<span class="hljs-string">"1"</span>,
+ <span class="hljs-string">"timeout"</span>:{
+ <span class="hljs-string">"strategy"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"interval"</span>:null,
+ <span class="hljs-string">"enable"</span>:<span class="hljs-literal">false</span>
+ },
+ <span class="hljs-string">"taskInstancePriority"</span>:<span class="hljs-string">"MEDIUM"</span>,
+ <span class="hljs-string">"workerGroup"</span>:<span class="hljs-string">"default"</span>,
+ <span class="hljs-string">"preTasks"</span>:[
+
+ ]
+}
+</code></pre>
+<h2>Flink Node</h2>
+<p><strong>The node data structure is as follows:</strong></p>
+<table>
+<thead>
+<tr>
+<th>No.</th>
+<th>parameter name</th>
+<th></th>
+<th>type</th>
+<th>description</th>
+<th>notes</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>1</td>
+<td>id</td>
+<td></td>
+<td>String</td>
+<td>task Id</td>
+<td></td>
+</tr>
+<tr>
+<td>2</td>
+<td>type</td>
+<td></td>
+<td>String</td>
+<td>task type</td>
+<td>FLINK</td>
+</tr>
+<tr>
+<td>3</td>
+<td>name</td>
+<td></td>
+<td>String</td>
+<td>task name</td>
+<td></td>
+</tr>
+<tr>
+<td>4</td>
+<td>params</td>
+<td></td>
+<td>Object</td>
+<td>customized parameters</td>
+<td>Json format</td>
+</tr>
+<tr>
+<td>5</td>
+<td></td>
+<td>mainClass</td>
+<td>String</td>
+<td>main class</td>
+<td></td>
+</tr>
+<tr>
+<td>6</td>
+<td></td>
+<td>mainArgs</td>
+<td>String</td>
+<td>execution arguments</td>
+<td></td>
+</tr>
+<tr>
+<td>7</td>
+<td></td>
+<td>others</td>
+<td>String</td>
+<td>other arguments</td>
+<td></td>
+</tr>
+<tr>
+<td>8</td>
+<td></td>
+<td>mainJar</td>
+<td>Object</td>
+<td>application jar package</td>
+<td></td>
+</tr>
+<tr>
+<td>9</td>
+<td></td>
+<td>deployMode</td>
+<td>String</td>
+<td>deployment mode</td>
+<td>local,client,cluster</td>
+</tr>
+<tr>
+<td>10</td>
+<td></td>
+<td>slot</td>
+<td>String</td>
+<td>slot count</td>
+<td></td>
+</tr>
+<tr>
+<td>11</td>
+<td></td>
+<td>taskManager</td>
+<td>String</td>
+<td>taskManager count</td>
+<td></td>
+</tr>
+<tr>
+<td>12</td>
+<td></td>
+<td>taskManagerMemory</td>
+<td>String</td>
+<td>taskManager memory size</td>
+<td></td>
+</tr>
+<tr>
+<td>13</td>
+<td></td>
+<td>jobManagerMemory</td>
+<td>String</td>
+<td>jobManager memory size</td>
+<td></td>
+</tr>
+<tr>
+<td>14</td>
+<td></td>
+<td>programType</td>
+<td>String</td>
+<td>program type</td>
+<td>JAVA,SCALA,PYTHON</td>
+</tr>
+<tr>
+<td>15</td>
+<td></td>
+<td>localParams</td>
+<td>Array</td>
+<td>local parameters</td>
+<td></td>
+</tr>
+<tr>
+<td>16</td>
+<td></td>
+<td>resourceList</td>
+<td>Array</td>
+<td>resource files</td>
+<td></td>
+</tr>
+<tr>
+<td>17</td>
+<td>description</td>
+<td></td>
+<td>String</td>
+<td>description</td>
+<td></td>
+</tr>
+<tr>
+<td>18</td>
+<td>runFlag</td>
+<td></td>
+<td>String</td>
+<td>execution flag</td>
+<td></td>
+</tr>
+<tr>
+<td>19</td>
+<td>conditionResult</td>
+<td></td>
+<td>Object</td>
+<td>condition branch</td>
+<td></td>
+</tr>
+<tr>
+<td>20</td>
+<td></td>
+<td>successNode</td>
+<td>Array</td>
+<td>jump node if success</td>
+<td></td>
+</tr>
+<tr>
+<td>21</td>
+<td></td>
+<td>failedNode</td>
+<td>Array</td>
+<td>jump node if failure</td>
+<td></td>
+</tr>
+<tr>
+<td>22</td>
+<td>dependence</td>
+<td></td>
+<td>Object</td>
+<td>task dependency</td>
+<td>mutual exclusion with params</td>
+</tr>
+<tr>
+<td>23</td>
+<td>maxRetryTimes</td>
+<td></td>
+<td>String</td>
+<td>max retry times</td>
+<td></td>
+</tr>
+<tr>
+<td>24</td>
+<td>retryInterval</td>
+<td></td>
+<td>String</td>
+<td>retry interval</td>
+<td></td>
+</tr>
+<tr>
+<td>25</td>
+<td>timeout</td>
+<td></td>
+<td>Object</td>
+<td>timeout</td>
+<td></td>
+</tr>
+<tr>
+<td>26</td>
+<td>taskInstancePriority</td>
+<td></td>
+<td>String</td>
+<td>task priority</td>
+<td></td>
+</tr>
+<tr>
+<td>27</td>
+<td>workerGroup</td>
+<td></td>
+<td>String</td>
+<td>Worker group</td>
+<td></td>
+</tr>
+<tr>
+<td>38</td>
+<td>preTasks</td>
+<td></td>
+<td>Array</td>
+<td>preposition tasks</td>
+<td></td>
+</tr>
+</tbody>
+</table>
+<p><strong>Node data example:</strong></p>
+<pre><code class="language-bash">{
+ <span class="hljs-string">"type"</span>:<span class="hljs-string">"FLINK"</span>,
+ <span class="hljs-string">"id"</span>:<span class="hljs-string">"tasks-17135"</span>,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"FlinkTask"</span>,
+ <span class="hljs-string">"params"</span>:{
+ <span class="hljs-string">"mainClass"</span>:<span class="hljs-string">"com.flink.demo"</span>,
+ <span class="hljs-string">"mainJar"</span>:{
+ <span class="hljs-string">"id"</span>:6
+ },
+ <span class="hljs-string">"deployMode"</span>:<span class="hljs-string">"cluster"</span>,
+ <span class="hljs-string">"resourceList"</span>:[
+ {
+ <span class="hljs-string">"id"</span>:3,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"run.sh"</span>,
+ <span class="hljs-string">"res"</span>:<span class="hljs-string">"run.sh"</span>
+ }
+ ],
+ <span class="hljs-string">"localParams"</span>:[
+
+ ],
+ <span class="hljs-string">"slot"</span>:1,
+ <span class="hljs-string">"taskManager"</span>:<span class="hljs-string">"2"</span>,
+ <span class="hljs-string">"jobManagerMemory"</span>:<span class="hljs-string">"1G"</span>,
+ <span class="hljs-string">"taskManagerMemory"</span>:<span class="hljs-string">"2G"</span>,
+ <span class="hljs-string">"executorCores"</span>:2,
+ <span class="hljs-string">"mainArgs"</span>:<span class="hljs-string">"100"</span>,
+ <span class="hljs-string">"others"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"programType"</span>:<span class="hljs-string">"SCALA"</span>
+ },
+ <span class="hljs-string">"description"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"runFlag"</span>:<span class="hljs-string">"NORMAL"</span>,
+ <span class="hljs-string">"conditionResult"</span>:{
+ <span class="hljs-string">"successNode"</span>:[
+ <span class="hljs-string">""</span>
+ ],
+ <span class="hljs-string">"failedNode"</span>:[
+ <span class="hljs-string">""</span>
+ ]
+ },
+ <span class="hljs-string">"dependence"</span>:{
+
+ },
+ <span class="hljs-string">"maxRetryTimes"</span>:<span class="hljs-string">"0"</span>,
+ <span class="hljs-string">"retryInterval"</span>:<span class="hljs-string">"1"</span>,
+ <span class="hljs-string">"timeout"</span>:{
+ <span class="hljs-string">"strategy"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"interval"</span>:null,
+ <span class="hljs-string">"enable"</span>:<span class="hljs-literal">false</span>
+ },
+ <span class="hljs-string">"taskInstancePriority"</span>:<span class="hljs-string">"MEDIUM"</span>,
+ <span class="hljs-string">"workerGroup"</span>:<span class="hljs-string">"default"</span>,
+ <span class="hljs-string">"preTasks"</span>:[
+
+ ]
+}
+</code></pre>
+<h2>HTTP Node</h2>
+<p><strong>The node data structure is as follows:</strong></p>
+<table>
+<thead>
+<tr>
+<th>No.</th>
+<th>parameter name</th>
+<th></th>
+<th>type</th>
+<th>description</th>
+<th>notes</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>1</td>
+<td>id</td>
+<td></td>
+<td>String</td>
+<td>task Id</td>
+<td></td>
+</tr>
+<tr>
+<td>2</td>
+<td>type</td>
+<td></td>
+<td>String</td>
+<td>task type</td>
+<td>HTTP</td>
+</tr>
+<tr>
+<td>3</td>
+<td>name</td>
+<td></td>
+<td>String</td>
+<td>task name</td>
+<td></td>
+</tr>
+<tr>
+<td>4</td>
+<td>params</td>
+<td></td>
+<td>Object</td>
+<td>customized parameters</td>
+<td>Json format</td>
+</tr>
+<tr>
+<td>5</td>
+<td></td>
+<td>url</td>
+<td>String</td>
+<td>request url</td>
+<td></td>
+</tr>
+<tr>
+<td>6</td>
+<td></td>
+<td>httpMethod</td>
+<td>String</td>
+<td>http method</td>
+<td>GET,POST,HEAD,PUT,DELETE</td>
+</tr>
+<tr>
+<td>7</td>
+<td></td>
+<td>httpParams</td>
+<td>Array</td>
+<td>http parameters</td>
+<td></td>
+</tr>
+<tr>
+<td>8</td>
+<td></td>
+<td>httpCheckCondition</td>
+<td>String</td>
+<td>validation of HTTP code status</td>
+<td>default code 200</td>
+</tr>
+<tr>
+<td>9</td>
+<td></td>
+<td>condition</td>
+<td>String</td>
+<td>validation conditions</td>
+<td></td>
+</tr>
+<tr>
+<td>10</td>
+<td></td>
+<td>localParams</td>
+<td>Array</td>
+<td>customized local parameters</td>
+<td></td>
+</tr>
+<tr>
+<td>11</td>
+<td>description</td>
+<td></td>
+<td>String</td>
+<td>description</td>
+<td></td>
+</tr>
+<tr>
+<td>12</td>
+<td>runFlag</td>
+<td></td>
+<td>String</td>
+<td>execution flag</td>
+<td></td>
+</tr>
+<tr>
+<td>13</td>
+<td>conditionResult</td>
+<td></td>
+<td>Object</td>
+<td>condition branch</td>
+<td></td>
+</tr>
+<tr>
+<td>14</td>
+<td></td>
+<td>successNode</td>
+<td>Array</td>
+<td>jump node if success</td>
+<td></td>
+</tr>
+<tr>
+<td>15</td>
+<td></td>
+<td>failedNode</td>
+<td>Array</td>
+<td>jump node if failure</td>
+<td></td>
+</tr>
+<tr>
+<td>16</td>
+<td>dependence</td>
+<td></td>
+<td>Object</td>
+<td>task dependency</td>
+<td>mutual exclusion with params</td>
+</tr>
+<tr>
+<td>17</td>
+<td>maxRetryTimes</td>
+<td></td>
+<td>String</td>
+<td>max retry times</td>
+<td></td>
+</tr>
+<tr>
+<td>18</td>
+<td>retryInterval</td>
+<td></td>
+<td>String</td>
+<td>retry interval</td>
+<td></td>
+</tr>
+<tr>
+<td>19</td>
+<td>timeout</td>
+<td></td>
+<td>Object</td>
+<td>timeout</td>
+<td></td>
+</tr>
+<tr>
+<td>20</td>
+<td>taskInstancePriority</td>
+<td></td>
+<td>String</td>
+<td>task priority</td>
+<td></td>
+</tr>
+<tr>
+<td>21</td>
+<td>workerGroup</td>
+<td></td>
+<td>String</td>
+<td>Worker group</td>
+<td></td>
+</tr>
+<tr>
+<td>22</td>
+<td>preTasks</td>
+<td></td>
+<td>Array</td>
+<td>preposition tasks</td>
+<td></td>
+</tr>
+</tbody>
+</table>
+<p><strong>Node data example:</strong></p>
+<pre><code class="language-bash">{
+ <span class="hljs-string">"type"</span>:<span class="hljs-string">"HTTP"</span>,
+ <span class="hljs-string">"id"</span>:<span class="hljs-string">"tasks-60499"</span>,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"HttpTask"</span>,
+ <span class="hljs-string">"params"</span>:{
+ <span class="hljs-string">"localParams"</span>:[
+
+ ],
+ <span class="hljs-string">"httpParams"</span>:[
+ {
+ <span class="hljs-string">"prop"</span>:<span class="hljs-string">"id"</span>,
+ <span class="hljs-string">"httpParametersType"</span>:<span class="hljs-string">"PARAMETER"</span>,
+ <span class="hljs-string">"value"</span>:<span class="hljs-string">"1"</span>
+ },
+ {
+ <span class="hljs-string">"prop"</span>:<span class="hljs-string">"name"</span>,
+ <span class="hljs-string">"httpParametersType"</span>:<span class="hljs-string">"PARAMETER"</span>,
+ <span class="hljs-string">"value"</span>:<span class="hljs-string">"Bo"</span>
+ }
+ ],
+ <span class="hljs-string">"url"</span>:<span class="hljs-string">"https://www.xxxxx.com:9012"</span>,
+ <span class="hljs-string">"httpMethod"</span>:<span class="hljs-string">"POST"</span>,
+ <span class="hljs-string">"httpCheckCondition"</span>:<span class="hljs-string">"STATUS_CODE_DEFAULT"</span>,
+ <span class="hljs-string">"condition"</span>:<span class="hljs-string">""</span>
+ },
+ <span class="hljs-string">"description"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"runFlag"</span>:<span class="hljs-string">"NORMAL"</span>,
+ <span class="hljs-string">"conditionResult"</span>:{
+ <span class="hljs-string">"successNode"</span>:[
+ <span class="hljs-string">""</span>
+ ],
+ <span class="hljs-string">"failedNode"</span>:[
+ <span class="hljs-string">""</span>
+ ]
+ },
+ <span class="hljs-string">"dependence"</span>:{
+
+ },
+ <span class="hljs-string">"maxRetryTimes"</span>:<span class="hljs-string">"0"</span>,
+ <span class="hljs-string">"retryInterval"</span>:<span class="hljs-string">"1"</span>,
+ <span class="hljs-string">"timeout"</span>:{
+ <span class="hljs-string">"strategy"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"interval"</span>:null,
+ <span class="hljs-string">"enable"</span>:<span class="hljs-literal">false</span>
+ },
+ <span class="hljs-string">"taskInstancePriority"</span>:<span class="hljs-string">"MEDIUM"</span>,
+ <span class="hljs-string">"workerGroup"</span>:<span class="hljs-string">"default"</span>,
+ <span class="hljs-string">"preTasks"</span>:[
+
+ ]
+}
+</code></pre>
+<h2>DataX Node</h2>
+<p><strong>The node data structure is as follows:</strong></p>
+<table>
+<thead>
+<tr>
+<th>No.</th>
+<th>parameter name</th>
+<th></th>
+<th>type</th>
+<th>description</th>
+<th>notes</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>1</td>
+<td>id</td>
+<td></td>
+<td>String</td>
+<td>task Id</td>
+<td></td>
+</tr>
+<tr>
+<td>2</td>
+<td>type</td>
+<td></td>
+<td>String</td>
+<td>task type</td>
+<td>DATAX</td>
+</tr>
+<tr>
+<td>3</td>
+<td>name</td>
+<td></td>
+<td>String</td>
+<td>task name</td>
+<td></td>
+</tr>
+<tr>
+<td>4</td>
+<td>params</td>
+<td></td>
+<td>Object</td>
+<td>customized parameters</td>
+<td>Json format</td>
+</tr>
+<tr>
+<td>5</td>
+<td></td>
+<td>customConfig</td>
+<td>Int</td>
+<td>specify whether use customized config</td>
+<td>0 none customized, 1 customized</td>
+</tr>
+<tr>
+<td>6</td>
+<td></td>
+<td>dsType</td>
+<td>String</td>
+<td>datasource type</td>
+<td></td>
+</tr>
+<tr>
+<td>7</td>
+<td></td>
+<td>dataSource</td>
+<td>Int</td>
+<td>datasource ID</td>
+<td></td>
+</tr>
+<tr>
+<td>8</td>
+<td></td>
+<td>dtType</td>
+<td>String</td>
+<td>target database type</td>
+<td></td>
+</tr>
+<tr>
+<td>9</td>
+<td></td>
+<td>dataTarget</td>
+<td>Int</td>
+<td>target database ID</td>
+<td></td>
+</tr>
+<tr>
+<td>10</td>
+<td></td>
+<td>sql</td>
+<td>String</td>
+<td>SQL statements</td>
+<td></td>
+</tr>
+<tr>
+<td>11</td>
+<td></td>
+<td>targetTable</td>
+<td>String</td>
+<td>target table</td>
+<td></td>
+</tr>
+<tr>
+<td>12</td>
+<td></td>
+<td>jobSpeedByte</td>
+<td>Int</td>
+<td>job speed limiting(bytes)</td>
+<td></td>
+</tr>
+<tr>
+<td>13</td>
+<td></td>
+<td>jobSpeedRecord</td>
+<td>Int</td>
+<td>job speed limiting(records)</td>
+<td></td>
+</tr>
+<tr>
+<td>14</td>
+<td></td>
+<td>preStatements</td>
+<td>Array</td>
+<td>preposition SQL</td>
+<td></td>
+</tr>
+<tr>
+<td>15</td>
+<td></td>
+<td>postStatements</td>
+<td>Array</td>
+<td>postposition SQL</td>
+<td></td>
+</tr>
+<tr>
+<td>16</td>
+<td></td>
+<td>json</td>
+<td>String</td>
+<td>customized configs</td>
+<td>valid if customConfig=1</td>
+</tr>
+<tr>
+<td>17</td>
+<td></td>
+<td>localParams</td>
+<td>Array</td>
+<td>customized parameters</td>
+<td>valid if customConfig=1</td>
+</tr>
+<tr>
+<td>18</td>
+<td>description</td>
+<td></td>
+<td>String</td>
+<td>description</td>
+<td></td>
+</tr>
+<tr>
+<td>19</td>
+<td>runFlag</td>
+<td></td>
+<td>String</td>
+<td>execution flag</td>
+<td></td>
+</tr>
+<tr>
+<td>20</td>
+<td>conditionResult</td>
+<td></td>
+<td>Object</td>
+<td>condition branch</td>
+<td></td>
+</tr>
+<tr>
+<td>21</td>
+<td></td>
+<td>successNode</td>
+<td>Array</td>
+<td>jump node if success</td>
+<td></td>
+</tr>
+<tr>
+<td>22</td>
+<td></td>
+<td>failedNode</td>
+<td>Array</td>
+<td>jump node if failure</td>
+<td></td>
+</tr>
+<tr>
+<td>23</td>
+<td>dependence</td>
+<td></td>
+<td>Object</td>
+<td>task dependency</td>
+<td>mutual exclusion with params</td>
+</tr>
+<tr>
+<td>24</td>
+<td>maxRetryTimes</td>
+<td></td>
+<td>String</td>
+<td>max retry times</td>
+<td></td>
+</tr>
+<tr>
+<td>25</td>
+<td>retryInterval</td>
+<td></td>
+<td>String</td>
+<td>retry interval</td>
+<td></td>
+</tr>
+<tr>
+<td>26</td>
+<td>timeout</td>
+<td></td>
+<td>Object</td>
+<td>timeout</td>
+<td></td>
+</tr>
+<tr>
+<td>27</td>
+<td>taskInstancePriority</td>
+<td></td>
+<td>String</td>
+<td>task priority</td>
+<td></td>
+</tr>
+<tr>
+<td>28</td>
+<td>workerGroup</td>
+<td></td>
+<td>String</td>
+<td>Worker group</td>
+<td></td>
+</tr>
+<tr>
+<td>29</td>
+<td>preTasks</td>
+<td></td>
+<td>Array</td>
+<td>preposition tasks</td>
+<td></td>
+</tr>
+</tbody>
+</table>
+<p><strong>Node data example:</strong></p>
+<pre><code class="language-bash">{
+ <span class="hljs-string">"type"</span>:<span class="hljs-string">"DATAX"</span>,
+ <span class="hljs-string">"id"</span>:<span class="hljs-string">"tasks-91196"</span>,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"DataxTask-DB"</span>,
+ <span class="hljs-string">"params"</span>:{
+ <span class="hljs-string">"customConfig"</span>:0,
+ <span class="hljs-string">"dsType"</span>:<span class="hljs-string">"MYSQL"</span>,
+ <span class="hljs-string">"dataSource"</span>:1,
+ <span class="hljs-string">"dtType"</span>:<span class="hljs-string">"MYSQL"</span>,
+ <span class="hljs-string">"dataTarget"</span>:1,
+ <span class="hljs-string">"sql"</span>:<span class="hljs-string">"select id, name ,age from user "</span>,
+ <span class="hljs-string">"targetTable"</span>:<span class="hljs-string">"emp"</span>,
+ <span class="hljs-string">"jobSpeedByte"</span>:524288,
+ <span class="hljs-string">"jobSpeedRecord"</span>:500,
+ <span class="hljs-string">"preStatements"</span>:[
+ <span class="hljs-string">"truncate table emp "</span>
+ ],
+ <span class="hljs-string">"postStatements"</span>:[
+ <span class="hljs-string">"truncate table user"</span>
+ ]
+ },
+ <span class="hljs-string">"description"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"runFlag"</span>:<span class="hljs-string">"NORMAL"</span>,
+ <span class="hljs-string">"conditionResult"</span>:{
+ <span class="hljs-string">"successNode"</span>:[
+ <span class="hljs-string">""</span>
+ ],
+ <span class="hljs-string">"failedNode"</span>:[
+ <span class="hljs-string">""</span>
+ ]
+ },
+ <span class="hljs-string">"dependence"</span>:{
+
+ },
+ <span class="hljs-string">"maxRetryTimes"</span>:<span class="hljs-string">"0"</span>,
+ <span class="hljs-string">"retryInterval"</span>:<span class="hljs-string">"1"</span>,
+ <span class="hljs-string">"timeout"</span>:{
+ <span class="hljs-string">"strategy"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"interval"</span>:null,
+ <span class="hljs-string">"enable"</span>:<span class="hljs-literal">false</span>
+ },
+ <span class="hljs-string">"taskInstancePriority"</span>:<span class="hljs-string">"MEDIUM"</span>,
+ <span class="hljs-string">"workerGroup"</span>:<span class="hljs-string">"default"</span>,
+ <span class="hljs-string">"preTasks"</span>:[
+
+ ]
+}
+</code></pre>
+<h2>Sqoop Node</h2>
+<p><strong>The node data structure is as follows:</strong></p>
+<table>
+<thead>
+<tr>
+<th>No.</th>
+<th>parameter name</th>
+<th></th>
+<th>type</th>
+<th>description</th>
+<th>notes</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>1</td>
+<td>id</td>
+<td></td>
+<td>String</td>
+<td>task ID</td>
+<td></td>
+</tr>
+<tr>
+<td>2</td>
+<td>type</td>
+<td></td>
+<td>String</td>
+<td>task type</td>
+<td>SQOOP</td>
+</tr>
+<tr>
+<td>3</td>
+<td>name</td>
+<td></td>
+<td>String</td>
+<td>task name</td>
+<td></td>
+</tr>
+<tr>
+<td>4</td>
+<td>params</td>
+<td></td>
+<td>Object</td>
+<td>customized parameters</td>
+<td>Json format</td>
+</tr>
+<tr>
+<td>5</td>
+<td></td>
+<td>concurrency</td>
+<td>Int</td>
+<td>concurrency rate</td>
+<td></td>
+</tr>
+<tr>
+<td>6</td>
+<td></td>
+<td>modelType</td>
+<td>String</td>
+<td>flow direction</td>
+<td>import,export</td>
+</tr>
+<tr>
+<td>7</td>
+<td></td>
+<td>sourceType</td>
+<td>String</td>
+<td>datasource type</td>
+<td></td>
+</tr>
+<tr>
+<td>8</td>
+<td></td>
+<td>sourceParams</td>
+<td>String</td>
+<td>datasource parameters</td>
+<td>JSON format</td>
+</tr>
+<tr>
+<td>9</td>
+<td></td>
+<td>targetType</td>
+<td>String</td>
+<td>target datasource type</td>
+<td></td>
+</tr>
+<tr>
+<td>10</td>
+<td></td>
+<td>targetParams</td>
+<td>String</td>
+<td>target datasource parameters</td>
+<td>JSON format</td>
+</tr>
+<tr>
+<td>11</td>
+<td></td>
+<td>localParams</td>
+<td>Array</td>
+<td>customized local parameters</td>
+<td></td>
+</tr>
+<tr>
+<td>12</td>
+<td>description</td>
+<td></td>
+<td>String</td>
+<td>description</td>
+<td></td>
+</tr>
+<tr>
+<td>13</td>
+<td>runFlag</td>
+<td></td>
+<td>String</td>
+<td>execution flag</td>
+<td></td>
+</tr>
+<tr>
+<td>14</td>
+<td>conditionResult</td>
+<td></td>
+<td>Object</td>
+<td>condition branch</td>
+<td></td>
+</tr>
+<tr>
+<td>15</td>
+<td></td>
+<td>successNode</td>
+<td>Array</td>
+<td>jump node if success</td>
+<td></td>
+</tr>
+<tr>
+<td>16</td>
+<td></td>
+<td>failedNode</td>
+<td>Array</td>
+<td>jump node if failure</td>
+<td></td>
+</tr>
+<tr>
+<td>17</td>
+<td>dependence</td>
+<td></td>
+<td>Object</td>
+<td>task dependency</td>
+<td>mutual exclusion with params</td>
+</tr>
+<tr>
+<td>18</td>
+<td>maxRetryTimes</td>
+<td></td>
+<td>String</td>
+<td>max retry times</td>
+<td></td>
+</tr>
+<tr>
+<td>19</td>
+<td>retryInterval</td>
+<td></td>
+<td>String</td>
+<td>retry interval</td>
+<td></td>
+</tr>
+<tr>
+<td>20</td>
+<td>timeout</td>
+<td></td>
+<td>Object</td>
+<td>timeout</td>
+<td></td>
+</tr>
+<tr>
+<td>21</td>
+<td>taskInstancePriority</td>
+<td></td>
+<td>String</td>
+<td>task priority</td>
+<td></td>
+</tr>
+<tr>
+<td>22</td>
+<td>workerGroup</td>
+<td></td>
+<td>String</td>
+<td>Worker group</td>
+<td></td>
+</tr>
+<tr>
+<td>23</td>
+<td>preTasks</td>
+<td></td>
+<td>Array</td>
+<td>preposition tasks</td>
+<td></td>
+</tr>
+</tbody>
+</table>
+<p><strong>Node data example:</strong></p>
+<pre><code class="language-bash">{
+ <span class="hljs-string">"type"</span>:<span class="hljs-string">"SQOOP"</span>,
+ <span class="hljs-string">"id"</span>:<span class="hljs-string">"tasks-82041"</span>,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"Sqoop Task"</span>,
+ <span class="hljs-string">"params"</span>:{
+ <span class="hljs-string">"concurrency"</span>:1,
+ <span class="hljs-string">"modelType"</span>:<span class="hljs-string">"import"</span>,
+ <span class="hljs-string">"sourceType"</span>:<span class="hljs-string">"MYSQL"</span>,
+ <span class="hljs-string">"targetType"</span>:<span class="hljs-string">"HDFS"</span>,
+ <span class="hljs-string">"sourceParams"</span>:<span class="hljs-string">"{"</span>srcType<span class="hljs-string">":"</span>MYSQL<span class="hljs-string">","</span>srcDatasource<span class="hljs-string">":1,"</span>srcTable<span class="hljs-string">":"</span><span class="hljs-string">","</span>srcQueryType<span class="hljs-string">":"</span>1<span class="hljs-string">","</span>srcQuerySql<span class="hljs-string">":"</span>selec id , name from user<span class="hljs-st [...]
+ <span class="hljs-string">"targetParams"</span>:<span class="hljs-string">"{"</span>targetPath<span class="hljs-string">":"</span>/user/hive/warehouse/ods.db/user<span class="hljs-string">","</span>deleteTargetDir<span class="hljs-string">":false,"</span>fileType<span class="hljs-string">":"</span>--as-avrodatafile<span class="hljs-string">","</span>compressionCodec<span class="hljs-string">":"</span>snappy<span class="hljs-string">","</span>fieldsTerminated<span class="h [...]
+ <span class="hljs-string">"localParams"</span>:[
+
+ ]
+ },
+ <span class="hljs-string">"description"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"runFlag"</span>:<span class="hljs-string">"NORMAL"</span>,
+ <span class="hljs-string">"conditionResult"</span>:{
+ <span class="hljs-string">"successNode"</span>:[
+ <span class="hljs-string">""</span>
+ ],
+ <span class="hljs-string">"failedNode"</span>:[
+ <span class="hljs-string">""</span>
+ ]
+ },
+ <span class="hljs-string">"dependence"</span>:{
+
+ },
+ <span class="hljs-string">"maxRetryTimes"</span>:<span class="hljs-string">"0"</span>,
+ <span class="hljs-string">"retryInterval"</span>:<span class="hljs-string">"1"</span>,
+ <span class="hljs-string">"timeout"</span>:{
+ <span class="hljs-string">"strategy"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"interval"</span>:null,
+ <span class="hljs-string">"enable"</span>:<span class="hljs-literal">false</span>
+ },
+ <span class="hljs-string">"taskInstancePriority"</span>:<span class="hljs-string">"MEDIUM"</span>,
+ <span class="hljs-string">"workerGroup"</span>:<span class="hljs-string">"default"</span>,
+ <span class="hljs-string">"preTasks"</span>:[
+
+ ]
+ }
+</code></pre>
+<h2>Condition Branch Node</h2>
+<p><strong>The node data structure is as follows:</strong></p>
+<table>
+<thead>
+<tr>
+<th>No.</th>
+<th>parameter name</th>
+<th></th>
+<th>type</th>
+<th>description</th>
+<th>notes</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>1</td>
+<td>id</td>
+<td></td>
+<td>String</td>
+<td>task ID</td>
+<td></td>
+</tr>
+<tr>
+<td>2</td>
+<td>type</td>
+<td></td>
+<td>String</td>
+<td>task type</td>
+<td>SHELL</td>
+</tr>
+<tr>
+<td>3</td>
+<td>name</td>
+<td></td>
+<td>String</td>
+<td>task name</td>
+<td></td>
+</tr>
+<tr>
+<td>4</td>
+<td>params</td>
+<td></td>
+<td>Object</td>
+<td>customized parameters</td>
+<td>null</td>
+</tr>
+<tr>
+<td>5</td>
+<td>description</td>
+<td></td>
+<td>String</td>
+<td>description</td>
+<td></td>
+</tr>
+<tr>
+<td>6</td>
+<td>runFlag</td>
+<td></td>
+<td>String</td>
+<td>execution flag</td>
+<td></td>
+</tr>
+<tr>
+<td>7</td>
+<td>conditionResult</td>
+<td></td>
+<td>Object</td>
+<td>condition branch</td>
+<td></td>
+</tr>
+<tr>
+<td>8</td>
+<td></td>
+<td>successNode</td>
+<td>Array</td>
+<td>jump to node if success</td>
+<td></td>
+</tr>
+<tr>
+<td>9</td>
+<td></td>
+<td>failedNode</td>
+<td>Array</td>
+<td>jump to node if failure</td>
+<td></td>
+</tr>
+<tr>
+<td>10</td>
+<td>dependence</td>
+<td></td>
+<td>Object</td>
+<td>task dependency</td>
+<td>mutual exclusion with params</td>
+</tr>
+<tr>
+<td>11</td>
+<td>maxRetryTimes</td>
+<td></td>
+<td>String</td>
+<td>max retry times</td>
+<td></td>
+</tr>
+<tr>
+<td>12</td>
+<td>retryInterval</td>
+<td></td>
+<td>String</td>
+<td>retry interval</td>
+<td></td>
+</tr>
+<tr>
+<td>13</td>
+<td>timeout</td>
+<td></td>
+<td>Object</td>
+<td>timeout</td>
+<td></td>
+</tr>
+<tr>
+<td>14</td>
+<td>taskInstancePriority</td>
+<td></td>
+<td>String</td>
+<td>task priority</td>
+<td></td>
+</tr>
+<tr>
+<td>15</td>
+<td>workerGroup</td>
+<td></td>
+<td>String</td>
+<td>Worker group</td>
+<td></td>
+</tr>
+<tr>
+<td>16</td>
+<td>preTasks</td>
+<td></td>
+<td>Array</td>
+<td>preposition tasks</td>
+<td></td>
+</tr>
+</tbody>
+</table>
+<p><strong>Node data example:</strong></p>
+<pre><code class="language-bash">{
+ <span class="hljs-string">"type"</span>:<span class="hljs-string">"CONDITIONS"</span>,
+ <span class="hljs-string">"id"</span>:<span class="hljs-string">"tasks-96189"</span>,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"条件"</span>,
+ <span class="hljs-string">"params"</span>:{
+
+ },
+ <span class="hljs-string">"description"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"runFlag"</span>:<span class="hljs-string">"NORMAL"</span>,
+ <span class="hljs-string">"conditionResult"</span>:{
+ <span class="hljs-string">"successNode"</span>:[
+ <span class="hljs-string">"test04"</span>
+ ],
+ <span class="hljs-string">"failedNode"</span>:[
+ <span class="hljs-string">"test05"</span>
+ ]
+ },
+ <span class="hljs-string">"dependence"</span>:{
+ <span class="hljs-string">"relation"</span>:<span class="hljs-string">"AND"</span>,
+ <span class="hljs-string">"dependTaskList"</span>:[
+
+ ]
+ },
+ <span class="hljs-string">"maxRetryTimes"</span>:<span class="hljs-string">"0"</span>,
+ <span class="hljs-string">"retryInterval"</span>:<span class="hljs-string">"1"</span>,
+ <span class="hljs-string">"timeout"</span>:{
+ <span class="hljs-string">"strategy"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"interval"</span>:null,
+ <span class="hljs-string">"enable"</span>:<span class="hljs-literal">false</span>
+ },
+ <span class="hljs-string">"taskInstancePriority"</span>:<span class="hljs-string">"MEDIUM"</span>,
+ <span class="hljs-string">"workerGroup"</span>:<span class="hljs-string">"default"</span>,
+ <span class="hljs-string">"preTasks"</span>:[
+ <span class="hljs-string">"test01"</span>,
+ <span class="hljs-string">"test02"</span>
+ ]
+}
+</code></pre>
+<h2>Subprocess Node</h2>
+<p><strong>The node data structure is as follows:</strong></p>
+<table>
+<thead>
+<tr>
+<th>No.</th>
+<th>parameter name</th>
+<th></th>
+<th>type</th>
+<th>description</th>
+<th>notes</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>1</td>
+<td>id</td>
+<td></td>
+<td>String</td>
+<td>task ID</td>
+<td></td>
+</tr>
+<tr>
+<td>2</td>
+<td>type</td>
+<td></td>
+<td>String</td>
+<td>task type</td>
+<td>SHELL</td>
+</tr>
+<tr>
+<td>3</td>
+<td>name</td>
+<td></td>
+<td>String</td>
+<td>task name</td>
+<td></td>
+</tr>
+<tr>
+<td>4</td>
+<td>params</td>
+<td></td>
+<td>Object</td>
+<td>customized parameters</td>
+<td>Json format</td>
+</tr>
+<tr>
+<td>5</td>
+<td></td>
+<td>processDefinitionId</td>
+<td>Int</td>
+<td>process definition ID</td>
+<td></td>
+</tr>
+<tr>
+<td>6</td>
+<td>description</td>
+<td></td>
+<td>String</td>
+<td>description</td>
+<td></td>
+</tr>
+<tr>
+<td>7</td>
+<td>runFlag</td>
+<td></td>
+<td>String</td>
+<td>execution flag</td>
+<td></td>
+</tr>
+<tr>
+<td>8</td>
+<td>conditionResult</td>
+<td></td>
+<td>Object</td>
+<td>condition branch</td>
+<td></td>
+</tr>
+<tr>
+<td>9</td>
+<td></td>
+<td>successNode</td>
+<td>Array</td>
+<td>jump to node if success</td>
+<td></td>
+</tr>
+<tr>
+<td>10</td>
+<td></td>
+<td>failedNode</td>
+<td>Array</td>
+<td>jump to node if failure</td>
+<td></td>
+</tr>
+<tr>
+<td>11</td>
+<td>dependence</td>
+<td></td>
+<td>Object</td>
+<td>task dependency</td>
+<td>mutual exclusion with params</td>
+</tr>
+<tr>
+<td>12</td>
+<td>maxRetryTimes</td>
+<td></td>
+<td>String</td>
+<td>max retry times</td>
+<td></td>
+</tr>
+<tr>
+<td>13</td>
+<td>retryInterval</td>
+<td></td>
+<td>String</td>
+<td>retry interval</td>
+<td></td>
+</tr>
+<tr>
+<td>14</td>
+<td>timeout</td>
+<td></td>
+<td>Object</td>
+<td>timeout</td>
+<td></td>
+</tr>
+<tr>
+<td>15</td>
+<td>taskInstancePriority</td>
+<td></td>
+<td>String</td>
+<td>task priority</td>
+<td></td>
+</tr>
+<tr>
+<td>16</td>
+<td>workerGroup</td>
+<td></td>
+<td>String</td>
+<td>Worker group</td>
+<td></td>
+</tr>
+<tr>
+<td>17</td>
+<td>preTasks</td>
+<td></td>
+<td>Array</td>
+<td>preposition tasks</td>
+<td></td>
+</tr>
+</tbody>
+</table>
+<p><strong>Node data example:</strong></p>
+<pre><code class="language-bash">{
+ <span class="hljs-string">"type"</span>:<span class="hljs-string">"SUB_PROCESS"</span>,
+ <span class="hljs-string">"id"</span>:<span class="hljs-string">"tasks-14806"</span>,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"SubProcessTask"</span>,
+ <span class="hljs-string">"params"</span>:{
+ <span class="hljs-string">"processDefinitionId"</span>:2
+ },
+ <span class="hljs-string">"description"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"runFlag"</span>:<span class="hljs-string">"NORMAL"</span>,
+ <span class="hljs-string">"conditionResult"</span>:{
+ <span class="hljs-string">"successNode"</span>:[
+ <span class="hljs-string">""</span>
+ ],
+ <span class="hljs-string">"failedNode"</span>:[
+ <span class="hljs-string">""</span>
+ ]
+ },
+ <span class="hljs-string">"dependence"</span>:{
+
+ },
+ <span class="hljs-string">"timeout"</span>:{
+ <span class="hljs-string">"strategy"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"interval"</span>:null,
+ <span class="hljs-string">"enable"</span>:<span class="hljs-literal">false</span>
+ },
+ <span class="hljs-string">"taskInstancePriority"</span>:<span class="hljs-string">"MEDIUM"</span>,
+ <span class="hljs-string">"workerGroup"</span>:<span class="hljs-string">"default"</span>,
+ <span class="hljs-string">"preTasks"</span>:[
+
+ ]
+ }
+</code></pre>
+<h2>DEPENDENT Node</h2>
+<p><strong>The node data structure is as follows:</strong></p>
+<table>
+<thead>
+<tr>
+<th>No.</th>
+<th>parameter name</th>
+<th></th>
+<th>type</th>
+<th>description</th>
+<th>notes</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>1</td>
+<td>id</td>
+<td></td>
+<td>String</td>
+<td>task ID</td>
+<td></td>
+</tr>
+<tr>
+<td>2</td>
+<td>type</td>
+<td></td>
+<td>String</td>
+<td>task type</td>
+<td>DEPENDENT</td>
+</tr>
+<tr>
+<td>3</td>
+<td>name</td>
+<td></td>
+<td>String</td>
+<td>task name</td>
+<td></td>
+</tr>
+<tr>
+<td>4</td>
+<td>params</td>
+<td></td>
+<td>Object</td>
+<td>customized parameters</td>
+<td>Json format</td>
+</tr>
+<tr>
+<td>5</td>
+<td></td>
+<td>rawScript</td>
+<td>String</td>
+<td>Shell script</td>
+<td></td>
+</tr>
+<tr>
+<td>6</td>
+<td></td>
+<td>localParams</td>
+<td>Array</td>
+<td>customized local parameters</td>
+<td></td>
+</tr>
+<tr>
+<td>7</td>
+<td></td>
+<td>resourceList</td>
+<td>Array</td>
+<td>resource files</td>
+<td></td>
+</tr>
+<tr>
+<td>8</td>
+<td>description</td>
+<td></td>
+<td>String</td>
+<td>description</td>
+<td></td>
+</tr>
+<tr>
+<td>9</td>
+<td>runFlag</td>
+<td></td>
+<td>String</td>
+<td>execution flag</td>
+<td></td>
+</tr>
+<tr>
+<td>10</td>
+<td>conditionResult</td>
+<td></td>
+<td>Object</td>
+<td>condition branch</td>
+<td></td>
+</tr>
+<tr>
+<td>11</td>
+<td></td>
+<td>successNode</td>
+<td>Array</td>
+<td>jump to node if success</td>
+<td></td>
+</tr>
+<tr>
+<td>12</td>
+<td></td>
+<td>failedNode</td>
+<td>Array</td>
+<td>jump to node if failure</td>
+<td></td>
+</tr>
+<tr>
+<td>13</td>
+<td>dependence</td>
+<td></td>
+<td>Object</td>
+<td>task dependency</td>
+<td>mutual exclusion with params</td>
+</tr>
+<tr>
+<td>14</td>
+<td></td>
+<td>relation</td>
+<td>String</td>
+<td>relation</td>
+<td>AND,OR</td>
+</tr>
+<tr>
+<td>15</td>
+<td></td>
+<td>dependTaskList</td>
+<td>Array</td>
+<td>dependent task list</td>
+<td></td>
+</tr>
+<tr>
+<td>16</td>
+<td>maxRetryTimes</td>
+<td></td>
+<td>String</td>
+<td>max retry times</td>
+<td></td>
+</tr>
+<tr>
+<td>17</td>
+<td>retryInterval</td>
+<td></td>
+<td>String</td>
+<td>retry interval</td>
+<td></td>
+</tr>
+<tr>
+<td>18</td>
+<td>timeout</td>
+<td></td>
+<td>Object</td>
+<td>timeout</td>
+<td></td>
+</tr>
+<tr>
+<td>19</td>
+<td>taskInstancePriority</td>
+<td></td>
+<td>String</td>
+<td>task priority</td>
+<td></td>
+</tr>
+<tr>
+<td>20</td>
+<td>workerGroup</td>
+<td></td>
+<td>String</td>
+<td>Worker group</td>
+<td></td>
+</tr>
+<tr>
+<td>21</td>
+<td>preTasks</td>
+<td></td>
+<td>Array</td>
+<td>preposition tasks</td>
+<td></td>
+</tr>
+</tbody>
+</table>
+<p><strong>Node data example:</strong></p>
+<pre><code class="language-bash">{
+ <span class="hljs-string">"type"</span>:<span class="hljs-string">"DEPENDENT"</span>,
+ <span class="hljs-string">"id"</span>:<span class="hljs-string">"tasks-57057"</span>,
+ <span class="hljs-string">"name"</span>:<span class="hljs-string">"DenpendentTask"</span>,
+ <span class="hljs-string">"params"</span>:{
+
+ },
+ <span class="hljs-string">"description"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"runFlag"</span>:<span class="hljs-string">"NORMAL"</span>,
+ <span class="hljs-string">"conditionResult"</span>:{
+ <span class="hljs-string">"successNode"</span>:[
+ <span class="hljs-string">""</span>
+ ],
+ <span class="hljs-string">"failedNode"</span>:[
+ <span class="hljs-string">""</span>
+ ]
+ },
+ <span class="hljs-string">"dependence"</span>:{
+ <span class="hljs-string">"relation"</span>:<span class="hljs-string">"AND"</span>,
+ <span class="hljs-string">"dependTaskList"</span>:[
+ {
+ <span class="hljs-string">"relation"</span>:<span class="hljs-string">"AND"</span>,
+ <span class="hljs-string">"dependItemList"</span>:[
+ {
+ <span class="hljs-string">"projectId"</span>:1,
+ <span class="hljs-string">"definitionId"</span>:7,
+ <span class="hljs-string">"definitionList"</span>:[
+ {
+ <span class="hljs-string">"value"</span>:8,
+ <span class="hljs-string">"label"</span>:<span class="hljs-string">"MRTask"</span>
+ },
+ {
+ <span class="hljs-string">"value"</span>:7,
+ <span class="hljs-string">"label"</span>:<span class="hljs-string">"FlinkTask"</span>
+ },
+ {
+ <span class="hljs-string">"value"</span>:6,
+ <span class="hljs-string">"label"</span>:<span class="hljs-string">"SparkTask"</span>
+ },
+ {
+ <span class="hljs-string">"value"</span>:5,
+ <span class="hljs-string">"label"</span>:<span class="hljs-string">"SqlTask-Update"</span>
+ },
+ {
+ <span class="hljs-string">"value"</span>:4,
+ <span class="hljs-string">"label"</span>:<span class="hljs-string">"SqlTask-Query"</span>
+ },
+ {
+ <span class="hljs-string">"value"</span>:3,
+ <span class="hljs-string">"label"</span>:<span class="hljs-string">"SubProcessTask"</span>
+ },
+ {
+ <span class="hljs-string">"value"</span>:2,
+ <span class="hljs-string">"label"</span>:<span class="hljs-string">"Python Task"</span>
+ },
+ {
+ <span class="hljs-string">"value"</span>:1,
+ <span class="hljs-string">"label"</span>:<span class="hljs-string">"Shell Task"</span>
+ }
+ ],
+ <span class="hljs-string">"depTasks"</span>:<span class="hljs-string">"ALL"</span>,
+ <span class="hljs-string">"cycle"</span>:<span class="hljs-string">"day"</span>,
+ <span class="hljs-string">"dateValue"</span>:<span class="hljs-string">"today"</span>
+ }
+ ]
+ },
+ {
+ <span class="hljs-string">"relation"</span>:<span class="hljs-string">"AND"</span>,
+ <span class="hljs-string">"dependItemList"</span>:[
+ {
+ <span class="hljs-string">"projectId"</span>:1,
+ <span class="hljs-string">"definitionId"</span>:5,
+ <span class="hljs-string">"definitionList"</span>:[
+ {
+ <span class="hljs-string">"value"</span>:8,
+ <span class="hljs-string">"label"</span>:<span class="hljs-string">"MRTask"</span>
+ },
+ {
+ <span class="hljs-string">"value"</span>:7,
+ <span class="hljs-string">"label"</span>:<span class="hljs-string">"FlinkTask"</span>
+ },
+ {
+ <span class="hljs-string">"value"</span>:6,
+ <span class="hljs-string">"label"</span>:<span class="hljs-string">"SparkTask"</span>
+ },
+ {
+ <span class="hljs-string">"value"</span>:5,
+ <span class="hljs-string">"label"</span>:<span class="hljs-string">"SqlTask-Update"</span>
+ },
+ {
+ <span class="hljs-string">"value"</span>:4,
+ <span class="hljs-string">"label"</span>:<span class="hljs-string">"SqlTask-Query"</span>
+ },
+ {
+ <span class="hljs-string">"value"</span>:3,
+ <span class="hljs-string">"label"</span>:<span class="hljs-string">"SubProcessTask"</span>
+ },
+ {
+ <span class="hljs-string">"value"</span>:2,
+ <span class="hljs-string">"label"</span>:<span class="hljs-string">"Python Task"</span>
+ },
+ {
+ <span class="hljs-string">"value"</span>:1,
+ <span class="hljs-string">"label"</span>:<span class="hljs-string">"Shell Task"</span>
+ }
+ ],
+ <span class="hljs-string">"depTasks"</span>:<span class="hljs-string">"SqlTask-Update"</span>,
+ <span class="hljs-string">"cycle"</span>:<span class="hljs-string">"day"</span>,
+ <span class="hljs-string">"dateValue"</span>:<span class="hljs-string">"today"</span>
+ }
+ ]
+ }
+ ]
+ },
+ <span class="hljs-string">"maxRetryTimes"</span>:<span class="hljs-string">"0"</span>,
+ <span class="hljs-string">"retryInterval"</span>:<span class="hljs-string">"1"</span>,
+ <span class="hljs-string">"timeout"</span>:{
+ <span class="hljs-string">"strategy"</span>:<span class="hljs-string">""</span>,
+ <span class="hljs-string">"interval"</span>:null,
+ <span class="hljs-string">"enable"</span>:<span class="hljs-literal">false</span>
+ },
+ <span class="hljs-string">"taskInstancePriority"</span>:<span class="hljs-string">"MEDIUM"</span>,
+ <span class="hljs-string">"workerGroup"</span>:<span class="hljs-string">"default"</span>,
+ <span class="hljs-string">"preTasks"</span>:[
+
+ ]
+ }
+</code></pre>
+</div></section><footer class="footer-container"><div class="footer-body"><img src="/img/ds_gray.svg"/><div class="cols-container"><div class="col col-12"><h3>Disclaimer</h3><p>Apache DolphinScheduler is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a man [...]
+ <script src="https://f.alicdn.com/react/15.4.1/react-with-addons.min.js"></script>
+ <script src="https://f.alicdn.com/react/15.4.1/react-dom.min.js"></script>
+ <script>window.rootPath = '';</script>
+ <script src="/build/documentation.md.js"></script>
+ <script>
+ var _hmt = _hmt || [];
+ (function() {
+ var hm = document.createElement("script");
+ hm.src = "https://hm.baidu.com/hm.js?4e7b4b400dd31fa015018a435c64d06f";
+ var s = document.getElementsByTagName("script")[0];
+ s.parentNode.insertBefore(hm, s);
+ })();
+ </script>
+</body>
+</html>
\ No newline at end of file
diff --git a/en-us/docs/1.3.4/user_doc/task-structure.json b/en-us/docs/1.3.4/user_doc/task-structure.json
new file mode 100644
index 0000000..fe4cf24
--- /dev/null
+++ b/en-us/docs/1.3.4/user_doc/task-structure.json
@@ -0,0 +1,6 @@
+{
+ "filename": "task-structure.md",
+ "__html": "<h1>Overall Tasks Storage Structure</h1>\n<p>All tasks created in Dolphinscheduler are saved in the t_ds_process_definition table.</p>\n<p>The following shows the 't_ds_process_definition' table structure:</p>\n<table>\n<thead>\n<tr>\n<th>No.</th>\n<th>field</th>\n<th>type</th>\n<th>description</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>1</td>\n<td>id</td>\n<td>int(11)</td>\n<td>primary key</td>\n</tr>\n<tr>\n<td>2</td>\n<td>name</td>\n<td>varchar(255)</td>\n<td>process defin [...]
+ "link": "/en-us/docs/1.3.4/user_doc/task-structure.html",
+ "meta": {}
+}
\ No newline at end of file