You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/12/28 10:59:23 UTC
[GitHub] [incubator-dolphinscheduler] lenboo opened a new issue #4325: [Feature][JsonSplit] Propose for process definithon json split
lenboo opened a new issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325
we have discussed the split design proposal in mail list for a long time, i make up a list about this. If there are not any further suggestions, i think we can start coding that.
the mail list:
https://lists.apache.org/thread.html/r5dff090f16cf2aaa76af3cd6aa374f6815de7607734d77e36ed40650%40%3Cdev.dolphinscheduler.apache.org%3E
The following tasks need to be carried out in the order of the main title. If you are interested in any task, please comment or reply below the title.
我们已经在邮件列表中讨论了很长一段时间的json拆分的设计方案，我对此做了一个列表。如果没有任何进一步的建议，我想我们可以开始编码了。
以下任务需要按照大标题的顺序进行，如果你对任何任务感兴趣，请在标题下面评论或者回复
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] JinyLeeChina commented on issue #4325: [Feature][JsonSplit] Propose for process definithon json split
Posted by GitBox <gi...@apache.org>.
JinyLeeChina commented on issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325#issuecomment-752876843
I will do this part
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] lenboo commented on issue #4325: [Feature][JsonSplit] Propose for process definithon json split
Posted by GitBox <gi...@apache.org>.
lenboo commented on issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325#issuecomment-751674432
#### 3. Modify the front part:
- 1. Workflow editing, saving workflow information, job relationship information, job information
- 2. Create workflow a. Reference project-job, B. Create job
- 3. API part: use controller to call new interface, front end to dock new interface, delete old interface
- 4. Job management list page, create, update, delete, view details
### 3. 修改前端部分：
- 1. 工作流编辑，保存工作流信息，作业关系信息，作业信息
- 2. 创建工作流 a.引用项目-作业，b.创建作业
- 3. api部分：使用controller调用新接口，前端对接新接口，将旧的接口删掉
- 4. 作业管理列表页，创建，更新，删除，查看详情
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] simon824 commented on issue #4325: [Feature][JsonSplit] Propose for process definithon json split
Posted by GitBox <gi...@apache.org>.
simon824 commented on issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325#issuecomment-752888798
I will join
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] lenboo edited a comment on issue #4325: [Feature][JsonSplit] Propose for process definithon json split
Posted by GitBox <gi...@apache.org>.
lenboo edited a comment on issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325#issuecomment-751674235
#### 2. Refactoring api: use new structure completely, add new functionality to service without modifying controller
- 1. Implement code encoding scheme
- suggestion: snowflake + check
`When need to add a new code, use snowflake algorithm (IP is worker)_ ID / or a fixed ID) to get a new code, check whether the code exists in the database, if so, regenerate the code, and then check...`
- 2. Refactoring workflow data related processing interfaces, including version processing (query, new, modify, delete, import, export, offline...)
- 3. Refactoring workflow instance data related processing interfaces (query, modification, Gantt chart)
- 4. Refactoring job instance query interface
- 5. Rebuild workflow instance, job instance related statistics interface (UI system homepage, project homepage statistics, related monitoring data)
- 6. Job management related interfaces, including version management (add delete check, go online...)
### 2. 重构api：全部使用新结构, service中新增功能，不用修改controller
- 1. 实现code编码方案
- 雪花+检查
`在需要新增code的时候，使用雪花算法（ip为worker_id/或是一个固定id）获取一个新的code，在数据库中检查此code是否存在，如果存在，重新生成code,再进行检查。。。`
- 2. 重构工作流数据相关处理接口，包含版本处理（查询、新建、修改、删除、导入、导出、上下线...）
- 3. 重构工作流实例数据相关处理接口（查询、修改、甘特图）
- 4. 重构作业实例查询接口
- 5. 重构工作流实例、作业实例相关统计接口（UI系统首页、项目首页统计数据、相关监控数据）
- 6. 作业管理相关接口，包含版本管理（增删改查，上下线...）
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] lenboo edited a comment on issue #4325: [Feature][JsonSplit] Propose for process definithon json split
Posted by GitBox <gi...@apache.org>.
lenboo edited a comment on issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325#issuecomment-751673724
### 1. New tables and data structures, DB query interface (new structures do not delete/modify existing workflow/task structures at this time):
- 1. New Workflow Definition Table: t_ds_Process_Definition
```
|Column Name|Description|
| -------------------------------| ---------------- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| task_type | job type |
| task_params | Job Custom Parameters |
| run_flag | run flag |
| task_priority | job priority |
| worker_group | worker grouping |
| fail_retry_times | Number of failed retries |
| fail_retry_interval | failed retry interval |
| timeout_flag | timeout flag |
| timeout_notify_strategy | Timeout Notification Policy |
| timeout_duration | timeout length |
| create_time | creation time |
| update_time | modification time |
```
- 2. Workflow Job Relationships Table: t_ds_Process_Task_Relation
```
| Column Name | Description |
| ----------------------- | ------------------------- ------------- |
| id | self incrementing ID |
| project_code | project code |
| process_definition_code | Workflow coding |
| pre_project_code | pre-quoted project code |
| pre_task_code | pre-reference job code |
| condition_type | condition type 0: None 1: Judgment 2: Delayed |
| condition_params | condition parameters (json) |
| post_project_code | post reference project code |
| post_task_code | post reference job code |
| create_time | creation time |
| update_time | modification time |
```
- 3. Job Definition Table: t_ds_Task_Definition
```
|Column Name|Description|
| -------------------------------| ---------------- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| task_type | job type |
| task_params | Job Custom Parameters |
| run_flag | run flag |
| task_priority | job priority |
| worker_group | worker grouping |
| fail_retry_times | Number of failed retries |
| fail_retry_interval | failed retry interval |
| timeout_flag | timeout flag |
| timeout_notify_strategy | Timeout Notification Policy |
| timeout_duration | timeout length |
| create_time | creation time |
| update_time | modification time |
```
- 4. Workflow Definition Log Table: t_ds_Process_Definition_Log
`Add operation type (add, modify, delete), operator, operation time based on workflow definition table`
- 5. Workflow job relationship log table: t_ds_Process_Task_Relation_Log
`Add workflow version, operation type (add, modify, delete), operator, operation time based on job relationship table`
- 6. Job Definition Log Table: t_ds_Task_Definition_Log
`Add operation type (add, modify, delete), operator, operation time based on job definition table`
- 7. Workflow Instance Table Modification: t_ds_Process_Instance
`Add Workflow Definition Version Number Field`
### 1. 新增表以及数据结构，db查询接口(新增结构暂时不要删除/修改现有的工作流/任务结构):
- 1. 新增工作流定义表：t_ds_process_definition
```
| 列名 | 描述 |
| ---- | ---- |
| id | 自增ID |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| release_state | 发布状态 |
| user_id | 所属用户ID |
| global_params | 全局参数 |
| flag | 流程是否可用：0 不可用，1 可用 |
|alert_group_id|告警人分组id |
| timeout | 超时时间 |
| tenant_id | 租户ID |
| locations | 节点坐标信息 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 2. 工作流作业关系表：t_ds_process_task_relation
```
| 列名 | 描述 |
| ----------------------- | -------------------------------------- |
| id | 自增ID |
| project_code | 项目编码 |
| process_definition_code | 工作流编码 |
| pre_project_code | 前置引用项目编码 |
| pre_task_code | 前置引用作业编码 |
| condition_type | 条件类型 0：无 1：判断 2：延迟 |
| condition_params | 条件参数（json） |
| post_project_code | 后置引用项目编码 |
| post_task_code | 后置引用作业编码 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 3. 作业定义表：t_ds_task_definition
```
| 列名 | 描述 |
| ----------------------- | -------------- |
| id | 自增id |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| task_type | 作业类型 |
| task_params | 作业自定义参数 |
| run_flag | 运行标志 |
| task_priority | 作业优先级 |
| worker_group | worker分组 |
| fail_retry_times | 失败重试次数 |
| fail_retry_interval | 失败重试间隔 |
| timeout_flag | 超时标志 |
| timeout_notify_strategy | 超时通知策略 |
| timeout_duration | 超时时长 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 4. 工作流定义日志表：t_ds_process_definition_log
`工作流定义表基础上增加操作类型（新增、修改、删除）、操作人、操作时间`
- 5. 工作流作业关系日志表：t_ds_process_task_relation_log
`作业关系表基础上增加工作流版本、操作类型（新增、修改、删除）、操作人、操作时间`
- 6. 作业定义日志表：t_ds_task_definition_log
`作业定义表基础上增加操作类型（新增、修改、删除）、操作人、操作时间`
- 7. 工作流实例表修改：t_ds_process_instance
`新增工作流定义版本号字段`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] lenboo edited a comment on issue #4325: [Feature][JsonSplit] Propose for process definithon json split
Posted by GitBox <gi...@apache.org>.
lenboo edited a comment on issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325#issuecomment-751673724
### 1. New tables and data structures, DB query interface (new structures do not delete/modify existing workflow/task structures at this time):
- 1. New Workflow Definition Table: t_ds_Process_Definithon
```
|Column Name|Description|
| -------------------------------| ---------------- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| task_type | job type |
| task_params | Job Custom Parameters |
| run_flag | run flag |
| task_priority | job priority |
| worker_group | worker grouping |
| fail_retry_times | Number of failed retries |
| fail_retry_interval | failed retry interval |
| timeout_flag | timeout flag |
| timeout_notify_strategy | Timeout Notification Policy |
| timeout_duration | timeout length |
| create_time | creation time |
| update_time | modification time |
```
- 2. Workflow Job Relationships Table: t_ds_Process_Task_Relation
```
| Column Name | Description |
| ----------------------- | ------------------------- ------------- |
| id | self incrementing ID |
| project_code | project code |
| process_definition_code | Workflow coding |
| pre_project_code | pre-quoted project code |
| pre_task_code | pre-reference job code |
| condition_type | condition type 0: None 1: Judgment 2: Delayed |
| condition_params | condition parameters (json) |
| post_project_code | post reference project code |
| post_task_code | post reference job code |
| create_time | creation time |
| update_time | modification time |
```
- 3. Job Definition Table: t_ds_Task_Definithon
```
|Column Name|Description|
| ---- | ---- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| release_state | Publish status |
| user_id | Owner id |
| global_params | global parameters |
| flag | Is the process available: 0 not available, 1 available |
| receivers | recipients |
| warning_group_id | warning_group_id |
| timeout | timeout |
| tenant_id | Tenant id |
| locations | node coordinate information |
| create_time | creation time |
| update_time | modification time |
```
- 4. Workflow Definition Log Table: t_ds_Process_Definithon_Log
`Add operation type (add, modify, delete), operator, operation time based on workflow definition table`
- 5. Workflow job relationship log table: t_ds_Process_Task_Relation_Log
`Add workflow version, operation type (add, modify, delete), operator, operation time based on job relationship table`
- 6. Job Definition Log Table: t_ds_Task_Definithon_Log
`Add operation type (add, modify, delete), operator, operation time based on job definition table`
- 7. Workflow Instance Table Modification: t_ds_Process_Instance
`Add Workflow Definition Version Number Field`
### 1. 新增表以及数据结构，db查询接口(新增结构暂时不要删除/修改现有的工作流/任务结构):
- 1. 新增工作流定义表：t_ds_process_definithon
```
| 列名 | 描述 |
| ---- | ---- |
| id | 自增ID |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| release_state | 发布状态 |
| user_id | 所属用户ID |
| global_params | 全局参数 |
| flag | 流程是否可用：0 不可用，1 可用 |
|alert_group_id|告警人分组id |
| timeout | 超时时间 |
| tenant_id | 租户ID |
| locations | 节点坐标信息 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 2. 工作流作业关系表：t_ds_process_task_relation
```
| 列名 | 描述 |
| ----------------------- | -------------------------------------- |
| id | 自增ID |
| project_code | 项目编码 |
| process_definition_code | 工作流编码 |
| pre_project_code | 前置引用项目编码 |
| pre_task_code | 前置引用作业编码 |
| condition_type | 条件类型 0：无 1：判断 2：延迟 |
| condition_params | 条件参数（json） |
| post_project_code | 后置引用项目编码 |
| post_task_code | 后置引用作业编码 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 3. 作业定义表：t_ds_task_definithon
```
| 列名 | 描述 |
| ----------------------- | -------------- |
| id | 自增id |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| task_type | 作业类型 |
| task_params | 作业自定义参数 |
| run_flag | 运行标志 |
| task_priority | 作业优先级 |
| worker_group | worker分组 |
| fail_retry_times | 失败重试次数 |
| fail_retry_interval | 失败重试间隔 |
| timeout_flag | 超时标志 |
| timeout_notify_strategy | 超时通知策略 |
| timeout_duration | 超时时长 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 4. 工作流定义日志表：t_ds_process_definithon_log
`工作流定义表基础上增加操作类型（新增、修改、删除）、操作人、操作时间`
- 5. 工作流作业关系日志表：t_ds_process_task_relation_log
`作业关系表基础上增加工作流版本、操作类型（新增、修改、删除）、操作人、操作时间`
- 6. 作业定义日志表：t_ds_task_definithon_log
`作业定义表基础上增加操作类型（新增、修改、删除）、操作人、操作时间`
- 7. 工作流实例表修改：t_ds_process_instance
`新增工作流定义版本号字段`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] lenboo edited a comment on issue #4325: [Feature][JsonSplit] Propose for process definithon json split
Posted by GitBox <gi...@apache.org>.
lenboo edited a comment on issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325#issuecomment-751673724
### 1. New tables and data structures, DB query interface (new structures do not delete/modify existing workflow/task structures at this time):
- 1. New Workflow Definition Table: t_ds_Process_Definithon
```
|Column Name|Description|
| -------------------------------| ---------------- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| task_type | job type |
| task_params | Job Custom Parameters |
| run_flag | run flag |
| task_priority | job priority |
| worker_group | worker grouping |
| fail_retry_times | Number of failed retries |
| fail_retry_interval | failed retry interval |
| timeout_flag | timeout flag |
| timeout_notify_strategy | Timeout Notification Policy |
| timeout_duration | timeout length |
| create_time | creation time |
| update_time | modification time |
```
- 2. Workflow Job Relationships Table: t_ds_Process_Task_Relation
```
|Column Name|Description|
| -------------------------------| ---------------- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| task_type | job type |
| task_params | Job Custom Parameters |
| run_flag | run flag |
| task_priority | job priority |
| worker_Group | worker grouping |
| fail_Retry_Times | Number of failed retries |
| fail_retry_interval | failed retry interval |
| timeout_flag | timeout flag |
| timeout_notify_strategy | Timeout Notification Policy |
| timeout_duration | timeout length |
| create_time | creation time |
| update_time | modification time |
```
- 3. Job Definition Table: t_ds_Task_Definithon
```
|Column Name|Description|
| ---- | ---- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| release_state | Publish status |
| user_id | Owner id |
| global_params | global parameters |
| flag | Is the process available: 0 not available, 1 available |
| receivers | recipients |
| receivers_CC | Cc |
| timeout | timeout |
| tenant_id | Tenant id |
| locations | node coordinate information |
| create_time | creation time |
| update_time | modification time |
```
- 4. Workflow Definition Log Table: t_ds_Process_Definithon_Log
`Add operation type (add, modify, delete), operator, operation time based on workflow definition table`
- 5. Workflow job relationship log table: t_ds_Process_Task_Relation_Log
`Add workflow version, operation type (add, modify, delete), operator, operation time based on job relationship table`
- 6. Job Definition Log Table: t_ds_Task_Definithon_Log
`Add operation type (add, modify, delete), operator, operation time based on job definition table`
- 7. Workflow Instance Table Modification: t_ds_Process_Instance
`Add Workflow Definition Version Number Field`
### 1. 新增表以及数据结构，db查询接口(新增结构暂时不要删除/修改现有的工作流/任务结构):
- 1. 新增工作流定义表：t_ds_process_definithon
```
| 列名 | 描述 |
| ----------------------- | -------------- |
| id | 自增id |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| task_type | 作业类型 |
| task_params | 作业自定义参数 |
| run_flag | 运行标志 |
| task_priority | 作业优先级 |
| worker_group | worker分组 |
| fail_retry_times | 失败重试次数 |
| fail_retry_interval | 失败重试间隔 |
| timeout_flag | 超时标志 |
| timeout_notify_strategy | 超时通知策略 |
| timeout_duration | 超时时长 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 2. 工作流作业关系表：t_ds_process_task_relation
```
| 列名 | 描述 |
| ----------------------- | -------------- |
| id | 自增id |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| task_type | 作业类型 |
| task_params | 作业自定义参数 |
| run_flag | 运行标志 |
| task_priority | 作业优先级 |
| worker_group | worker分组 |
| fail_retry_times | 失败重试次数 |
| fail_retry_interval | 失败重试间隔 |
| timeout_flag | 超时标志 |
| timeout_notify_strategy | 超时通知策略 |
| timeout_duration | 超时时长 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 3. 作业定义表：t_ds_task_definithon
```
| 列名 | 描述 |
| ---- | ---- |
| id | 自增id |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| release_state | 发布状态 |
| user_id | 所属用户id |
| global_params | 全局参数 |
| flag | 流程是否可用：0 不可用，1 可用 |
| receivers | 收件人 |
| receivers_cc | 抄送人 |
| timeout | 超时时间 |
| tenant_id | 租户id |
| locations | 节点坐标信息 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 4. 工作流定义日志表：t_ds_process_definithon_log
`工作流定义表基础上增加操作类型（新增、修改、删除）、操作人、操作时间`
- 5. 工作流作业关系日志表：t_ds_process_task_relation_log
`作业关系表基础上增加工作流版本、操作类型（新增、修改、删除）、操作人、操作时间`
- 6. 作业定义日志表：t_ds_task_definithon_log
`作业定义表基础上增加操作类型（新增、修改、删除）、操作人、操作时间`
- 7. 工作流实例表修改：t_ds_process_instance
`新增工作流定义版本号字段`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] lenboo edited a comment on issue #4325: [Feature][JsonSplit] Propose for process definithon json split
Posted by GitBox <gi...@apache.org>.
lenboo edited a comment on issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325#issuecomment-751673724
### 1. New tables and data structures, DB query interface (new structures do not delete/modify existing workflow/task structures at this time):
- 1. New Workflow Definition Table: t_ds_Process_Definithon
```
|Column Name|Description|
| -------------------------------| ---------------- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| task_type | job type |
| task_params | Job Custom Parameters |
| run_flag | run flag |
| task_priority | job priority |
| worker_group | worker grouping |
| fail_retry_times | Number of failed retries |
| fail_retry_interval | failed retry interval |
| timeout_flag | timeout flag |
| timeout_notify_strategy | Timeout Notification Policy |
| timeout_duration | timeout length |
| create_time | creation time |
| update_time | modification time |
```
- 2. Workflow Job Relationships Table: t_ds_Process_Task_Relation
```
|Column Name|Description|
| -------------------------------| ---------------- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| task_type | job type |
| task_params | Job Custom Parameters |
| run_flag | run flag |
| task_priority | job priority |
| worker_Group | worker grouping |
| fail_Retry_Times | Number of failed retries |
| fail_retry_interval | failed retry interval |
| timeout_flag | timeout flag |
| timeout_notify_strategy | Timeout Notification Policy |
| timeout_duration | timeout length |
| create_time | creation time |
| update_time | modification time |
```
- 3. Job Definition Table: t_ds_Task_Definithon
```
|Column Name|Description|
| ---- | ---- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| release_state | Publish status |
| user_id | Owner id |
| global_params | global parameters |
| flag | Is the process available: 0 not available, 1 available |
| receivers | recipients |
| warning_group_id | warning_group_id |
| timeout | timeout |
| tenant_id | Tenant id |
| locations | node coordinate information |
| create_time | creation time |
| update_time | modification time |
```
- 4. Workflow Definition Log Table: t_ds_Process_Definithon_Log
`Add operation type (add, modify, delete), operator, operation time based on workflow definition table`
- 5. Workflow job relationship log table: t_ds_Process_Task_Relation_Log
`Add workflow version, operation type (add, modify, delete), operator, operation time based on job relationship table`
- 6. Job Definition Log Table: t_ds_Task_Definithon_Log
`Add operation type (add, modify, delete), operator, operation time based on job definition table`
- 7. Workflow Instance Table Modification: t_ds_Process_Instance
`Add Workflow Definition Version Number Field`
### 1. 新增表以及数据结构，db查询接口(新增结构暂时不要删除/修改现有的工作流/任务结构):
- 1. 新增工作流定义表：t_ds_process_definithon
```
| 列名 | 描述 |
| ----------------------- | -------------- |
| id | 自增id |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| task_type | 作业类型 |
| task_params | 作业自定义参数 |
| run_flag | 运行标志 |
| task_priority | 作业优先级 |
| worker_group | worker分组 |
| fail_retry_times | 失败重试次数 |
| fail_retry_interval | 失败重试间隔 |
| timeout_flag | 超时标志 |
| timeout_notify_strategy | 超时通知策略 |
| timeout_duration | 超时时长 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 2. 工作流作业关系表：t_ds_process_task_relation
```
| 列名 | 描述 |
| ----------------------- | -------------- |
| id | 自增id |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| task_type | 作业类型 |
| task_params | 作业自定义参数 |
| run_flag | 运行标志 |
| task_priority | 作业优先级 |
| worker_group | worker分组 |
| fail_retry_times | 失败重试次数 |
| fail_retry_interval | 失败重试间隔 |
| timeout_flag | 超时标志 |
| timeout_notify_strategy | 超时通知策略 |
| timeout_duration | 超时时长 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 3. 作业定义表：t_ds_task_definithon
```
| 列名 | 描述 |
| ---- | ---- |
| id | 自增id |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| release_state | 发布状态 |
| user_id | 所属用户id |
| global_params | 全局参数 |
| flag | 流程是否可用：0 不可用，1 可用 |
| warning_group_id | 告警人 |
| timeout | 超时时间 |
| tenant_id | 租户id |
| locations | 节点坐标信息 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 4. 工作流定义日志表：t_ds_process_definithon_log
`工作流定义表基础上增加操作类型（新增、修改、删除）、操作人、操作时间`
- 5. 工作流作业关系日志表：t_ds_process_task_relation_log
`作业关系表基础上增加工作流版本、操作类型（新增、修改、删除）、操作人、操作时间`
- 6. 作业定义日志表：t_ds_task_definithon_log
`作业定义表基础上增加操作类型（新增、修改、删除）、操作人、操作时间`
- 7. 工作流实例表修改：t_ds_process_instance
`新增工作流定义版本号字段`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] lenboo closed issue #4325: [Feature][JsonSplit] Propose for process definithon json split
Posted by GitBox <gi...@apache.org>.
lenboo closed issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] lenboo commented on issue #4325: [Feature][JsonSplit] Propose for process definithon json split
Posted by GitBox <gi...@apache.org>.
lenboo commented on issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325#issuecomment-751674235
#### 2. Refactoring api: use new structure completely, add new functionality to service without modifying controller
- 1. Implement code encoding scheme
- 2. Refactoring workflow data related processing interfaces, including version processing (query, new, modify, delete, import, export, offline...)
- 3. Refactoring workflow instance data related processing interfaces (query, modification, Gantt chart)
- 4. Refactoring job instance query interface
- 5. Rebuild workflow instance, job instance related statistics interface (UI system homepage, project homepage statistics, related monitoring data)
- 6. Job management related interfaces, including version management (add delete check, go online...)
### 2. 重构api：全部使用新结构, service中新增功能，不用修改controller
- 1. 实现code编码方案
- 2. 重构工作流数据相关处理接口，包含版本处理（查询、新建、修改、删除、导入、导出、上下线...）
- 3. 重构工作流实例数据相关处理接口（查询、修改、甘特图）
- 4. 重构作业实例查询接口
- 5. 重构工作流实例、作业实例相关统计接口（UI系统首页、项目首页统计数据、相关监控数据）
- 6. 作业管理相关接口，包含版本管理（增删改查，上下线...）
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] lenboo edited a comment on issue #4325: [Feature][JsonSplit] Propose for process definithon json split
Posted by GitBox <gi...@apache.org>.
lenboo edited a comment on issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325#issuecomment-751673724
### 1. New tables and data structures, DB query interface (new structures do not delete/modify existing workflow/task structures at this time):
- 1. New Workflow Definition Table: t_ds_Process_Definithon
```
|Column Name|Description|
| -------------------------------| ---------------- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| task_type | job type |
| task_params | Job Custom Parameters |
| run_flag | run flag |
| task_priority | job priority |
| worker_group | worker grouping |
| fail_retry_times | Number of failed retries |
| fail_retry_interval | failed retry interval |
| timeout_flag | timeout flag |
| timeout_notify_strategy | Timeout Notification Policy |
| timeout_duration | timeout length |
| create_time | creation time |
| update_time | modification time |
```
- 2. Workflow Job Relationships Table: t_ds_Process_Task_Relation
```
| Column Name | Description |
| ----------------------- | ------------------------- ------------- |
| id | self incrementing ID |
| project_code | project code |
| process_definition_code | Workflow coding |
| pre_project_code | pre-quoted project code |
| pre_task_code | pre-reference job code |
| condition_type | condition type 0: None 1: Judgment 2: Delayed |
| condition_params | condition parameters (json) |
| post_project_code | post reference project code |
| post_task_code | post reference job code |
| create_time | creation time |
| update_time | modification time |
```
- 3. Job Definition Table: t_ds_Task_Definithon
```
|Column Name|Description|
| ---- | ---- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| release_state | Publish status |
| user_id | Owner id |
| global_params | global parameters |
| flag | Is the process available: 0 not available, 1 available |
| receivers | recipients |
| warning_group_id | warning_group_id |
| timeout | timeout |
| tenant_id | Tenant id |
| locations | node coordinate information |
| create_time | creation time |
| update_time | modification time |
```
- 4. Workflow Definition Log Table: t_ds_Process_Definithon_Log
`Add operation type (add, modify, delete), operator, operation time based on workflow definition table`
- 5. Workflow job relationship log table: t_ds_Process_Task_Relation_Log
`Add workflow version, operation type (add, modify, delete), operator, operation time based on job relationship table`
- 6. Job Definition Log Table: t_ds_Task_Definithon_Log
`Add operation type (add, modify, delete), operator, operation time based on job definition table`
- 7. Workflow Instance Table Modification: t_ds_Process_Instance
`Add Workflow Definition Version Number Field`
### 1. 新增表以及数据结构，db查询接口(新增结构暂时不要删除/修改现有的工作流/任务结构):
- 1. 新增工作流定义表：t_ds_process_definithon
```
| 列名 | 描述 |
| ---- | ---- |
| id | 自增ID |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| release_state | 发布状态 |
| user_id | 所属用户ID |
| global_params | 全局参数 |
| flag | 流程是否可用：0 不可用，1 可用 |
| receivers | 收件人 |
| receivers_cc | 抄送人 |
| timeout | 超时时间 |
| tenant_id | 租户ID |
| locations | 节点坐标信息 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 2. 工作流作业关系表：t_ds_process_task_relation
```
| 列名 | 描述 |
| ----------------------- | -------------------------------------- |
| id | 自增ID |
| project_code | 项目编码 |
| process_definition_code | 工作流编码 |
| pre_project_code | 前置引用项目编码 |
| pre_task_code | 前置引用作业编码 |
| condition_type | 条件类型 0：无 1：判断 2：延迟 |
| condition_params | 条件参数（json） |
| post_project_code | 后置引用项目编码 |
| post_task_code | 后置引用作业编码 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 3. 作业定义表：t_ds_task_definithon
```
| 列名 | 描述 |
| ---- | ---- |
| id | 自增id |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| release_state | 发布状态 |
| user_id | 所属用户id |
| global_params | 全局参数 |
| flag | 流程是否可用：0 不可用，1 可用 |
| warning_group_id | 告警人 |
| timeout | 超时时间 |
| tenant_id | 租户id |
| locations | 节点坐标信息 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 4. 工作流定义日志表：t_ds_process_definithon_log
`工作流定义表基础上增加操作类型（新增、修改、删除）、操作人、操作时间`
- 5. 工作流作业关系日志表：t_ds_process_task_relation_log
`作业关系表基础上增加工作流版本、操作类型（新增、修改、删除）、操作人、操作时间`
- 6. 作业定义日志表：t_ds_task_definithon_log
`作业定义表基础上增加操作类型（新增、修改、删除）、操作人、操作时间`
- 7. 工作流实例表修改：t_ds_process_instance
`新增工作流定义版本号字段`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] lenboo edited a comment on issue #4325: [Feature][JsonSplit] Propose for process definithon json split
Posted by GitBox <gi...@apache.org>.
lenboo edited a comment on issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325#issuecomment-751673724
### 1. New tables and data structures, DB query interface (new structures do not delete/modify existing workflow/task structures at this time):
- 1. New Workflow Definition Table: t_ds_Process_Definithon
```
|Column Name|Description|
| -------------------------------| ---------------- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| task_type | job type |
| task_params | Job Custom Parameters |
| run_flag | run flag |
| task_priority | job priority |
| worker_group | worker grouping |
| fail_retry_times | Number of failed retries |
| fail_retry_interval | failed retry interval |
| timeout_flag | timeout flag |
| timeout_notify_strategy | Timeout Notification Policy |
| timeout_duration | timeout length |
| create_time | creation time |
| update_time | modification time |
```
- 2. Workflow Job Relationships Table: t_ds_Process_Task_Relation
```
| Column Name | Description |
| ----------------------- | ------------------------- ------------- |
| id | self incrementing ID |
| project_code | project code |
| process_definition_code | Workflow coding |
| pre_project_code | pre-quoted project code |
| pre_task_code | pre-reference job code |
| condition_type | condition type 0: None 1: Judgment 2: Delayed |
| condition_params | condition parameters (json) |
| post_project_code | post reference project code |
| post_task_code | post reference job code |
| create_time | creation time |
| update_time | modification time |
```
- 3. Job Definition Table: t_ds_Task_Definithon
```
|Column Name|Description|
| ---- | ---- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| release_state | Publish status |
| user_id | Owner id |
| global_params | global parameters |
| flag | Is the process available: 0 not available, 1 available |
| receivers | recipients |
| warning_group_id | warning_group_id |
| timeout | timeout |
| tenant_id | Tenant id |
| locations | node coordinate information |
| create_time | creation time |
| update_time | modification time |
```
- 4. Workflow Definition Log Table: t_ds_Process_Definithon_Log
`Add operation type (add, modify, delete), operator, operation time based on workflow definition table`
- 5. Workflow job relationship log table: t_ds_Process_Task_Relation_Log
`Add workflow version, operation type (add, modify, delete), operator, operation time based on job relationship table`
- 6. Job Definition Log Table: t_ds_Task_Definithon_Log
`Add operation type (add, modify, delete), operator, operation time based on job definition table`
- 7. Workflow Instance Table Modification: t_ds_Process_Instance
`Add Workflow Definition Version Number Field`
### 1. 新增表以及数据结构，db查询接口(新增结构暂时不要删除/修改现有的工作流/任务结构):
- 1. 新增工作流定义表：t_ds_process_definithon
```
| 列名 | 描述 |
| ---- | ---- |
| id | 自增ID |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| release_state | 发布状态 |
| user_id | 所属用户ID |
| global_params | 全局参数 |
| flag | 流程是否可用：0 不可用，1 可用 |
|work_group_id|告警人分组id |
| timeout | 超时时间 |
| tenant_id | 租户ID |
| locations | 节点坐标信息 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 2. 工作流作业关系表：t_ds_process_task_relation
```
| 列名 | 描述 |
| ----------------------- | -------------------------------------- |
| id | 自增ID |
| project_code | 项目编码 |
| process_definition_code | 工作流编码 |
| pre_project_code | 前置引用项目编码 |
| pre_task_code | 前置引用作业编码 |
| condition_type | 条件类型 0：无 1：判断 2：延迟 |
| condition_params | 条件参数（json） |
| post_project_code | 后置引用项目编码 |
| post_task_code | 后置引用作业编码 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 3. 作业定义表：t_ds_task_definithon
```
| 列名 | 描述 |
| ---- | ---- |
| id | 自增id |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| release_state | 发布状态 |
| user_id | 所属用户id |
| global_params | 全局参数 |
| flag | 流程是否可用：0 不可用，1 可用 |
| warning_group_id | 告警人 |
| timeout | 超时时间 |
| tenant_id | 租户id |
| locations | 节点坐标信息 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 4. 工作流定义日志表：t_ds_process_definithon_log
`工作流定义表基础上增加操作类型（新增、修改、删除）、操作人、操作时间`
- 5. 工作流作业关系日志表：t_ds_process_task_relation_log
`作业关系表基础上增加工作流版本、操作类型（新增、修改、删除）、操作人、操作时间`
- 6. 作业定义日志表：t_ds_task_definithon_log
`作业定义表基础上增加操作类型（新增、修改、删除）、操作人、操作时间`
- 7. 工作流实例表修改：t_ds_process_instance
`新增工作流定义版本号字段`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] lenboo commented on issue #4325: [Feature][JsonSplit] Propose for process definithon json split
Posted by GitBox <gi...@apache.org>.
lenboo commented on issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325#issuecomment-751674549
#### 4. master refactoring
`Replace the existing workflow/task structure in master with a new structure definition`
### 4. master重构
`将master里现有的工作流/任务结构，换成新的结构定义`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] lenboo commented on issue #4325: [Feature][JsonSplit] Propose for process definithon json split
Posted by GitBox <gi...@apache.org>.
lenboo commented on issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325#issuecomment-751673724
### 1. New tables and data structures, DB query interface (new structures do not delete/modify existing workflow/task structures at this time):
- 1. New Workflow Definition Table: t_ds_Process_Definithon
```
|Column Name|Description|
| -------------------------------| ---------------- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| task_type | job type |
| task_params | Job Custom Parameters |
| run_flag | run flag |
| task_priority | job priority |
| worker_group | worker grouping |
| fail_retry_times | Number of failed retries |
| fail_retry_interval | failed retry interval |
| timeout_flag | timeout flag |
| timeout_notify_strategy | Timeout Notification Policy |
| timeout_duration | timeout length |
| create_time | creation time |
| update_time | modification time |
```
- 2. Workflow Job Relationships Table: t_ds_Process_Task_Relation
```
|Column Name|Description|
| -------------------------------| ---------------- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| task_type | job type |
| task_params | Job Custom Parameters |
| run_flag | run flag |
| task_priority | job priority |
| worker_Group | worker grouping |
| fail_Retry_Times | Number of failed retries |
| fail_retry_interval | failed retry interval |
| timeout_flag | timeout flag |
| timeout_notify_strategy | Timeout Notification Policy |
| timeout_duration | timeout length |
| create_time | creation time |
| update_time | modification time |
```
- 3. Job Definition Table: t_ds_Task_Definithon
```
|Column Name|Description|
| ---- | ---- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| release_state | Publish status |
| user_id | Owner id |
| global_params | global parameters |
| flag | Is the process available: 0 not available, 1 available |
| receivers | recipients |
| receivers_CC | Cc |
| timeout | timeout |
| tenant_id | Tenant id |
| locations | node coordinate information |
| create_time | creation time |
| update_time | modification time |
```
- 4. Workflow Definition Log Table: t_ds_Process_Definithon_Log
`Add operation type (add, modify, delete), operator, operation time based on workflow definition table`
- 5. Workflow job relationship log table: t_ds_Process_Task_Relation_Log
`Add workflow version, operation type (add, modify, delete), operator, operation time based on job relationship table`
- 6. Job Definition Log Table: t_ds_Task_Definithon_Log
`Add operation type (add, modify, delete), operator, operation time based on job definition table`
- 7. Workflow Instance Table Modification: t_ds_Process_Instance
`Add Workflow Definition Version Number Field`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] lenboo edited a comment on issue #4325: [Feature][JsonSplit] Propose for process definithon json split
Posted by GitBox <gi...@apache.org>.
lenboo edited a comment on issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325#issuecomment-751673724
### 1. New tables and data structures, DB query interface (new structures do not delete/modify existing workflow/task structures at this time):
- 1. New Workflow Definition Table: t_ds_Process_Definithon
```
|Column Name|Description|
| -------------------------------| ---------------- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| task_type | job type |
| task_params | Job Custom Parameters |
| run_flag | run flag |
| task_priority | job priority |
| worker_group | worker grouping |
| fail_retry_times | Number of failed retries |
| fail_retry_interval | failed retry interval |
| timeout_flag | timeout flag |
| timeout_notify_strategy | Timeout Notification Policy |
| timeout_duration | timeout length |
| create_time | creation time |
| update_time | modification time |
```
- 2. Workflow Job Relationships Table: t_ds_Process_Task_Relation
```
| Column Name | Description |
| ----------------------- | ------------------------- ------------- |
| id | self incrementing ID |
| project_code | project code |
| process_definition_code | Workflow coding |
| pre_project_code | pre-quoted project code |
| pre_task_code | pre-reference job code |
| condition_type | condition type 0: None 1: Judgment 2: Delayed |
| condition_params | condition parameters (json) |
| post_project_code | post reference project code |
| post_task_code | post reference job code |
| create_time | creation time |
| update_time | modification time |
```
- 3. Job Definition Table: t_ds_Task_Definithon
```
|Column Name|Description|
| -------------------------------| ---------------- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| task_type | job type |
| task_params | Job Custom Parameters |
| run_flag | run flag |
| task_priority | job priority |
| worker_group | worker grouping |
| fail_retry_times | Number of failed retries |
| fail_retry_interval | failed retry interval |
| timeout_flag | timeout flag |
| timeout_notify_strategy | Timeout Notification Policy |
| timeout_duration | timeout length |
| create_time | creation time |
| update_time | modification time |
```
- 4. Workflow Definition Log Table: t_ds_Process_Definithon_Log
`Add operation type (add, modify, delete), operator, operation time based on workflow definition table`
- 5. Workflow job relationship log table: t_ds_Process_Task_Relation_Log
`Add workflow version, operation type (add, modify, delete), operator, operation time based on job relationship table`
- 6. Job Definition Log Table: t_ds_Task_Definithon_Log
`Add operation type (add, modify, delete), operator, operation time based on job definition table`
- 7. Workflow Instance Table Modification: t_ds_Process_Instance
`Add Workflow Definition Version Number Field`
### 1. 新增表以及数据结构，db查询接口(新增结构暂时不要删除/修改现有的工作流/任务结构):
- 1. 新增工作流定义表：t_ds_process_definithon
```
| 列名 | 描述 |
| ---- | ---- |
| id | 自增ID |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| release_state | 发布状态 |
| user_id | 所属用户ID |
| global_params | 全局参数 |
| flag | 流程是否可用：0 不可用，1 可用 |
|alert_group_id|告警人分组id |
| timeout | 超时时间 |
| tenant_id | 租户ID |
| locations | 节点坐标信息 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 2. 工作流作业关系表：t_ds_process_task_relation
```
| 列名 | 描述 |
| ----------------------- | -------------------------------------- |
| id | 自增ID |
| project_code | 项目编码 |
| process_definition_code | 工作流编码 |
| pre_project_code | 前置引用项目编码 |
| pre_task_code | 前置引用作业编码 |
| condition_type | 条件类型 0：无 1：判断 2：延迟 |
| condition_params | 条件参数（json） |
| post_project_code | 后置引用项目编码 |
| post_task_code | 后置引用作业编码 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 3. 作业定义表：t_ds_task_definithon
```
| 列名 | 描述 |
| ----------------------- | -------------- |
| id | 自增id |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| task_type | 作业类型 |
| task_params | 作业自定义参数 |
| run_flag | 运行标志 |
| task_priority | 作业优先级 |
| worker_group | worker分组 |
| fail_retry_times | 失败重试次数 |
| fail_retry_interval | 失败重试间隔 |
| timeout_flag | 超时标志 |
| timeout_notify_strategy | 超时通知策略 |
| timeout_duration | 超时时长 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 4. 工作流定义日志表：t_ds_process_definithon_log
`工作流定义表基础上增加操作类型（新增、修改、删除）、操作人、操作时间`
- 5. 工作流作业关系日志表：t_ds_process_task_relation_log
`作业关系表基础上增加工作流版本、操作类型（新增、修改、删除）、操作人、操作时间`
- 6. 作业定义日志表：t_ds_task_definithon_log
`作业定义表基础上增加操作类型（新增、修改、删除）、操作人、操作时间`
- 7. 工作流实例表修改：t_ds_process_instance
`新增工作流定义版本号字段`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] lenboo edited a comment on issue #4325: [Feature][JsonSplit] Propose for process definithon json split
Posted by GitBox <gi...@apache.org>.
lenboo edited a comment on issue #4325:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4325#issuecomment-751673724
### 1. New tables and data structures, DB query interface (new structures do not delete/modify existing workflow/task structures at this time):
- 1. New Workflow Definition Table: t_ds_Process_Definithon
```
|Column Name|Description|
| -------------------------------| ---------------- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| task_type | job type |
| task_params | Job Custom Parameters |
| run_flag | run flag |
| task_priority | job priority |
| worker_group | worker grouping |
| fail_retry_times | Number of failed retries |
| fail_retry_interval | failed retry interval |
| timeout_flag | timeout flag |
| timeout_notify_strategy | Timeout Notification Policy |
| timeout_duration | timeout length |
| create_time | creation time |
| update_time | modification time |
```
- 2. Workflow Job Relationships Table: t_ds_Process_Task_Relation
```
|Column Name|Description|
| -------------------------------| ---------------- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| task_type | job type |
| task_params | Job Custom Parameters |
| run_flag | run flag |
| task_priority | job priority |
| worker_Group | worker grouping |
| fail_Retry_Times | Number of failed retries |
| fail_retry_interval | failed retry interval |
| timeout_flag | timeout flag |
| timeout_notify_strategy | Timeout Notification Policy |
| timeout_duration | timeout length |
| create_time | creation time |
| update_time | modification time |
```
- 3. Job Definition Table: t_ds_Task_Definithon
```
|Column Name|Description|
| ---- | ---- |
| id | Self-Increasing id |
| code | encoding (original name field)|
| version | version |
| description | description |
| project_code | project code |
| release_state | Publish status |
| user_id | Owner id |
| global_params | global parameters |
| flag | Is the process available: 0 not available, 1 available |
| receivers | recipients |
| warning_group_id | warning_group_id |
| timeout | timeout |
| tenant_id | Tenant id |
| locations | node coordinate information |
| create_time | creation time |
| update_time | modification time |
```
- 4. Workflow Definition Log Table: t_ds_Process_Definithon_Log
`Add operation type (add, modify, delete), operator, operation time based on workflow definition table`
- 5. Workflow job relationship log table: t_ds_Process_Task_Relation_Log
`Add workflow version, operation type (add, modify, delete), operator, operation time based on job relationship table`
- 6. Job Definition Log Table: t_ds_Task_Definithon_Log
`Add operation type (add, modify, delete), operator, operation time based on job definition table`
- 7. Workflow Instance Table Modification: t_ds_Process_Instance
`Add Workflow Definition Version Number Field`
### 1. 新增表以及数据结构，db查询接口(新增结构暂时不要删除/修改现有的工作流/任务结构):
- 1. 新增工作流定义表：t_ds_process_definithon
```
| 列名 | 描述 |
| ---- | ---- |
| id | 自增ID |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| release_state | 发布状态 |
| user_id | 所属用户ID |
| global_params | 全局参数 |
| flag | 流程是否可用：0 不可用，1 可用 |
| receivers | 收件人 |
| receivers_cc | 抄送人 |
| timeout | 超时时间 |
| tenant_id | 租户ID |
| locations | 节点坐标信息 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 2. 工作流作业关系表：t_ds_process_task_relation
```
| 列名 | 描述 |
| ----------------------- | -------------------------------------- |
| id | 自增ID |
| project_code | 项目编码 |
| process_definition_code | 工作流编码 |
| pre_project_code | 前置引用项目编码 |
| pre_task_code | 前置引用作业编码 |
| condition_type | 条件类型 0：无 1：判断 2：延迟 |
| condition_params | 条件参数（json） |
| post_project_code | 后置引用项目编码 |
| post_task_code | 后置引用作业编码 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 3. 作业定义表：t_ds_task_definithon
```
| 列名 | 描述 |
| ---- | ---- |
| id | 自增id |
| code | 编码（原name字段） |
| version | 版本 |
| description | 描述 |
| project_code | 项目编码 |
| release_state | 发布状态 |
| user_id | 所属用户id |
| global_params | 全局参数 |
| flag | 流程是否可用：0 不可用，1 可用 |
| warning_group_id | 告警人 |
| timeout | 超时时间 |
| tenant_id | 租户id |
| locations | 节点坐标信息 |
| create_time | 创建时间 |
| update_time | 修改时间 |
```
- 4. 工作流定义日志表：t_ds_process_definithon_log
`工作流定义表基础上增加操作类型（新增、修改、删除）、操作人、操作时间`
- 5. 工作流作业关系日志表：t_ds_process_task_relation_log
`作业关系表基础上增加工作流版本、操作类型（新增、修改、删除）、操作人、操作时间`
- 6. 作业定义日志表：t_ds_task_definithon_log
`作业定义表基础上增加操作类型（新增、修改、删除）、操作人、操作时间`
- 7. 工作流实例表修改：t_ds_process_instance
`新增工作流定义版本号字段`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org