You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@dolphinscheduler.apache.org by yongqing <59...@qq.com> on 2020/12/21 06:14:06 UTC
回复： [DISCUSS]Code solution for project/workflow/task
+1
------------------ 原始邮件 ------------------
发件人: "dev" <dailidong66@gmail.com>;
发送时间: 2020年12月15日(星期二) 下午3:53
收件人: "dev"<dev@dolphinscheduler.apache.org>;
主题: Re: [DISCUSS]Code solution for project/workflow/task
hi developers，
8 days have passed. we have reached a conclusion that Schema 3 will be
our final solution for the project/workflow/task.
thanks everybody
Best Regards
---------------
DolphinScheduler(Incubator) PPMC
Lidong Dai 代立冬
dailidong66@gmail.com
---------------
On Wed, Dec 9, 2020 at 7:18 PM 3656562 <3656562@qq.com> wrote:
> Schema 3 + 1
>
>
> 原始邮件
> 发件人:leon baoleonbao@apache.org
> 收件人:devdev@dolphinscheduler.apache.org
> 发送时间:2020年12月7日(周一) 23:56
> 主题:[DISCUSS]Code solution for project/workflow/task
>
>
> ##1. Background *It is suggested to take a look at the design scheme of
> splitting workflow JSON* the mail link about json splitting:
> https://lists.apache.org/thread.html/r84ed046de63899476f067dd207496428812a8687045bb4485072ee2d%40%3Cdev.dolphinscheduler.apache.org%3E
> The current business coding is proposed to be designed on the basis of
> splitting workflow JSON, with three purposes: -The references among
> workflow, relationship and job, and the external reference of dependent job
> and sub workflow job are described by project code, workflow code and
> business code -It supports the version management of workflow, job and
> resource, and provides a stable business code -Solve the problems of
> workflow, job copy, migration, export and import scene ID auto increment,
> conflict and so on ##2. Coding design ###2.1 scheme I Project table,
> workflow table and job table introduce a new business code field User
> manual input / automatically generated by the system. Here, the UI prompts
> the user to input the code. Internal reference is established through the
> code. Subsequent modification costs are high Project code is globally
> unique; workflow code and job code are project level unique Because the
> possibility of business code modification is very low, all codes in the
> current design do not support modification. The purpose of code
> modification can be achieved through the function of export and import When
> the workflow and job are copied, the user needs to input the new business
> code, and the system will check the conflict When the workflow and job are
> exported and imported, the business code (which can be edited) is displayed
> on the UI, and the user can choose whether to change the business code;
> when exporting, the UI adds the option of whether to export dependent jobs
> at the same time When the workflow and job are migrated, the business code
> (which can be edited) is displayed on the UI, and the user can choose
> whether to change the business code; the project code can be directly
> updated after the migration ###2.2 scheme 2 On the basis of scheme 1, there
> will be three description words: code, name, description. Code and name are
> likely to be repeatedly entered in practical application, while description
> is rarely used. Therefore, the current scheme is to replace code with name
> without adding a code field. As for the input of English name or Chinese
> name, it is up to the user to decide whether to input English name or
> Chinese name, and whether to use description for specific function
> description field ###2.3 scheme 3 Project code, workflow code and job code
> are globally unique. They are only responsible for the uniqueness of
> identification data and are not related to business. They are automatically
> generated by the system and do not provide page modification methods. 1.
> Workflow, job create new workflow (task) / copy workflow from template:
> generate new definition_ code/task_ code 2. Export: export exports all the
> contents of the current project / workflow / task as a JSON file 3. Import:
> check the definition_code/task_code exist -If it is a duplicate of
> definition_code/task_code in another project , you will be prompted and the
> import failed. -If it is the same as the definition_code/task_code in this
> project , you will be prompted to override -If there is no duplicate, all
> codes remain unchanged ##Design relation 3 The project code field is
> maintained in the workflow table to identify which project the workflow
> belongs to The relation table maintains the item code and workflow code
> fields, and identifies the workflow of which project the relationship
> belongs The relation table maintains the project code and job code of the
> reference node (supports jobs that reference other projects) The item code
> is maintained in the operation table to identify which project the job
> belongs to The referenced workflow / job is determined by project code,
> workflow code and job code The referenced workflow is determined by item
> code and workflow code in sub workflow--
> ==================================================================== ##
> 1.背景 *建议先看一下拆分工作流json的设计方案* 关联邮件地址：
> https://lists.apache.org/thread.html/r84ed046de63899476f067dd207496428812a8687045bb4485072ee2d%40%3Cdev.dolphinscheduler.apache.org%3E
> 当前业务编码是建议在拆分工作流json基础上设计的，目的有3个： -
> 工作流、关系、作业之间的引用，以及依赖作业、子工作流作业对外部的引用通过项目编码、工作流编码、业务编码进行描述 -
> 支持工作流、作业、资源的版本管理，提供一个稳定不变的业务编码 - 解决工作流、作业的复制、迁移、导出导入场景的ID自增、冲突等问题 ## 2.编码设计
> ### 2.1 方案一 项目表、工作流表、作业表引入一个新的业务编码字段
> 用户手工输入/系统自动生成，这里UI提示建议用户输入编码，内部通过编码建立引用，后续修改成本高 项目编码全局唯一；工作流编码、作业编码项目级唯一
> 因为业务编码修改的可能性非常低，所以当前设计所有编码都不支持修改，可以通过导出导入功能达到修改编码的目的
> 工作流、作业复制时，需要用户输入新业务编码，系统做冲突检查提醒
> 工作流、作业导出导入时，UI上显示业务编码（可编辑），用户可以选择是否变更业务编码；导出时UI增加是否同时导出依赖作业选项
> 工作流、作业迁移时，UI上显示业务编码（可编辑），用户可以选择是否变更业务编码；迁移直接更新项目编码就可以了 ### 2.2 方案二
> 在方案一基础上，会有三个描述字：code、name、description，code和name在实际应用中很可能是重复输入的，而description又很少应用，所以当前方案是用name替换code，不需要新增code字段，至于输入英文name还是中文name由用户自己决定，段具体的功能描述还是用description字段
> ### 2.3 方案三 项目编码、工作流编码、作业编码全局唯一，只负责标识数据的唯一，不和业务相关，由系统自动生成，不提供页面修改方式。 1.
> 工作流、作业从模板创建新工作流（任务）/复制工作流: 生成新的definition_code/task_code 2. 导出：
> 导出将当前项目/工作流/任务所有内容导出为json文件 3. 导入： 检查definition_code/task_code是否存在 -
> 如果和别的项目的definition_code/task_code重复，则提示并导入失败。 -
> 如果和本项目内definition_code/task_code重复，则提示覆盖 - 如果没有重复的，则所有code不变导入 ## 3.引用关系设计
> 工作流表维护项目编码字段，标识工作流归属于哪个项目 关系表维护项目编码、工作流编码字段，标识关系归属于哪个项目的工作流
> 关系表维护引用节点的项目编码、作业编码（支持引用其他项目的作业） 作业表维护项目编码，标识作业归属于哪个项目
> 依赖作业内部通过项目编码、工作流编码、作业编码确定引用的工作流/作业 子工作流内部通过项目编码、工作流编码确定引用的工作流
> DolphinScheduler(Incubator) PPMC BaoLiang 鲍亮 leonbao@apache.org