[DISCUSS] Import and export changes too much #3937

[boyi <zh...@163.com>]

hi:


In order to support the import and export of workflow definition, data source and resource file, code field is added


But found a lot of changes, please give your opinion, discuss how to do


The address of issue and code is as follows:


—————————————————————————————————————

为了支持工作流定义,数据源,资源文件的导入导出, 增加了code字段.
但是发现改动很多,麻烦各位发表一下意见,讨论应该怎么做.
issue和code地址如下:

—————————————————————————————————————

issues:

https://github.com/apache/incubator-dolphinscheduler/issues/3937

code:

https://github.com/BoYiZhang/incubator-dolphinscheduler/tree/feature_3937




--------------------------------------------------------------------------

issue: #3937 

This feature is the front of feature #3820

Code field is added to support subsequent import and export functions

db table change
t_ds_project [Add field code ]

t_ds_process_definition [ Add field code ,parent_code,is_directory, update project_id -> project_code , resource_ids -> resource_codes ]

t_ds_process_definition_version [ update process_definition_id -> process_definition_code , resource_ids -> resource_codes ]

t_ds_datasource [Add field code]
t_ds_udfs [add field code, update field resource_id -> resource_code ]



—————————————————————————————————————





SQL change script :



use dolphinscheduler ;

-- 1. 更改数据库表结构
ALTER TABLE `t_ds_datasource` ADD COLUMN `code` VARCHAR(128)  NOT NULL COMMENT '编码' AFTER `id` ;

ALTER TABLE `t_ds_resources` ADD COLUMN `code` VARCHAR(128) NOT NULL COMMENT '唯一编码' AFTER `id` ;
ALTER TABLE `t_ds_resources` CHANGE COLUMN `pid` `parent_code` VARCHAR(128) DEFAULT '-1' COMMENT 'parent code' ;

ALTER TABLE `t_ds_udfs` ADD COLUMN `code` VARCHAR(128) NOT NULL COMMENT '唯一编码' AFTER `id` ;
ALTER TABLE `t_ds_udfs` CHANGE COLUMN `resource_id` `resource_code` VARCHAR(128) NOT NULL COMMENT 'resource code'  ;

ALTER TABLE `t_ds_project` ADD COLUMN `code` VARCHAR(128) NOT NULL COMMENT '项目唯一编码' AFTER `id` ;

ALTER TABLE `t_ds_process_definition` ADD COLUMN `code` VARCHAR(128) NOT NULL COMMENT '全局变量唯一编码' AFTER `id` ;
ALTER TABLE `t_ds_process_definition` CHANGE COLUMN `project_id` `project_code` VARCHAR(128) NOT NULL  COMMENT 'project code'  ;
ALTER TABLE `t_ds_process_definition` CHANGE `resource_ids` `resource_codes` VARCHAR(128) ;


ALTER TABLE `t_ds_process_definition_version` CHANGE `process_definition_id` `process_definition_code` VARCHAR(128) ;
ALTER TABLE `t_ds_process_definition_version` CHANGE `resource_ids` `resource_codes` VARCHAR(128) ;

ALTER TABLE `t_ds_schedules` CHANGE `process_definition_id` `process_definition_code` VARCHAR(128) ;




-- 2.构建唯一索引
ALTER TABLE `t_ds_datasource`  ADD UNIQUE INDEX `code_UNIQUE` (`code` ASC)  ;

ALTER TABLE `t_ds_resources`  ADD UNIQUE INDEX `code_UNIQUE` (`code` ASC) ;

ALTER TABLE `t_ds_udfs` ADD UNIQUE INDEX `code_UNIQUE` (`code` ASC) ;

ALTER TABLE `t_ds_project` ADD UNIQUE INDEX `code_UNIQUE` (`code` ASC) ;

ALTER TABLE `t_ds_process_definition`  ADD UNIQUE INDEX `process_definition_unique_code` (`code` ASC, `project_code` ASC);


-- 3.升级之前的历史数据 [如果没有数据,则不做变动]

UPDATE `t_ds_datasource` SET code = id ;
UPDATE `t_ds_resources` SET code = id ;
UPDATE `t_ds_udfs` SET code = id ;
UPDATE `t_ds_project` SET code = id ;
UPDATE `t_ds_process_definition` SET code = id  ;



// 4. 升级工作流定义中的关于依赖任务和子流程定义中的数据结构[如果没有数据,则不做变动]

这部分需要运行升级代码










issues:

https://github.com/apache/incubator-dolphinscheduler/issues/3937

code:

https://github.com/BoYiZhang/incubator-dolphinscheduler/tree/feature_3937