DISCUSS] Import and export changes too much #3937

[boyi <zh...@163.com>]

In order to support import and export, the code field is added. 

The code part has been basically completed

But there are big changes and risks




We need to discuss how to deal with it




issues:

https://github.com/apache/incubator-dolphinscheduler/issues/3937

code:

https://github.com/BoYiZhang/incubator-dolphinscheduler/tree/feature_3937




--------------------------------------------------------------------------

issue: #3937 

This feature is the front of feature #3820

Code field is added to support subsequent import and export functions

db table change
t_ds_project [Add field code ]

t_ds_process_definition [ Add field code ,parent_code,is_directory, update project_id -> project_code , resource_ids -> resource_codes ]

t_ds_process_definition_version [ update process_definition_id -> process_definition_code , resource_ids -> resource_codes ]

t_ds_datasource [Add field code]
t_ds_udfs [add field code, update field resource_id -> resource_code ]

SQL change script :



use dolphinscheduler ;

-- 1. 更改数据库表结构
ALTER TABLE `t_ds_datasource` ADD COLUMN `code` VARCHAR(128)  NOT NULL COMMENT '编码' AFTER `id` ;

ALTER TABLE `t_ds_resources` ADD COLUMN `code` VARCHAR(128) NOT NULL COMMENT '唯一编码' AFTER `id` ;
ALTER TABLE `t_ds_resources` CHANGE COLUMN `pid` `parent_code` VARCHAR(128) DEFAULT '-1' COMMENT 'parent code' ;

ALTER TABLE `t_ds_udfs` ADD COLUMN `code` VARCHAR(128) NOT NULL COMMENT '唯一编码' AFTER `id` ;
ALTER TABLE `t_ds_udfs` CHANGE COLUMN `resource_id` `resource_code` VARCHAR(128) NOT NULL COMMENT 'resource code'  ;

ALTER TABLE `t_ds_project` ADD COLUMN `code` VARCHAR(128) NOT NULL COMMENT '项目唯一编码' AFTER `id` ;

ALTER TABLE `t_ds_process_definition` ADD COLUMN `code` VARCHAR(128) NOT NULL COMMENT '全局变量唯一编码' AFTER `id` ;
ALTER TABLE `t_ds_process_definition` CHANGE COLUMN `project_id` `project_code` VARCHAR(128) NOT NULL  COMMENT 'project code'  ;
ALTER TABLE `t_ds_process_definition` CHANGE `resource_ids` `resource_codes` VARCHAR(128) ;


ALTER TABLE `t_ds_process_definition_version` CHANGE `process_definition_id` `process_definition_code` VARCHAR(128) ;
ALTER TABLE `t_ds_process_definition_version` CHANGE `resource_ids` `resource_codes` VARCHAR(128) ;

ALTER TABLE `t_ds_schedules` CHANGE `process_definition_id` `process_definition_code` VARCHAR(128) ;




-- 2.构建唯一索引
ALTER TABLE `t_ds_datasource`  ADD UNIQUE INDEX `code_UNIQUE` (`code` ASC)  ;

ALTER TABLE `t_ds_resources`  ADD UNIQUE INDEX `code_UNIQUE` (`code` ASC) ;

ALTER TABLE `t_ds_udfs` ADD UNIQUE INDEX `code_UNIQUE` (`code` ASC) ;

ALTER TABLE `t_ds_project` ADD UNIQUE INDEX `code_UNIQUE` (`code` ASC) ;

ALTER TABLE `t_ds_process_definition`  ADD UNIQUE INDEX `process_definition_unique_code` (`code` ASC, `project_code` ASC);


-- 3.升级之前的历史数据 [如果没有数据,则不做变动]

UPDATE `t_ds_datasource` SET code = id ;
UPDATE `t_ds_resources` SET code = id ;
UPDATE `t_ds_udfs` SET code = id ;
UPDATE `t_ds_project` SET code = id ;
UPDATE `t_ds_process_definition` SET code = id  ;



// 4. 升级工作流定义中的关于依赖任务和子流程定义中的数据结构[如果没有数据,则不做变动]

这部分需要运行升级代码










issues:

https://github.com/apache/incubator-dolphinscheduler/issues/3937

code:

https://github.com/BoYiZhang/incubator-dolphinscheduler/tree/feature_3937









--------------------------------------
BoYi ZhangE-mail : zhangboyi_mx@163.com