You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/09/09 16:11:22 UTC
[GitHub] [dolphinscheduler] wendongdi opened a new pull request, #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
wendongdi opened a new pull request, #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887
<!--Thanks very much for contributing to Apache DolphinScheduler. Please review https://dolphinscheduler.apache.org/en-us/community/development/pull-request.html before opening a pull request.-->
## Purpose of the pull request
fix #11875
## Brief change log
defining procedures to determine whether an index or column already exists, and if it exists, delete it and then create a new one
- dolphinscheduler-dao/src/main/resources/sql/upgrade/2.0.3_schema/mysql/dolphinscheduler_ddl.sql
- dolphinscheduler-dao/src/main/resources/sql/upgrade/2.0.4_schema/mysql/dolphinscheduler_ddl.sql
- dolphinscheduler-dao/src/main/resources/sql/upgrade/3.1.1_schema/mysql/dolphinscheduler_ddl.sql
fix bugs
- dolphinscheduler-dao/src/main/resources/sql/upgrade/3.1.0_schema/mysql/dolphinscheduler_ddl.sql
## Verify this pull request
This pull request is already covered by existing tests:
test steps:
1. create an empty mysql database
2. run `org.apache.dolphinscheduler.tools.datasource.UpgradeDolphinScheduler` with profile 'init'
3. run `org.apache.dolphinscheduler.tools.datasource.UpgradeDolphinScheduler` with profile 'upgrade'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on a diff in pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#discussion_r969104279
##########
dolphinscheduler-dao/src/main/resources/sql/upgrade/3.1.1_schema/mysql/dolphinscheduler_ddl.sql:
##########
@@ -16,7 +16,34 @@
*/
-ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
-ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+drop procedure if exists add_column_if_not_exists;
+delimiter d//
+create procedure add_column_if_not_exists(target_table_name varchar(256), target_column varchar(256),
+ add_statement varchar(256))
+begin
+ declare target_database varchar(256);
+ select database() into target_database;
+ IF EXISTS(SELECT *
+ FROM information_schema.COLUMNS
+ WHERE COLUMN_NAME = target_column
+ AND TABLE_NAME = target_table_name
+ ) THEN
+ set @statement = concat('alter table ',target_table_name,' drop column ',target_column);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+ END IF;
+ set @statement = concat(add_statement);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+end;
+d//
+delimiter ;
+-- ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
+-- ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+
+call add_column_if_not_exists('t_ds_worker_group','other_params_json',"ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json'");
+call add_column_if_not_exists('t_ds_process_instance','state_history',"ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`");
Review Comment:
this PR using statement `add_column_if_not_exists('t_ds_worker_group','other_params_json',"ALTER TABLE t_ds_worker_group ADD COLUMN other_params_json text DEFAULT NULL COMMENT 'other params json'");`
I think the `ALTER TABLE t_ds_worker_group ADD COLUMN other_params_json` is unnecessary because we already declaration `t_ds_worker_group` and `other_params_json` in previous, maybe we should directly use `add_column_if_not_exists('t_ds_worker_group','other_params_json',"text DEFAULT NULL COMMENT 'other params json'");`
mean
```diff
- add_column_if_not_exists('t_ds_worker_group','other_params_json',"ALTER TABLE t_ds_worker_group ADD COLUMN other_params_json text DEFAULT NULL COMMENT 'other params json'");
+ add_column_if_not_exists('t_ds_worker_group','other_params_json',"text DEFAULT NULL COMMENT 'other params json'");
```
##########
dolphinscheduler-dao/src/main/resources/sql/upgrade/3.1.1_schema/mysql/dolphinscheduler_ddl.sql:
##########
@@ -16,7 +16,34 @@
*/
-ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
-ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+drop procedure if exists add_column_if_not_exists;
+delimiter d//
+create procedure add_column_if_not_exists(target_table_name varchar(256), target_column varchar(256),
+ add_statement varchar(256))
+begin
+ declare target_database varchar(256);
+ select database() into target_database;
+ IF EXISTS(SELECT *
+ FROM information_schema.COLUMNS
+ WHERE COLUMN_NAME = target_column
+ AND TABLE_NAME = target_table_name
+ ) THEN
+ set @statement = concat('alter table ',target_table_name,' drop column ',target_column);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+ END IF;
+ set @statement = concat(add_statement);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+end;
+d//
+delimiter ;
+-- ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
+-- ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+
+call add_column_if_not_exists('t_ds_worker_group','other_params_json',"ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json'");
+call add_column_if_not_exists('t_ds_process_instance','state_history',"ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`");
Review Comment:
this PR using statement `add_column_if_not_exists('t_ds_worker_group','other_params_json',"ALTER TABLE t_ds_worker_group ADD COLUMN other_params_json text DEFAULT NULL COMMENT 'other params json'");`
I think the `ALTER TABLE t_ds_worker_group ADD COLUMN other_params_json` is unnecessary because we already declaration `t_ds_worker_group` and `other_params_json` in previous, maybe we should directly use `add_column_if_not_exists('t_ds_worker_group','other_params_json',"text DEFAULT NULL COMMENT 'other params json'");`
means
```diff
- add_column_if_not_exists('t_ds_worker_group','other_params_json',"ALTER TABLE t_ds_worker_group ADD COLUMN other_params_json text DEFAULT NULL COMMENT 'other params json'");
+ add_column_if_not_exists('t_ds_worker_group','other_params_json',"text DEFAULT NULL COMMENT 'other params json'");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#issuecomment-1242597338
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=11887)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=11887&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=11887&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on a diff in pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on code in PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#discussion_r969105966
##########
dolphinscheduler-dao/src/main/resources/sql/upgrade/3.1.1_schema/mysql/dolphinscheduler_ddl.sql:
##########
@@ -16,7 +16,34 @@
*/
-ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
-ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+drop procedure if exists add_column_if_not_exists;
+delimiter d//
+create procedure add_column_if_not_exists(target_table_name varchar(256), target_column varchar(256),
+ add_statement varchar(256))
+begin
+ declare target_database varchar(256);
+ select database() into target_database;
+ IF EXISTS(SELECT *
+ FROM information_schema.COLUMNS
+ WHERE COLUMN_NAME = target_column
+ AND TABLE_NAME = target_table_name
+ ) THEN
+ set @statement = concat('alter table ',target_table_name,' drop column ',target_column);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+ END IF;
+ set @statement = concat(add_statement);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+end;
+d//
+delimiter ;
+-- ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
+-- ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+
+call add_column_if_not_exists('t_ds_worker_group','other_params_json',"ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json'");
+call add_column_if_not_exists('t_ds_process_instance','state_history',"ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`");
Review Comment:
Agree with jiajie
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on a diff in pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#discussion_r969080887
##########
dolphinscheduler-dao/src/main/resources/sql/upgrade/3.1.1_schema/mysql/dolphinscheduler_ddl.sql:
##########
@@ -16,7 +16,34 @@
*/
-ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
-ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+drop procedure if exists add_column_if_not_exists;
+delimiter d//
+create procedure add_column_if_not_exists(target_table_name varchar(256), target_column varchar(256),
+ add_statement varchar(256))
+begin
+ declare target_database varchar(256);
+ select database() into target_database;
+ IF EXISTS(SELECT *
+ FROM information_schema.COLUMNS
+ WHERE COLUMN_NAME = target_column
+ AND TABLE_NAME = target_table_name
+ ) THEN
+ set @statement = concat('alter table ',target_table_name,' drop column ',target_column);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+ END IF;
+ set @statement = concat(add_statement);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+end;
+d//
+delimiter ;
+-- ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
+-- ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+
+call add_column_if_not_exists('t_ds_worker_group','other_params_json',"ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json'");
+call add_column_if_not_exists('t_ds_process_instance','state_history',"ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`");
Review Comment:
do you have any idea about this? @caishunfeng @ruanwenjun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on a diff in pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#discussion_r969074721
##########
dolphinscheduler-dao/src/main/resources/sql/upgrade/3.1.1_schema/mysql/dolphinscheduler_ddl.sql:
##########
@@ -16,7 +16,34 @@
*/
-ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
-ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+drop procedure if exists add_column_if_not_exists;
+delimiter d//
+create procedure add_column_if_not_exists(target_table_name varchar(256), target_column varchar(256),
+ add_statement varchar(256))
+begin
+ declare target_database varchar(256);
+ select database() into target_database;
+ IF EXISTS(SELECT *
+ FROM information_schema.COLUMNS
+ WHERE COLUMN_NAME = target_column
+ AND TABLE_NAME = target_table_name
+ ) THEN
+ set @statement = concat('alter table ',target_table_name,' drop column ',target_column);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+ END IF;
+ set @statement = concat(add_statement);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+end;
+d//
+delimiter ;
+-- ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
+-- ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+
+call add_column_if_not_exists('t_ds_worker_group','other_params_json',"ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json'");
+call add_column_if_not_exists('t_ds_process_instance','state_history',"ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`");
Review Comment:
in add column procedure function, we using the whole alert statement `ALTER TABLE t_ds_worker_group ADD COLUMN other_params_json`, but the variable is already in previous parameter named `t_ds_worker_group` and `other_params_json`, I am not sure should we directly using the whole statement or should we only use the suffix like `text DEFAULT NULL COMMENT 'other params json'`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter commented on pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#issuecomment-1242595211
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/11887?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#11887](https://codecov.io/gh/apache/dolphinscheduler/pull/11887?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (03b611a) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/3664d85143a319e3319cf9bd09108e87492c75bf?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3664d85) will **decrease** coverage by `0.03%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## dev #11887 +/- ##
============================================
- Coverage 38.55% 38.51% -0.04%
+ Complexity 4037 4030 -7
============================================
Files 993 993
Lines 36698 36670 -28
Branches 4273 4267 -6
============================================
- Hits 14148 14123 -25
+ Misses 20923 20922 -1
+ Partials 1627 1625 -2
```
| [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/11887?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...eduler/server/worker/task/WorkerHeartBeatTask.java](https://codecov.io/gh/apache/dolphinscheduler/pull/11887/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci13b3JrZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL1dvcmtlckhlYXJ0QmVhdFRhc2suamF2YQ==) | `68.08% <0.00%> (-8.52%)` | :arrow_down: |
| [...erver/master/processor/queue/TaskEventService.java](https://codecov.io/gh/apache/dolphinscheduler/pull/11887/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1tYXN0ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9wcm9jZXNzb3IvcXVldWUvVGFza0V2ZW50U2VydmljZS5qYXZh) | `75.00% <0.00%> (-5.36%)` | :arrow_down: |
| [...org/apache/dolphinscheduler/remote/utils/Host.java](https://codecov.io/gh/apache/dolphinscheduler/pull/11887/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1yZW1vdGUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcmVtb3RlL3V0aWxzL0hvc3QuamF2YQ==) | `42.55% <0.00%> (-2.13%)` | :arrow_down: |
| [...dolphinscheduler/remote/future/ResponseFuture.java](https://codecov.io/gh/apache/dolphinscheduler/pull/11887/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1yZW1vdGUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcmVtb3RlL2Z1dHVyZS9SZXNwb25zZUZ1dHVyZS5qYXZh) | `81.96% <0.00%> (-1.64%)` | :arrow_down: |
| [...e/dolphinscheduler/remote/NettyRemotingClient.java](https://codecov.io/gh/apache/dolphinscheduler/pull/11887/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1yZW1vdGUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcmVtb3RlL05ldHR5UmVtb3RpbmdDbGllbnQuamF2YQ==) | `51.38% <0.00%> (-1.39%)` | :arrow_down: |
| [...che/dolphinscheduler/common/utils/HadoopUtils.java](https://codecov.io/gh/apache/dolphinscheduler/pull/11887/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL3V0aWxzL0hhZG9vcFV0aWxzLmphdmE=) | `15.94% <0.00%> (-0.73%)` | :arrow_down: |
| [.../apache/dolphinscheduler/common/utils/OSUtils.java](https://codecov.io/gh/apache/dolphinscheduler/pull/11887/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL3V0aWxzL09TVXRpbHMuamF2YQ==) | `34.14% <0.00%> (-0.40%)` | :arrow_down: |
| [...uler/api/service/impl/DataAnalysisServiceImpl.java](https://codecov.io/gh/apache/dolphinscheduler/pull/11887/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvYXBpL3NlcnZpY2UvaW1wbC9EYXRhQW5hbHlzaXNTZXJ2aWNlSW1wbC5qYXZh) | `81.51% <0.00%> (-0.31%)` | :arrow_down: |
| [...heduler/api/service/impl/ResourcesServiceImpl.java](https://codecov.io/gh/apache/dolphinscheduler/pull/11887/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvYXBpL3NlcnZpY2UvaW1wbC9SZXNvdXJjZXNTZXJ2aWNlSW1wbC5qYXZh) | `52.26% <0.00%> (-0.21%)` | :arrow_down: |
| [...pache/dolphinscheduler/common/utils/DateUtils.java](https://codecov.io/gh/apache/dolphinscheduler/pull/11887/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL3V0aWxzL0RhdGVVdGlscy5qYXZh) | `74.21% <0.00%> (-0.17%)` | :arrow_down: |
| ... and [11 more](https://codecov.io/gh/apache/dolphinscheduler/pull/11887/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#discussion_r969076941
##########
dolphinscheduler-dao/src/main/resources/sql/upgrade/3.1.1_schema/mysql/dolphinscheduler_ddl.sql:
##########
@@ -16,7 +16,34 @@
*/
-ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
-ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+drop procedure if exists add_column_if_not_exists;
+delimiter d//
+create procedure add_column_if_not_exists(target_table_name varchar(256), target_column varchar(256),
+ add_statement varchar(256))
+begin
+ declare target_database varchar(256);
+ select database() into target_database;
+ IF EXISTS(SELECT *
+ FROM information_schema.COLUMNS
+ WHERE COLUMN_NAME = target_column
+ AND TABLE_NAME = target_table_name
+ ) THEN
+ set @statement = concat('alter table ',target_table_name,' drop column ',target_column);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+ END IF;
+ set @statement = concat(add_statement);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+end;
+d//
+delimiter ;
+-- ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
+-- ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+
+call add_column_if_not_exists('t_ds_worker_group','other_params_json',"ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json'");
+call add_column_if_not_exists('t_ds_process_instance','state_history',"ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`");
Review Comment:
I'm not quite sure about it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wendongdi commented on pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
wendongdi commented on PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#issuecomment-1246486182
@zhongjiajie big bro，what do u think of these changes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#issuecomment-1245097434
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=11887)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=11887&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=11887&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=11887&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#issuecomment-1245053136
I had approve to run the CI test.
@zhongjiajie do you have time to take a look aggin?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#issuecomment-1246530278
> @zhongjiajie big bro，what do u think of these changes?
LGTM, thanks for your contribution
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on a diff in pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#discussion_r969074721
##########
dolphinscheduler-dao/src/main/resources/sql/upgrade/3.1.1_schema/mysql/dolphinscheduler_ddl.sql:
##########
@@ -16,7 +16,34 @@
*/
-ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
-ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+drop procedure if exists add_column_if_not_exists;
+delimiter d//
+create procedure add_column_if_not_exists(target_table_name varchar(256), target_column varchar(256),
+ add_statement varchar(256))
+begin
+ declare target_database varchar(256);
+ select database() into target_database;
+ IF EXISTS(SELECT *
+ FROM information_schema.COLUMNS
+ WHERE COLUMN_NAME = target_column
+ AND TABLE_NAME = target_table_name
+ ) THEN
+ set @statement = concat('alter table ',target_table_name,' drop column ',target_column);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+ END IF;
+ set @statement = concat(add_statement);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+end;
+d//
+delimiter ;
+-- ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
+-- ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+
+call add_column_if_not_exists('t_ds_worker_group','other_params_json',"ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json'");
+call add_column_if_not_exists('t_ds_process_instance','state_history',"ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`");
Review Comment:
in add column procedure function, we using the whole alert statement `ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json`, but the variable is already in previous parameter named `t_ds_worker_group` and `other_params_json`, I am not sure should we directly using the whole statement or should we only use the suffix like `text DEFAULT NULL COMMENT 'other params json'`
##########
dolphinscheduler-dao/src/main/resources/sql/upgrade/3.1.1_schema/mysql/dolphinscheduler_ddl.sql:
##########
@@ -16,7 +16,34 @@
*/
-ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
-ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+drop procedure if exists add_column_if_not_exists;
+delimiter d//
+create procedure add_column_if_not_exists(target_table_name varchar(256), target_column varchar(256),
+ add_statement varchar(256))
+begin
+ declare target_database varchar(256);
+ select database() into target_database;
+ IF EXISTS(SELECT *
+ FROM information_schema.COLUMNS
+ WHERE COLUMN_NAME = target_column
+ AND TABLE_NAME = target_table_name
+ ) THEN
+ set @statement = concat('alter table ',target_table_name,' drop column ',target_column);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+ END IF;
+ set @statement = concat(add_statement);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+end;
+d//
+delimiter ;
+-- ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
+-- ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+
Review Comment:
also remove those lines
```suggestion
```
##########
dolphinscheduler-dao/src/main/resources/sql/upgrade/2.0.3_schema/mysql/dolphinscheduler_ddl.sql:
##########
@@ -15,7 +15,40 @@
* limitations under the License.
*/
+drop procedure if exists re_add_index;
+delimiter d//
+create procedure re_add_index(target_table_name varchar(256),
+ target_index_type varchar(8), target_index_name varchar(256),
+ target_columns varchar(512), using_str varchar(256))
+begin
+ declare target_database varchar(256);
+ select database() into target_database;
+ IF EXISTS(SELECT *
+ FROM information_schema.statistics
+ WHERE table_schema = target_database
+ AND table_name = target_table_name
+ AND index_name = target_index_name) THEN
+ set @statement = concat('drop index ', target_index_name, ' on ', target_table_name);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+ END IF;
+ set @statement =
+ concat('alter table ', target_table_name, ' add ', target_index_type, ' ', target_index_name,
+ '(', target_columns,
+ ') ', using_str);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+end;
+d//
+delimiter ;
+
ALTER TABLE `t_ds_task_instance` MODIFY COLUMN `task_params` longtext COMMENT 'job custom parameters' AFTER `app_link`;
-ALTER TABLE `t_ds_process_task_relation` ADD KEY `idx_code` (`project_code`, `process_definition_code`) USING BTREE;
-ALTER TABLE `t_ds_process_task_relation_log` ADD KEY `idx_process_code_version` (`process_definition_code`,`process_definition_version`) USING BTREE;
-ALTER TABLE `t_ds_task_definition_log` ADD INDEX `idx_code_version` (`code`,`version`) USING BTREE;
\ No newline at end of file
+# ALTER TABLE `t_ds_process_task_relation` ADD KEY `idx_code` (`project_code`, `process_definition_code`) USING BTREE;
+# ALTER TABLE `t_ds_process_task_relation_log` ADD KEY `idx_process_code_version` (`process_definition_code`,`process_definition_version`) USING BTREE;
+# ALTER TABLE `t_ds_task_definition_log` ADD INDEX `idx_code_version` (`code`,`version`) USING BTREE;
+
Review Comment:
can we remove those line if we do not need anymore?
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wendongdi commented on a diff in pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
wendongdi commented on code in PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#discussion_r969152746
##########
dolphinscheduler-dao/src/main/resources/sql/upgrade/2.0.3_schema/mysql/dolphinscheduler_ddl.sql:
##########
@@ -15,7 +15,40 @@
* limitations under the License.
*/
+drop procedure if exists re_add_index;
+delimiter d//
+create procedure re_add_index(target_table_name varchar(256),
+ target_index_type varchar(8), target_index_name varchar(256),
+ target_columns varchar(512), using_str varchar(256))
+begin
+ declare target_database varchar(256);
+ select database() into target_database;
+ IF EXISTS(SELECT *
+ FROM information_schema.statistics
+ WHERE table_schema = target_database
+ AND table_name = target_table_name
+ AND index_name = target_index_name) THEN
+ set @statement = concat('drop index ', target_index_name, ' on ', target_table_name);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+ END IF;
+ set @statement =
+ concat('alter table ', target_table_name, ' add ', target_index_type, ' ', target_index_name,
+ '(', target_columns,
+ ') ', using_str);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+end;
+d//
+delimiter ;
+
ALTER TABLE `t_ds_task_instance` MODIFY COLUMN `task_params` longtext COMMENT 'job custom parameters' AFTER `app_link`;
-ALTER TABLE `t_ds_process_task_relation` ADD KEY `idx_code` (`project_code`, `process_definition_code`) USING BTREE;
-ALTER TABLE `t_ds_process_task_relation_log` ADD KEY `idx_process_code_version` (`process_definition_code`,`process_definition_version`) USING BTREE;
-ALTER TABLE `t_ds_task_definition_log` ADD INDEX `idx_code_version` (`code`,`version`) USING BTREE;
\ No newline at end of file
+# ALTER TABLE `t_ds_process_task_relation` ADD KEY `idx_code` (`project_code`, `process_definition_code`) USING BTREE;
+# ALTER TABLE `t_ds_process_task_relation_log` ADD KEY `idx_process_code_version` (`process_definition_code`,`process_definition_version`) USING BTREE;
+# ALTER TABLE `t_ds_task_definition_log` ADD INDEX `idx_code_version` (`code`,`version`) USING BTREE;
+
Review Comment:
of course
##########
dolphinscheduler-dao/src/main/resources/sql/upgrade/3.1.1_schema/mysql/dolphinscheduler_ddl.sql:
##########
@@ -16,7 +16,34 @@
*/
-ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
-ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+drop procedure if exists add_column_if_not_exists;
+delimiter d//
+create procedure add_column_if_not_exists(target_table_name varchar(256), target_column varchar(256),
+ add_statement varchar(256))
+begin
+ declare target_database varchar(256);
+ select database() into target_database;
+ IF EXISTS(SELECT *
+ FROM information_schema.COLUMNS
+ WHERE COLUMN_NAME = target_column
+ AND TABLE_NAME = target_table_name
+ ) THEN
+ set @statement = concat('alter table ',target_table_name,' drop column ',target_column);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+ END IF;
+ set @statement = concat(add_statement);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+end;
+d//
+delimiter ;
+-- ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
+-- ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+
Review Comment:
of course
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie merged pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
zhongjiajie merged PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on pull request #11887: [Fix][db] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#issuecomment-1246567343
@zhuangchong @zhongjiajie should this pr cherry-pick to 3.0.1?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#issuecomment-1242596657
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=11887)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=11887&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=11887&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on a diff in pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#discussion_r969074878
##########
dolphinscheduler-dao/src/main/resources/sql/upgrade/3.1.1_schema/mysql/dolphinscheduler_ddl.sql:
##########
@@ -16,7 +16,34 @@
*/
-ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
-ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+drop procedure if exists add_column_if_not_exists;
+delimiter d//
+create procedure add_column_if_not_exists(target_table_name varchar(256), target_column varchar(256),
+ add_statement varchar(256))
+begin
+ declare target_database varchar(256);
+ select database() into target_database;
+ IF EXISTS(SELECT *
+ FROM information_schema.COLUMNS
+ WHERE COLUMN_NAME = target_column
+ AND TABLE_NAME = target_table_name
+ ) THEN
+ set @statement = concat('alter table ',target_table_name,' drop column ',target_column);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+ END IF;
+ set @statement = concat(add_statement);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+end;
+d//
+delimiter ;
+-- ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
+-- ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+
+call add_column_if_not_exists('t_ds_worker_group','other_params_json',"ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json'");
+call add_column_if_not_exists('t_ds_process_instance','state_history',"ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`");
Review Comment:
WDTY @SbloodyS
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wendongdi commented on a diff in pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
wendongdi commented on code in PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#discussion_r969177544
##########
dolphinscheduler-dao/src/main/resources/sql/upgrade/3.1.1_schema/mysql/dolphinscheduler_ddl.sql:
##########
@@ -16,7 +16,34 @@
*/
-ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
-ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+drop procedure if exists add_column_if_not_exists;
+delimiter d//
+create procedure add_column_if_not_exists(target_table_name varchar(256), target_column varchar(256),
+ add_statement varchar(256))
+begin
+ declare target_database varchar(256);
+ select database() into target_database;
+ IF EXISTS(SELECT *
+ FROM information_schema.COLUMNS
+ WHERE COLUMN_NAME = target_column
+ AND TABLE_NAME = target_table_name
+ ) THEN
+ set @statement = concat('alter table ',target_table_name,' drop column ',target_column);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+ END IF;
+ set @statement = concat(add_statement);
+ PREPARE STMT FROM @statement;
+ EXECUTE STMT;
+end;
+d//
+delimiter ;
+-- ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json';
+-- ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`;
+
+call add_column_if_not_exists('t_ds_worker_group','other_params_json',"ALTER TABLE `t_ds_worker_group` ADD COLUMN `other_params_json` text DEFAULT NULL COMMENT 'other params json'");
+call add_column_if_not_exists('t_ds_process_instance','state_history',"ALTER TABLE `t_ds_process_instance` ADD COLUMN `state_history` text DEFAULT NULL COMMENT 'state history desc' AFTER `state`");
Review Comment:
@SbloodyS @zhongjiajie @caishunfeng I have removed those useless lines , and recode the `add_column_if_not_exists` : It will not recreate columns while only change the properties of existing columns , or create non-existent columns .
I have amended the commit already .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #11887: [Fix][dolphinscheduler-api] fix init&upgrade mysql-meta-schema bugs
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #11887:
URL: https://github.com/apache/dolphinscheduler/pull/11887#issuecomment-1245105150
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=11887)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=11887&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=11887&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=11887&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=11887&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org