You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/03/01 05:26:28 UTC

[GitHub] [dolphinscheduler] ouyangyewei opened a new pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

ouyangyewei opened a new pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595


   fix #8594 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] caishunfeng commented on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
caishunfeng commented on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1068698179


   > @caishunfeng There are two ways I think of, one is to remove the initDatabase as you said, but it means that it needs to be executed **manually** in the future, and the other can consider abstracting the stored procedure, such as `CreateIndexIfNotExists`, `CreateUniqueKeyIfNotExists`, `DropIndexIfExists`
   
   What about split `initDatabase` as a alone step in docker sh? like
   
   > case "$1" in
       (all)
           ...
           waitDatabase
           initDatabase
           .....
       ;;
       (initDatabase)
           initDatabase
       ;;
       (api-server)
           ...
           waitDatabase
          ...
       ;;


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] ouyangyewei edited a comment on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
ouyangyewei edited a comment on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1056669201


   > Hi @ouyangyewei I have a question, this is called by the upgraded script, it should be called manually once, it doesn't matter how much the api replica is; BTW, You should also modify the ddl of pgsql.
   
   When deployed through K8S, if the dolphinscheduler-api is multi-replicas, the first Pod will successfully create an index, but other Pods will fail to start and raise an exception: java.sql.SQLSyntaxErrorException: Duplicate key name 'idx_code_version'


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] ouyangyewei commented on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
ouyangyewei commented on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1056668611


   > 
   
   When deployed through K8S, if the API is multi-replicas, the first Pod will successfully create an index, but other Pods will fail to start and raise an exception: java.sql.SQLSyntaxErrorException: Duplicate key name 'idx_code_version'
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] ouyangyewei edited a comment on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
ouyangyewei edited a comment on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1056669201


   > Hi @ouyangyewei I have a question, this is called by the upgraded script, it should be called manually once, it doesn't matter how much the api replica is; BTW, You should also modify the ddl of pgsql.
   
   When deployed through K8S, if the dolphinscheduler-api is multi-replicas, the first Pod will call and create indexs successfully, but other Pods will fail to start and raise an exception: java.sql.SQLSyntaxErrorException: Duplicate key name 'idx_code_version'


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] caishunfeng commented on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
caishunfeng commented on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1057648099


   > Hi @caishunfeng , when deployed through K8S, if the `dolphinscheduler-api` is multi-replicas, the first Pod will call and create indexs successfully, but other Pods will fail to start and raise an exception: `java.sql.SQLSyntaxErrorException: Duplicate key name 'idx_code_version'`
   
   Can you upgrade ddl manually? The upgrade script and api-server are not related. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] ouyangyewei commented on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
ouyangyewei commented on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1056064834


   These changes have been verified during the upgrade from 2.0.2 to 2.0.4


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] ouyangyewei commented on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
ouyangyewei commented on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1056669201


   > Hi @ouyangyewei I have a question, this is called by the upgraded script, it should be called manually once, it doesn't matter how much the api replica is; BTW, You should also modify the ddl of pgsql.
   
   When deployed through K8S, if the API is multi-replicas, the first Pod will successfully create an index, but other Pods will fail to start and raise an exception: java.sql.SQLSyntaxErrorException: Duplicate key name 'idx_code_version'


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] ouyangyewei commented on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
ouyangyewei commented on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1068662432


   @caishunfeng There are two ways I think of, one is to remove the initDatabase as you said, but it means that it needs to be executed **manually** in the future, and the other can consider abstracting the stored procedure, such as `CreateIndexIfNotExists`, `CreateUniqueKeyIfNotExists`, `DropIndexIfExists`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] caishunfeng commented on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
caishunfeng commented on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1068649690


   > @caishunfeng Related. The problem occurs when deploying through kubernetes clustering. Recently tested, it happens when upgrading from 2.0.2 to 2.0.5
   > 
   > After a simple analysis of the code, the tracking is as follows:
   
   Thanks for your feedback, and I think remove `initDatabase` in `api-server` is better, what do you think?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] caishunfeng commented on a change in pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
caishunfeng commented on a change in pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#discussion_r827554814



##########
File path: dolphinscheduler-dao/src/main/resources/sql/upgrade/2.1.0_schema/mysql/dolphinscheduler_ddl.sql
##########
@@ -15,15 +15,42 @@
  * limitations under the License.
 */
 
-ALTER TABLE `t_ds_task_instance` ADD INDEX `idx_code_version` (`task_code`, `task_definition_version`) USING BTREE;
+/************************************
+ * Procedure
+ ************************************/
+delimiter d//
+
+DROP PROCEDURE IF EXISTS create_index_if_not_exists d//
+CREATE PROCEDURE create_index_if_not_exists (
+    IN tableName varchar(128),
+    IN indexName varchar(128),
+    IN indexColumns varchar(128)
+)
+BEGIN
+    IF NOT EXISTS (SELECT * FROM INFORMATION_SCHEMA.STATISTICS WHERE TABLE_SCHEMA = DATABASE() AND table_name = tableName AND index_name = indexName)
+    THEN
+        SET @sqlstmt = CONCAT('ALTER TABLE `', tableName , '` ADD KEY `', indexName, '` (', indexColumns, ') USING BTREE');
+        PREPARE stmt FROM @sqlstmt;
+        EXECUTE stmt;
+        DEALLOCATE PREPARE stmt;
+    END IF;
+END;
+
+d//
+delimiter ;
+
+/************************************
+ * DDL
+ ************************************/
+call create_index_if_not_exists('t_ds_task_instance', 'idx_code_version', 'task_code, task_definition_version');
 ALTER TABLE `t_ds_task_instance` MODIFY COLUMN `task_params` longtext COMMENT 'job custom parameters' AFTER `app_link`;
 ALTER TABLE `t_ds_process_task_relation` ADD KEY `idx_code` (`project_code`, `process_definition_code`) USING BTREE;

Review comment:
       Will there be the same problem?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1055047600


   Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; ![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')
   
   [![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=BUG)  
   [![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=VULNERABILITY)  
   [![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=SECURITY_HOTSPOT)  
   [![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=CODE_SMELL)
   
   [![No Coverage information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png 'No Coverage information')](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=8595&metric=coverage&view=list) No Coverage information  
   [![No Duplication information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/NoDuplicationInfo-16px.png 'No Duplication information')](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=8595&metric=duplicated_lines_density&view=list) No Duplication information
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1055341753


   Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; ![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')
   
   [![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=BUG)  
   [![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=VULNERABILITY)  
   [![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=SECURITY_HOTSPOT)  
   [![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=CODE_SMELL)
   
   [![No Coverage information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png 'No Coverage information')](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=8595&metric=coverage&view=list) No Coverage information  
   [![No Duplication information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/NoDuplicationInfo-16px.png 'No Duplication information')](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=8595&metric=duplicated_lines_density&view=list) No Duplication information
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] ouyangyewei edited a comment on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
ouyangyewei edited a comment on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1056669201


   > Hi @ouyangyewei I have a question, this is called by the upgraded script, it should be called manually once, it doesn't matter how much the api replica is; BTW, You should also modify the ddl of pgsql.
   
   Hi @caishunfeng , When deployed through K8S, if the `dolphinscheduler-api` is multi-replicas, the first Pod will call and create indexs successfully, but other Pods will fail to start and raise an exception: `java.sql.SQLSyntaxErrorException: Duplicate key name 'idx_code_version'`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1055047600


   Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; ![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')
   
   [![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=BUG)  
   [![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=VULNERABILITY)  
   [![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=SECURITY_HOTSPOT)  
   [![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=8595&resolved=false&types=CODE_SMELL)
   
   [![No Coverage information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png 'No Coverage information')](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=8595&metric=coverage&view=list) No Coverage information  
   [![No Duplication information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/NoDuplicationInfo-16px.png 'No Duplication information')](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=8595&metric=duplicated_lines_density&view=list) No Duplication information
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] caishunfeng commented on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
caishunfeng commented on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1056431523


   Hi @ouyangyewei  I have a question, this is called by the upgraded script, it should be called manually once, it doesn't matter how much the api replica is;
   BTW, You should also modify the ddl of pgsql.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] ouyangyewei edited a comment on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
ouyangyewei edited a comment on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1056669201


   > Hi @ouyangyewei I have a question, this is called by the upgraded script, it should be called manually once, it doesn't matter how much the api replica is; BTW, You should also modify the ddl of pgsql.
   
   When deployed through K8S, if the `dolphinscheduler-api` is multi-replicas, the first Pod will call and create indexs successfully, but other Pods will fail to start and raise an exception: `java.sql.SQLSyntaxErrorException: Duplicate key name 'idx_code_version'`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] ouyangyewei commented on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
ouyangyewei commented on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1068107190


   @caishunfeng Related. The problem occurs when deploying through kubernetes clustering. Recently tested, it happens when upgrading from 2.0.2 to 2.0.4
   
   After a simple analysis of the code, the tracking is as follows:
   
   ### Code Trace
   
   1. docker/build/Dockerfile
   ![image](https://user-images.githubusercontent.com/3365997/158407007-69412f95-d7e4-45fe-9ae9-bbb248312cfe.png)
   
   2. Assign args 'api-server' to container
   ![image](https://user-images.githubusercontent.com/3365997/158407560-e6c519c2-b9cc-4483-91ad-06b6aa92be29.png)
   
   3. docker/build/startup.sh
   ![image](https://user-images.githubusercontent.com/3365997/158407174-1279d875-4601-484e-9af5-e3747bfb7f37.png)
   
   4. Execute CreateDolphinScheduler.main() once
   ![image](https://user-images.githubusercontent.com/3365997/158407911-ffb3d6d8-529a-4935-9904-21dd16a729e5.png)
   ![image](https://user-images.githubusercontent.com/3365997/158409381-089712e8-bbb3-4091-8e02-e35f046c1f62.png)
   ![image](https://user-images.githubusercontent.com/3365997/158409515-9255c632-893f-41fd-9b8d-5dc058e0c44d.png)
   
   ### Log Trace
   ```
   init env variables
   generate dolphinscheduler config
   try to connect zookeeper ...
   zookeeper connection is ok
   try to connect mysql ...
   mysql connection is ok
   import sql data
   
     .   ____          _            __ _ _
    /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
   ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
    \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
     '  |____| .__|_| |_|_| |_\__, | / / / /
    =========|_|==============|___/=/_/_/_/
    :: Spring Boot ::                (v2.5.6)
   
   log4j:WARN No appenders could be found for logger (org.jboss.logging).
   log4j:WARN Please initialize the log4j system properly.
   log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
   2022-03-15 21:20:46.418  INFO 34 --- [           main] o.a.d.d.u.shell.CreateDolphinScheduler   : Starting CreateDolphinScheduler v2.0.5 using Java 1.8.0_322 on pre-workflow-api-56c459578b-4jm49 with PID 34 (/opt/apache-dolphinscheduler-2.0.5-bin/lib/dolphinscheduler-dao-2.0.5.jar started by root in /opt/apache-dolphinscheduler-2.0.5-bin)
   2022-03-15 21:20:46.421  INFO 34 --- [           main] o.a.d.d.u.shell.CreateDolphinScheduler   : The following profiles are active: shell-create,shell-cli,default,mysql
   2022-03-15 21:20:47.801  INFO 34 --- [           main] ru.yandex.clickhouse.ClickHouseDriver    : Driver registered
   Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
   2022-03-15 21:20:47.932  INFO 34 --- [           main] com.zaxxer.hikari.HikariDataSource       : DolphinScheduler - Starting...
   2022-03-15 21:20:47.937  WARN 34 --- [           main] com.zaxxer.hikari.util.DriverDataSource  : Registered driver with driverClassName=com.mysql.jdbc.Driver was not found, trying direct instantiation.
   2022-03-15 21:20:48.168  INFO 34 --- [           main] com.zaxxer.hikari.HikariDataSource       : DolphinScheduler - Start completed.
    _ _   |_  _ _|_. ___ _ |    _
   | | |\/|_)(_| | |_\  |_)||_|_\
        /               |
                           3.2.0
   2022-03-15 21:20:49.977  INFO 34 --- [           main] o.a.d.d.u.shell.CreateDolphinScheduler   : Started CreateDolphinScheduler in 4.171 seconds (JVM running for 4.62)
   2022-03-15 21:20:50.005  INFO 34 --- [           main] o.a.d.d.upgrade.DolphinSchedulerManager  : The database has been initialized. Skip the initialization step
   2022-03-15 21:20:50.021  INFO 34 --- [           main] o.a.d.d.upgrade.DolphinSchedulerManager  : upgrade DolphinScheduler metadata version from 2.0.3 to 2.0.4
   2022-03-15 21:20:50.021  INFO 34 --- [           main] o.a.d.d.upgrade.DolphinSchedulerManager  : Begin upgrading DolphinScheduler's table structure
   2022-03-15 21:20:50.021  INFO 34 --- [           main] o.a.d.dao.upgrade.UpgradeDao             : pre_workflow
   2022-03-15 21:20:50.024  INFO 34 --- [           main] o.a.d.common.utils.ScriptRunner          : sql: /*  * Licensed to the Apache Software Foundation (ASF) under one or more  * contributor license agreements.  See the NOTICE file distributed with  * this work for additional information regarding copyright ownership.  * The ASF licenses this file to You under the Apache License, Version 2.0  * (the "License"); you may not use this file except in compliance with  * the License.  You may obtain a copy of the License at  *  *    http://www.apache.org/licenses/LICENSE-2.0  *  * Unless required by applicable law or agreed to in writing, software  * distributed under the License is distributed on an "AS IS" BASIS,  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  * See the License for the specific language governing permissions and  * limitations under the License. */ ALTER TABLE `t_ds_task_instance` ADD INDEX `idx_code_version` (`task_code`, `task_definition
 _version`) USING BTREE
   2022-03-15 21:20:50.046 ERROR 34 --- [           main] o.a.d.common.utils.ScriptRunner          : SQLException
   
   java.sql.SQLSyntaxErrorException: Duplicate key name 'idx_code_version'
   	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120) ~[mysql-connector-java-8.0.16.jar:8.0.16]
   	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97) ~[mysql-connector-java-8.0.16.jar:8.0.16]
   	at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) ~[mysql-connector-java-8.0.16.jar:8.0.16]
   	at com.mysql.cj.jdbc.StatementImpl.executeInternal(StatementImpl.java:782) ~[mysql-connector-java-8.0.16.jar:8.0.16]
   	at com.mysql.cj.jdbc.StatementImpl.execute(StatementImpl.java:666) ~[mysql-connector-java-8.0.16.jar:8.0.16]
   	at com.zaxxer.hikari.pool.ProxyStatement.execute(ProxyStatement.java:94) ~[HikariCP-4.0.3.jar:na]
   	at com.zaxxer.hikari.pool.HikariProxyStatement.execute(HikariProxyStatement.java) ~[HikariCP-4.0.3.jar:na]
   	at org.apache.dolphinscheduler.common.utils.ScriptRunner.runScript(ScriptRunner.java:117) [dolphinscheduler-common-2.0.5.jar:2.0.5]
   	at org.apache.dolphinscheduler.common.utils.ScriptRunner.runScript(ScriptRunner.java:72) [dolphinscheduler-common-2.0.5.jar:2.0.5]
   	at org.apache.dolphinscheduler.dao.upgrade.UpgradeDao.upgradeDolphinSchedulerDDL(UpgradeDao.java:333) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
   	at org.apache.dolphinscheduler.dao.upgrade.UpgradeDao.upgradeDolphinScheduler(UpgradeDao.java:125) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
   	at org.apache.dolphinscheduler.dao.upgrade.DolphinSchedulerManager.upgradeDolphinScheduler(DolphinSchedulerManager.java:105) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
   	at org.apache.dolphinscheduler.dao.upgrade.shell.CreateDolphinScheduler$CreateRunner.run(CreateDolphinScheduler.java:57) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
   	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:791) [spring-boot-2.5.6.jar:2.5.6]
   	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:775) [spring-boot-2.5.6.jar:2.5.6]
   	at org.springframework.boot.SpringApplication.run(SpringApplication.java:345) [spring-boot-2.5.6.jar:2.5.6]
   	at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:143) [spring-boot-2.5.6.jar:2.5.6]
   	at org.apache.dolphinscheduler.dao.upgrade.shell.CreateDolphinScheduler.main(CreateDolphinScheduler.java:40) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] ouyangyewei commented on a change in pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
ouyangyewei commented on a change in pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#discussion_r827566506



##########
File path: dolphinscheduler-dao/src/main/resources/sql/upgrade/2.1.0_schema/mysql/dolphinscheduler_ddl.sql
##########
@@ -15,15 +15,42 @@
  * limitations under the License.
 */
 
-ALTER TABLE `t_ds_task_instance` ADD INDEX `idx_code_version` (`task_code`, `task_definition_version`) USING BTREE;
+/************************************
+ * Procedure
+ ************************************/
+delimiter d//
+
+DROP PROCEDURE IF EXISTS create_index_if_not_exists d//
+CREATE PROCEDURE create_index_if_not_exists (
+    IN tableName varchar(128),
+    IN indexName varchar(128),
+    IN indexColumns varchar(128)
+)
+BEGIN
+    IF NOT EXISTS (SELECT * FROM INFORMATION_SCHEMA.STATISTICS WHERE TABLE_SCHEMA = DATABASE() AND table_name = tableName AND index_name = indexName)
+    THEN
+        SET @sqlstmt = CONCAT('ALTER TABLE `', tableName , '` ADD KEY `', indexName, '` (', indexColumns, ') USING BTREE');
+        PREPARE stmt FROM @sqlstmt;
+        EXECUTE stmt;
+        DEALLOCATE PREPARE stmt;
+    END IF;
+END;
+
+d//
+delimiter ;
+
+/************************************
+ * DDL
+ ************************************/
+call create_index_if_not_exists('t_ds_task_instance', 'idx_code_version', 'task_code, task_definition_version');
 ALTER TABLE `t_ds_task_instance` MODIFY COLUMN `task_params` longtext COMMENT 'job custom parameters' AFTER `app_link`;
 ALTER TABLE `t_ds_process_task_relation` ADD KEY `idx_code` (`project_code`, `process_definition_code`) USING BTREE;

Review comment:
       Yes, as long as the `create index if not exists` is not performed before the index is created in ddl, there will be this problem




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] ouyangyewei commented on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
ouyangyewei commented on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1055063281


   @JinyLeeChina @lenboo pls review, tks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] ouyangyewei edited a comment on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
ouyangyewei edited a comment on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1056669201


   > Hi @ouyangyewei I have a question, this is called by the upgraded script, it should be called manually once, it doesn't matter how much the api replica is; BTW, You should also modify the ddl of pgsql.
   
   Hi @caishunfeng , when deployed through K8S, if the `dolphinscheduler-api` is multi-replicas, the first Pod will call and create indexs successfully, but other Pods will fail to start and raise an exception: `java.sql.SQLSyntaxErrorException: Duplicate key name 'idx_code_version'`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] codecov-commenter commented on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1055042560


   # [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/8595?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8595](https://codecov.io/gh/apache/dolphinscheduler/pull/8595?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f1e89ef) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/a3e2c709fce14f6c58ab365458c185f25327b0d7?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a3e2c70) will **not change** coverage.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/dolphinscheduler/pull/8595/graphs/tree.svg?width=650&height=150&src=pr&token=bv9iXXRLi9&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/dolphinscheduler/pull/8595?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@            Coverage Diff            @@
   ##                dev    #8595   +/-   ##
   =========================================
     Coverage     45.31%   45.31%           
     Complexity     4059     4059           
   =========================================
     Files           696      696           
     Lines         26978    26978           
     Branches       2889     2889           
   =========================================
     Hits          12224    12224           
     Misses        13602    13602           
     Partials       1152     1152           
   ```
   
   
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/8595?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/8595?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [a3e2c70...f1e89ef](https://codecov.io/gh/apache/dolphinscheduler/pull/8595?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] ouyangyewei edited a comment on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
ouyangyewei edited a comment on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1056669201


   > Hi @ouyangyewei I have a question, this is called by the upgraded script, it should be called manually once, it doesn't matter how much the api replica is; BTW, You should also modify the ddl of pgsql.
   
   When deployed through K8S, if the dolphinscheduler-api is multi-replicas, the first Pod will call successfully and then create an index, but other Pods will fail to start and raise an exception: java.sql.SQLSyntaxErrorException: Duplicate key name 'idx_code_version'


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] ouyangyewei removed a comment on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
ouyangyewei removed a comment on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1056668611


   > 
   
   When deployed through K8S, if the API is multi-replicas, the first Pod will successfully create an index, but other Pods will fail to start and raise an exception: java.sql.SQLSyntaxErrorException: Duplicate key name 'idx_code_version'
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] ouyangyewei edited a comment on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
ouyangyewei edited a comment on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1068107190


   @caishunfeng Related. The problem occurs when deploying through kubernetes clustering. Recently tested, it happens when upgrading from 2.0.2 to 2.0.5
   
   After a simple analysis of the code, the tracking is as follows:
   
   ### Code Trace
   
   1. docker/build/Dockerfile
   ![image](https://user-images.githubusercontent.com/3365997/158407007-69412f95-d7e4-45fe-9ae9-bbb248312cfe.png)
   
   2. Assign args 'api-server' to container
   ![image](https://user-images.githubusercontent.com/3365997/158407560-e6c519c2-b9cc-4483-91ad-06b6aa92be29.png)
   
   3. docker/build/startup.sh
   ![image](https://user-images.githubusercontent.com/3365997/158407174-1279d875-4601-484e-9af5-e3747bfb7f37.png)
   
   4. Execute CreateDolphinScheduler.main() once
   ![image](https://user-images.githubusercontent.com/3365997/158407911-ffb3d6d8-529a-4935-9904-21dd16a729e5.png)
   ![image](https://user-images.githubusercontent.com/3365997/158409381-089712e8-bbb3-4091-8e02-e35f046c1f62.png)
   ![image](https://user-images.githubusercontent.com/3365997/158409515-9255c632-893f-41fd-9b8d-5dc058e0c44d.png)
   
   ### Log Trace
   ```
   init env variables
   generate dolphinscheduler config
   try to connect zookeeper ...
   zookeeper connection is ok
   try to connect mysql ...
   mysql connection is ok
   import sql data
   
     .   ____          _            __ _ _
    /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
   ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
    \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
     '  |____| .__|_| |_|_| |_\__, | / / / /
    =========|_|==============|___/=/_/_/_/
    :: Spring Boot ::                (v2.5.6)
   
   log4j:WARN No appenders could be found for logger (org.jboss.logging).
   log4j:WARN Please initialize the log4j system properly.
   log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
   2022-03-15 21:20:46.418  INFO 34 --- [           main] o.a.d.d.u.shell.CreateDolphinScheduler   : Starting CreateDolphinScheduler v2.0.5 using Java 1.8.0_322 on pre-workflow-api-56c459578b-4jm49 with PID 34 (/opt/apache-dolphinscheduler-2.0.5-bin/lib/dolphinscheduler-dao-2.0.5.jar started by root in /opt/apache-dolphinscheduler-2.0.5-bin)
   2022-03-15 21:20:46.421  INFO 34 --- [           main] o.a.d.d.u.shell.CreateDolphinScheduler   : The following profiles are active: shell-create,shell-cli,default,mysql
   2022-03-15 21:20:47.801  INFO 34 --- [           main] ru.yandex.clickhouse.ClickHouseDriver    : Driver registered
   Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
   2022-03-15 21:20:47.932  INFO 34 --- [           main] com.zaxxer.hikari.HikariDataSource       : DolphinScheduler - Starting...
   2022-03-15 21:20:47.937  WARN 34 --- [           main] com.zaxxer.hikari.util.DriverDataSource  : Registered driver with driverClassName=com.mysql.jdbc.Driver was not found, trying direct instantiation.
   2022-03-15 21:20:48.168  INFO 34 --- [           main] com.zaxxer.hikari.HikariDataSource       : DolphinScheduler - Start completed.
    _ _   |_  _ _|_. ___ _ |    _
   | | |\/|_)(_| | |_\  |_)||_|_\
        /               |
                           3.2.0
   2022-03-15 21:20:49.977  INFO 34 --- [           main] o.a.d.d.u.shell.CreateDolphinScheduler   : Started CreateDolphinScheduler in 4.171 seconds (JVM running for 4.62)
   2022-03-15 21:20:50.005  INFO 34 --- [           main] o.a.d.d.upgrade.DolphinSchedulerManager  : The database has been initialized. Skip the initialization step
   2022-03-15 21:20:50.021  INFO 34 --- [           main] o.a.d.d.upgrade.DolphinSchedulerManager  : upgrade DolphinScheduler metadata version from 2.0.3 to 2.0.4
   2022-03-15 21:20:50.021  INFO 34 --- [           main] o.a.d.d.upgrade.DolphinSchedulerManager  : Begin upgrading DolphinScheduler's table structure
   2022-03-15 21:20:50.021  INFO 34 --- [           main] o.a.d.dao.upgrade.UpgradeDao             : pre_workflow
   2022-03-15 21:20:50.024  INFO 34 --- [           main] o.a.d.common.utils.ScriptRunner          : sql: /*  * Licensed to the Apache Software Foundation (ASF) under one or more  * contributor license agreements.  See the NOTICE file distributed with  * this work for additional information regarding copyright ownership.  * The ASF licenses this file to You under the Apache License, Version 2.0  * (the "License"); you may not use this file except in compliance with  * the License.  You may obtain a copy of the License at  *  *    http://www.apache.org/licenses/LICENSE-2.0  *  * Unless required by applicable law or agreed to in writing, software  * distributed under the License is distributed on an "AS IS" BASIS,  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  * See the License for the specific language governing permissions and  * limitations under the License. */ ALTER TABLE `t_ds_task_instance` ADD INDEX `idx_code_version` (`task_code`, `task_definition
 _version`) USING BTREE
   2022-03-15 21:20:50.046 ERROR 34 --- [           main] o.a.d.common.utils.ScriptRunner          : SQLException
   
   java.sql.SQLSyntaxErrorException: Duplicate key name 'idx_code_version'
   	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120) ~[mysql-connector-java-8.0.16.jar:8.0.16]
   	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97) ~[mysql-connector-java-8.0.16.jar:8.0.16]
   	at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) ~[mysql-connector-java-8.0.16.jar:8.0.16]
   	at com.mysql.cj.jdbc.StatementImpl.executeInternal(StatementImpl.java:782) ~[mysql-connector-java-8.0.16.jar:8.0.16]
   	at com.mysql.cj.jdbc.StatementImpl.execute(StatementImpl.java:666) ~[mysql-connector-java-8.0.16.jar:8.0.16]
   	at com.zaxxer.hikari.pool.ProxyStatement.execute(ProxyStatement.java:94) ~[HikariCP-4.0.3.jar:na]
   	at com.zaxxer.hikari.pool.HikariProxyStatement.execute(HikariProxyStatement.java) ~[HikariCP-4.0.3.jar:na]
   	at org.apache.dolphinscheduler.common.utils.ScriptRunner.runScript(ScriptRunner.java:117) [dolphinscheduler-common-2.0.5.jar:2.0.5]
   	at org.apache.dolphinscheduler.common.utils.ScriptRunner.runScript(ScriptRunner.java:72) [dolphinscheduler-common-2.0.5.jar:2.0.5]
   	at org.apache.dolphinscheduler.dao.upgrade.UpgradeDao.upgradeDolphinSchedulerDDL(UpgradeDao.java:333) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
   	at org.apache.dolphinscheduler.dao.upgrade.UpgradeDao.upgradeDolphinScheduler(UpgradeDao.java:125) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
   	at org.apache.dolphinscheduler.dao.upgrade.DolphinSchedulerManager.upgradeDolphinScheduler(DolphinSchedulerManager.java:105) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
   	at org.apache.dolphinscheduler.dao.upgrade.shell.CreateDolphinScheduler$CreateRunner.run(CreateDolphinScheduler.java:57) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
   	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:791) [spring-boot-2.5.6.jar:2.5.6]
   	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:775) [spring-boot-2.5.6.jar:2.5.6]
   	at org.springframework.boot.SpringApplication.run(SpringApplication.java:345) [spring-boot-2.5.6.jar:2.5.6]
   	at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:143) [spring-boot-2.5.6.jar:2.5.6]
   	at org.apache.dolphinscheduler.dao.upgrade.shell.CreateDolphinScheduler.main(CreateDolphinScheduler.java:40) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [dolphinscheduler] ouyangyewei commented on pull request #8595: [Bug] [dolphinscheduler-sql] create index if not exists #8594

Posted by GitBox <gi...@apache.org>.
ouyangyewei commented on pull request #8595:
URL: https://github.com/apache/dolphinscheduler/pull/8595#issuecomment-1068669029


   @caishunfeng I think initDatabase() is a convenience to help users automatically step up from a very low version to a higher version. Otherwise, the user needs to check the current version in `t_ds_version`, and then manually execute DDL for each subsequent version


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org