You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/06/10 01:36:04 UTC

[GitHub] [incubator-dolphinscheduler] Eights-Li opened a new pull request #2943: Sqoop task optimization

Eights-Li opened a new pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943


   ## What is the purpose of the pull request
   #2917 #2763 
   *sqoop task optimization*
   
   ## Brief change log
     - *Add CUSTOM and TAMELATE job for sqoop task*
     - *support hadoop params like mr job name, etc*
     - *suooprt sqoop advanced params like --direct & --fetch-size, etc*
     - *Add sqoop task front-end selection box cascade display*
     - *fix bug : java map fields and hive map fields generate error*
   
   ## Verify this pull request
     - */server/worker/task/sqoop/SqoopTaskTest.java*
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] break60 commented on pull request #2943: Sqoop task optimization

Posted by GitBox <gi...@apache.org>.
break60 commented on pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943#issuecomment-641694633


   front-end +1


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] sonarcloud[bot] removed a comment on pull request #2943: Sqoop task optimization

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943#issuecomment-641680630


   SonarCloud Quality Gate failed.
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug.png' alt='Bug' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability.png' alt='Vulnerability' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) (and [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot.png' alt='Security Hotspot' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=SECURITY_HOTSPOT) to review)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell.png' alt='Code Smell' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL)
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/25.png' alt='28.3%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_coverage&view=list) [28.3% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_coverage&view=list)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3.png' alt='0.0%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_duplicated_lines_density&view=list)
   
   <img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/message_warning.png' alt='warning' width='16' height='16' /> The version of Java (1.8.0_252) you have used to run this analysis is deprecated and we will stop accepting it from October 2020. Please update to at least Java 11.
   Read more [here](https://sonarcloud.io/documentation/upcoming/)
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] sonarcloud[bot] commented on pull request #2943: Sqoop task optimization

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943#issuecomment-641689116


   Kudos, SonarCloud Quality Gate passed!
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug.png' alt='Bug' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability.png' alt='Vulnerability' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) (and [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot.png' alt='Security Hotspot' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=SECURITY_HOTSPOT) to review)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell.png' alt='Code Smell' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL)
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/60.png' alt='69.0%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_coverage&view=list) [69.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_coverage&view=list)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3.png' alt='0.0%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_duplicated_lines_density&view=list)
   
   <img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/message_warning.png' alt='warning' width='16' height='16' /> The version of Java (1.8.0_252) you have used to run this analysis is deprecated and we will stop accepting it from October 2020. Please update to at least Java 11.
   Read more [here](https://sonarcloud.io/documentation/upcoming/)
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] sonarcloud[bot] commented on pull request #2943: Sqoop task optimization

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943#issuecomment-642050499


   Kudos, SonarCloud Quality Gate passed!
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug.png' alt='Bug' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability.png' alt='Vulnerability' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) (and [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot.png' alt='Security Hotspot' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=SECURITY_HOTSPOT) to review)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell.png' alt='Code Smell' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL) [1 Code Smell](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL)
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/60.png' alt='68.4%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_coverage&view=list) [68.4% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_coverage&view=list)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3.png' alt='0.0%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_duplicated_lines_density&view=list)
   
   <img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/message_warning.png' alt='warning' width='16' height='16' /> The version of Java (1.8.0_252) you have used to run this analysis is deprecated and we will stop accepting it from October 2020. Please update to at least Java 11.
   Read more [here](https://sonarcloud.io/documentation/upcoming/)
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] sonarcloud[bot] removed a comment on pull request #2943: Sqoop task optimization

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943#issuecomment-641689116


   Kudos, SonarCloud Quality Gate passed!
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug.png' alt='Bug' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability.png' alt='Vulnerability' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) (and [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot.png' alt='Security Hotspot' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=SECURITY_HOTSPOT) to review)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell.png' alt='Code Smell' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL)
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/60.png' alt='69.0%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_coverage&view=list) [69.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_coverage&view=list)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3.png' alt='0.0%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_duplicated_lines_density&view=list)
   
   <img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/message_warning.png' alt='warning' width='16' height='16' /> The version of Java (1.8.0_252) you have used to run this analysis is deprecated and we will stop accepting it from October 2020. Please update to at least Java 11.
   Read more [here](https://sonarcloud.io/documentation/upcoming/)
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] codecov-commenter commented on pull request #2943: Sqoop task optimization

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943#issuecomment-641679748


   # [Codecov](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=h1) Report
   > Merging [#2943](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=desc) into [dev](https://codecov.io/gh/apache/incubator-dolphinscheduler/commit/b21259d6005ee7e8d760ea32302d668c552463f7&el=desc) will **decrease** coverage by `0.07%`.
   > The diff coverage is `29.34%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/graphs/tree.svg?width=650&height=150&src=pr&token=bv9iXXRLi9)](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff              @@
   ##                dev    #2943      +/-   ##
   ============================================
   - Coverage     37.21%   37.13%   -0.08%     
   - Complexity     2560     2569       +9     
   ============================================
     Files           435      436       +1     
     Lines         20020    20079      +59     
     Branches       2426     2443      +17     
   ============================================
   + Hits           7451     7457       +6     
   - Misses        11904    11960      +56     
   + Partials        665      662       -3     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | [...ommon/task/sqoop/targets/TargetMysqlParameter.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL3Rhc2svc3Fvb3AvdGFyZ2V0cy9UYXJnZXRNeXNxbFBhcmFtZXRlci5qYXZh) | `100.00% <ø> (ø)` | `19.00 <0.00> (ø)` | |
   | [...ver/master/consumer/TaskPriorityQueueConsumer.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9jb25zdW1lci9UYXNrUHJpb3JpdHlRdWV1ZUNvbnN1bWVyLmphdmE=) | `51.82% <0.00%> (-7.00%)` | `16.00 <0.00> (ø)` | |
   | [...r/worker/task/sqoop/generator/CommonGenerator.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL3Nxb29wL2dlbmVyYXRvci9Db21tb25HZW5lcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [...worker/task/sqoop/generator/SqoopJobGenerator.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL3Nxb29wL2dlbmVyYXRvci9TcW9vcEpvYkdlbmVyYXRvci5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [.../sqoop/generator/sources/MysqlSourceGenerator.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL3Nxb29wL2dlbmVyYXRvci9zb3VyY2VzL015c3FsU291cmNlR2VuZXJhdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [.../sqoop/generator/targets/MysqlTargetGenerator.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL3Nxb29wL2dlbmVyYXRvci90YXJnZXRzL015c3FsVGFyZ2V0R2VuZXJhdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
   | [...inscheduler/common/task/sqoop/SqoopParameters.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL3Rhc2svc3Fvb3AvU3Fvb3BQYXJhbWV0ZXJzLmphdmE=) | `74.00% <58.06%> (-6.00%)` | `25.00 <12.00> (+10.00)` | :arrow_down: |
   | [...he/dolphinscheduler/common/enums/SqoopJobType.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1Nxb29wSm9iVHlwZS5qYXZh) | `88.88% <88.88%> (ø)` | `3.00 <3.00> (?)` | |
   | [.../org/apache/dolphinscheduler/api/enums/Status.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvYXBpL2VudW1zL1N0YXR1cy5qYXZh) | `100.00% <100.00%> (ø)` | `5.00 <0.00> (ø)` | |
   | [...duler/server/entity/SqoopTaskExecutionContext.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL2VudGl0eS9TcW9vcFRhc2tFeGVjdXRpb25Db250ZXh0LmphdmE=) | `35.00% <0.00%> (-30.00%)` | `7.00% <0.00%> (-3.00%)` | |
   | ... and [4 more](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=footer). Last update [b21259d...5d83d79](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] codecov-commenter edited a comment on pull request #2943: Sqoop task optimization

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943#issuecomment-641679748


   # [Codecov](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=h1) Report
   > Merging [#2943](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=desc) into [dev](https://codecov.io/gh/apache/incubator-dolphinscheduler/commit/2749c7e5369459c1a410508cc4769db37f4a0043&el=desc) will **increase** coverage by `0.94%`.
   > The diff coverage is `60.86%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/graphs/tree.svg?width=650&height=150&src=pr&token=bv9iXXRLi9)](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff              @@
   ##                dev    #2943      +/-   ##
   ============================================
   + Coverage     37.20%   38.14%   +0.94%     
   - Complexity     2559     2621      +62     
   ============================================
     Files           435      436       +1     
     Lines         20020    20079      +59     
     Branches       2426     2443      +17     
   ============================================
   + Hits           7449     7660     +211     
   + Misses        11904    11711     -193     
   - Partials        667      708      +41     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | [...ommon/task/sqoop/targets/TargetMysqlParameter.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL3Rhc2svc3Fvb3AvdGFyZ2V0cy9UYXJnZXRNeXNxbFBhcmFtZXRlci5qYXZh) | `100.00% <ø> (ø)` | `19.00 <0.00> (ø)` | |
   | [...ver/master/consumer/TaskPriorityQueueConsumer.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9jb25zdW1lci9UYXNrUHJpb3JpdHlRdWV1ZUNvbnN1bWVyLmphdmE=) | `51.82% <0.00%> (-7.00%)` | `16.00 <0.00> (ø)` | |
   | [.../sqoop/generator/sources/MysqlSourceGenerator.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL3Nxb29wL2dlbmVyYXRvci9zb3VyY2VzL015c3FsU291cmNlR2VuZXJhdG9yLmphdmE=) | `58.00% <42.85%> (+58.00%)` | `5.00 <0.00> (+5.00)` | |
   | [...inscheduler/common/task/sqoop/SqoopParameters.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL3Rhc2svc3Fvb3AvU3Fvb3BQYXJhbWV0ZXJzLmphdmE=) | `74.00% <58.06%> (-6.00%)` | `25.00 <12.00> (+10.00)` | :arrow_down: |
   | [...worker/task/sqoop/generator/SqoopJobGenerator.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL3Nxb29wL2dlbmVyYXRvci9TcW9vcEpvYkdlbmVyYXRvci5qYXZh) | `74.07% <72.72%> (+74.07%)` | `10.00 <2.00> (+10.00)` | |
   | [...he/dolphinscheduler/common/enums/SqoopJobType.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1Nxb29wSm9iVHlwZS5qYXZh) | `88.88% <88.88%> (ø)` | `3.00 <3.00> (?)` | |
   | [...r/worker/task/sqoop/generator/CommonGenerator.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL3Nxb29wL2dlbmVyYXRvci9Db21tb25HZW5lcmF0b3IuamF2YQ==) | `83.87% <89.47%> (+83.87%)` | `6.00 <0.00> (+6.00)` | |
   | [.../org/apache/dolphinscheduler/api/enums/Status.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvYXBpL2VudW1zL1N0YXR1cy5qYXZh) | `100.00% <100.00%> (ø)` | `5.00 <0.00> (ø)` | |
   | [.../sqoop/generator/targets/MysqlTargetGenerator.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL3Nxb29wL2dlbmVyYXRvci90YXJnZXRzL015c3FsVGFyZ2V0R2VuZXJhdG9yLmphdmE=) | `75.75% <100.00%> (+75.75%)` | `4.00 <0.00> (+4.00)` | |
   | ... and [17 more](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=footer). Last update [2749c7e...a420644](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] sonarcloud[bot] commented on pull request #2943: Sqoop task optimization

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943#issuecomment-641680630


   SonarCloud Quality Gate failed.
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug.png' alt='Bug' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability.png' alt='Vulnerability' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) (and [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot.png' alt='Security Hotspot' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=SECURITY_HOTSPOT) to review)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell.png' alt='Code Smell' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL)
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/25.png' alt='28.3%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_coverage&view=list) [28.3% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_coverage&view=list)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3.png' alt='0.0%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_duplicated_lines_density&view=list)
   
   <img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/message_warning.png' alt='warning' width='16' height='16' /> The version of Java (1.8.0_252) you have used to run this analysis is deprecated and we will stop accepting it from October 2020. Please update to at least Java 11.
   Read more [here](https://sonarcloud.io/documentation/upcoming/)
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] Eights-Li commented on a change in pull request #2943: Sqoop task optimization

Posted by GitBox <gi...@apache.org>.
Eights-Li commented on a change in pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943#discussion_r438141605



##########
File path: dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/sqoop/generator/SqoopJobGenerator.java
##########
@@ -62,14 +63,23 @@ private void createSqoopJobGenerator(String sourceType,String targetType){
      * @return
      */
     public String generateSqoopJob(SqoopParameters sqoopParameters,TaskExecutionContext taskExecutionContext){
-        createSqoopJobGenerator(sqoopParameters.getSourceType(),sqoopParameters.getTargetType());
-        if(sourceGenerator == null || targetGenerator == null){
-            return null;
+
+        String sqoopScripts = "";
+
+        if (SqoopJobType.TEMPLATE.getDescp().equals(sqoopParameters.getJobType())) {
+            createSqoopJobGenerator(sqoopParameters.getSourceType(),sqoopParameters.getTargetType());
+            if(sourceGenerator == null || targetGenerator == null){
+                return null;
+            }

Review comment:
       yes, if null should throw a runtimeException




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] sonarcloud[bot] removed a comment on pull request #2943: Sqoop task optimization

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943#issuecomment-641689868


   Kudos, SonarCloud Quality Gate passed!
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug.png' alt='Bug' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability.png' alt='Vulnerability' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) (and [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot.png' alt='Security Hotspot' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=SECURITY_HOTSPOT) to review)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell.png' alt='Code Smell' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL)
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/60.png' alt='69.0%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_coverage&view=list) [69.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_coverage&view=list)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3.png' alt='0.0%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_duplicated_lines_density&view=list)
   
   <img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/message_warning.png' alt='warning' width='16' height='16' /> The version of Java (1.8.0_252) you have used to run this analysis is deprecated and we will stop accepting it from October 2020. Please update to at least Java 11.
   Read more [here](https://sonarcloud.io/documentation/upcoming/)
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] Eights-Li commented on pull request #2943: Sqoop task optimization

Posted by GitBox <gi...@apache.org>.
Eights-Li commented on pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943#issuecomment-641691019


   @break60 please front-end. thx
   @zixi0825 hi, this pr for sqoop task
   @743294668 bro, Can this pr meet your requirements?
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] sonarcloud[bot] commented on pull request #2943: Sqoop task optimization

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943#issuecomment-641689868


   Kudos, SonarCloud Quality Gate passed!
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug.png' alt='Bug' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=BUG)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability.png' alt='Vulnerability' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=VULNERABILITY) (and [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot.png' alt='Security Hotspot' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=SECURITY_HOTSPOT) to review)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell.png' alt='Code Smell' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=2943&resolved=false&types=CODE_SMELL)
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/60.png' alt='69.0%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_coverage&view=list) [69.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_coverage&view=list)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3.png' alt='0.0%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=2943&metric=new_duplicated_lines_density&view=list)
   
   <img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/message_warning.png' alt='warning' width='16' height='16' /> The version of Java (1.8.0_252) you have used to run this analysis is deprecated and we will stop accepting it from October 2020. Please update to at least Java 11.
   Read more [here](https://sonarcloud.io/documentation/upcoming/)
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] codecov-commenter edited a comment on pull request #2943: Sqoop task optimization

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943#issuecomment-641679748


   # [Codecov](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=h1) Report
   > Merging [#2943](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=desc) into [dev](https://codecov.io/gh/apache/incubator-dolphinscheduler/commit/2749c7e5369459c1a410508cc4769db37f4a0043&el=desc) will **increase** coverage by `0.78%`.
   > The diff coverage is `60.86%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/graphs/tree.svg?width=650&height=150&src=pr&token=bv9iXXRLi9)](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff              @@
   ##                dev    #2943      +/-   ##
   ============================================
   + Coverage     37.20%   37.99%   +0.78%     
   - Complexity     2559     2613      +54     
   ============================================
     Files           435      436       +1     
     Lines         20020    20079      +59     
     Branches       2426     2443      +17     
   ============================================
   + Hits           7449     7629     +180     
   + Misses        11904    11740     -164     
   - Partials        667      710      +43     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | [...ommon/task/sqoop/targets/TargetMysqlParameter.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL3Rhc2svc3Fvb3AvdGFyZ2V0cy9UYXJnZXRNeXNxbFBhcmFtZXRlci5qYXZh) | `100.00% <ø> (ø)` | `19.00 <0.00> (ø)` | |
   | [...ver/master/consumer/TaskPriorityQueueConsumer.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9jb25zdW1lci9UYXNrUHJpb3JpdHlRdWV1ZUNvbnN1bWVyLmphdmE=) | `51.82% <0.00%> (-7.00%)` | `16.00 <0.00> (ø)` | |
   | [.../sqoop/generator/sources/MysqlSourceGenerator.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL3Nxb29wL2dlbmVyYXRvci9zb3VyY2VzL015c3FsU291cmNlR2VuZXJhdG9yLmphdmE=) | `58.00% <42.85%> (+58.00%)` | `5.00 <0.00> (+5.00)` | |
   | [...inscheduler/common/task/sqoop/SqoopParameters.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL3Rhc2svc3Fvb3AvU3Fvb3BQYXJhbWV0ZXJzLmphdmE=) | `74.00% <58.06%> (-6.00%)` | `25.00 <12.00> (+10.00)` | :arrow_down: |
   | [...worker/task/sqoop/generator/SqoopJobGenerator.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL3Nxb29wL2dlbmVyYXRvci9TcW9vcEpvYkdlbmVyYXRvci5qYXZh) | `74.07% <72.72%> (+74.07%)` | `10.00 <2.00> (+10.00)` | |
   | [...he/dolphinscheduler/common/enums/SqoopJobType.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1Nxb29wSm9iVHlwZS5qYXZh) | `88.88% <88.88%> (ø)` | `3.00 <3.00> (?)` | |
   | [...r/worker/task/sqoop/generator/CommonGenerator.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL3Nxb29wL2dlbmVyYXRvci9Db21tb25HZW5lcmF0b3IuamF2YQ==) | `83.87% <89.47%> (+83.87%)` | `6.00 <0.00> (+6.00)` | |
   | [.../org/apache/dolphinscheduler/api/enums/Status.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvYXBpL2VudW1zL1N0YXR1cy5qYXZh) | `100.00% <100.00%> (ø)` | `5.00 <0.00> (ø)` | |
   | [.../sqoop/generator/targets/MysqlTargetGenerator.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL3Nxb29wL2dlbmVyYXRvci90YXJnZXRzL015c3FsVGFyZ2V0R2VuZXJhdG9yLmphdmE=) | `75.75% <100.00%> (+75.75%)` | `4.00 <0.00> (+4.00)` | |
   | [.../server/worker/processor/TaskExecuteProcessor.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci9wcm9jZXNzb3IvVGFza0V4ZWN1dGVQcm9jZXNzb3IuamF2YQ==) | `10.16% <0.00%> (-27.12%)` | `1.00% <0.00%> (-2.00%)` | |
   | ... and [22 more](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=footer). Last update [2749c7e...a64c2b3](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] codecov-commenter edited a comment on pull request #2943: Sqoop task optimization

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943#issuecomment-641679748


   # [Codecov](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=h1) Report
   > Merging [#2943](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=desc) into [dev](https://codecov.io/gh/apache/incubator-dolphinscheduler/commit/2749c7e5369459c1a410508cc4769db37f4a0043&el=desc) will **increase** coverage by `0.87%`.
   > The diff coverage is `60.86%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/graphs/tree.svg?width=650&height=150&src=pr&token=bv9iXXRLi9)](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff              @@
   ##                dev    #2943      +/-   ##
   ============================================
   + Coverage     37.20%   38.07%   +0.87%     
   - Complexity     2559     2616      +57     
   ============================================
     Files           435      436       +1     
     Lines         20020    20079      +59     
     Branches       2426     2443      +17     
   ============================================
   + Hits           7449     7646     +197     
   + Misses        11904    11721     -183     
   - Partials        667      712      +45     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | [...ommon/task/sqoop/targets/TargetMysqlParameter.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL3Rhc2svc3Fvb3AvdGFyZ2V0cy9UYXJnZXRNeXNxbFBhcmFtZXRlci5qYXZh) | `100.00% <ø> (ø)` | `19.00 <0.00> (ø)` | |
   | [...ver/master/consumer/TaskPriorityQueueConsumer.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9jb25zdW1lci9UYXNrUHJpb3JpdHlRdWV1ZUNvbnN1bWVyLmphdmE=) | `51.82% <0.00%> (-7.00%)` | `16.00 <0.00> (ø)` | |
   | [.../sqoop/generator/sources/MysqlSourceGenerator.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL3Nxb29wL2dlbmVyYXRvci9zb3VyY2VzL015c3FsU291cmNlR2VuZXJhdG9yLmphdmE=) | `58.00% <42.85%> (+58.00%)` | `5.00 <0.00> (+5.00)` | |
   | [...inscheduler/common/task/sqoop/SqoopParameters.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL3Rhc2svc3Fvb3AvU3Fvb3BQYXJhbWV0ZXJzLmphdmE=) | `74.00% <58.06%> (-6.00%)` | `25.00 <12.00> (+10.00)` | :arrow_down: |
   | [...worker/task/sqoop/generator/SqoopJobGenerator.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL3Nxb29wL2dlbmVyYXRvci9TcW9vcEpvYkdlbmVyYXRvci5qYXZh) | `74.07% <72.72%> (+74.07%)` | `10.00 <2.00> (+10.00)` | |
   | [...he/dolphinscheduler/common/enums/SqoopJobType.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1Nxb29wSm9iVHlwZS5qYXZh) | `88.88% <88.88%> (ø)` | `3.00 <3.00> (?)` | |
   | [...r/worker/task/sqoop/generator/CommonGenerator.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL3Nxb29wL2dlbmVyYXRvci9Db21tb25HZW5lcmF0b3IuamF2YQ==) | `83.87% <89.47%> (+83.87%)` | `6.00 <0.00> (+6.00)` | |
   | [.../org/apache/dolphinscheduler/api/enums/Status.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvYXBpL2VudW1zL1N0YXR1cy5qYXZh) | `100.00% <100.00%> (ø)` | `5.00 <0.00> (ø)` | |
   | [.../sqoop/generator/targets/MysqlTargetGenerator.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3dvcmtlci90YXNrL3Nxb29wL2dlbmVyYXRvci90YXJnZXRzL015c3FsVGFyZ2V0R2VuZXJhdG9yLmphdmE=) | `75.75% <100.00%> (+75.75%)` | `4.00 <0.00> (+4.00)` | |
   | [...he/dolphinscheduler/api/service/LoggerService.java](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree#diff-ZG9scGhpbnNjaGVkdWxlci1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvYXBpL3NlcnZpY2UvTG9nZ2VyU2VydmljZS5qYXZh) | `82.60% <0.00%> (-8.70%)` | `9.00% <0.00%> (-1.00%)` | |
   | ... and [20 more](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=footer). Last update [2749c7e...a64c2b3](https://codecov.io/gh/apache/incubator-dolphinscheduler/pull/2943?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] Rubik-W commented on a change in pull request #2943: Sqoop task optimization

Posted by GitBox <gi...@apache.org>.
Rubik-W commented on a change in pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943#discussion_r438067714



##########
File path: dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/sqoop/generator/SqoopJobGenerator.java
##########
@@ -62,14 +63,23 @@ private void createSqoopJobGenerator(String sourceType,String targetType){
      * @return
      */
     public String generateSqoopJob(SqoopParameters sqoopParameters,TaskExecutionContext taskExecutionContext){
-        createSqoopJobGenerator(sqoopParameters.getSourceType(),sqoopParameters.getTargetType());
-        if(sourceGenerator == null || targetGenerator == null){
-            return null;
+
+        String sqoopScripts = "";
+
+        if (SqoopJobType.TEMPLATE.getDescp().equals(sqoopParameters.getJobType())) {
+            createSqoopJobGenerator(sqoopParameters.getSourceType(),sqoopParameters.getTargetType());
+            if(sourceGenerator == null || targetGenerator == null){
+                return null;
+            }

Review comment:
       If return null, I see the follow-up code, the task will eventually be judged as successful.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-dolphinscheduler] Rubik-W merged pull request #2943: Sqoop task optimization

Posted by GitBox <gi...@apache.org>.
Rubik-W merged pull request #2943:
URL: https://github.com/apache/incubator-dolphinscheduler/pull/2943


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org