You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/11/05 03:01:40 UTC
[GitHub] [dolphinscheduler] Bulls1986 opened a new pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Bulls1986 opened a new pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697
…royed, the channel cannot be cleaned up from the channel in time, resulting in the scheduling loss after the next scheduling task is triggered in a short time interval
<!--Thanks very much for contributing to Apache DolphinScheduler. Please review https://dolphinscheduler.apache.org/en-us/community/development/pull-request.html before opening a pull request.-->
## Purpose of the pull request
<!--(For example: This pull request adds checkstyle plugin).-->
## Brief change log
<!--*(for example:)*
- *Add maven-checkstyle-plugin to root pom.xml*
-->
## Verify this pull request
<!--*(Please pick either of the following options)*-->
This pull request is code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
<!--*(example:)*
- *Added dolphinscheduler-dao tests for end-to-end.*
- *Added CronUtilsTest to verify the change.*
- *Manually verified the change by testing locally.* -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Bulls1986 edited a comment on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
Bulls1986 edited a comment on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961592284
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961682101
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] lenboo commented on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
lenboo commented on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961711031
there are some code style errors.
@Bulls1986
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Bulls1986 edited a comment on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
Bulls1986 edited a comment on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961594671
This problem occurs when all nodes are deployed on k8s, and all registered addresses are svc hostname.This problem occurs when worker pods have some problem and frequently restart. At this time, the underlying ip of the same svc host has changed, but the channel in the cache is still connected to the previous pods.So here only need to clean up when the sending fails and exceed the limit, next time you will go to the channel creation logic again, at this time, the svc will be parsed into the pods ip of the latest worker
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961682101
Kudos, SonarCloud Quality Gate passed! 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Bulls1986 commented on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
Bulls1986 commented on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961592284
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Bulls1986 edited a comment on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
Bulls1986 edited a comment on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961594671
This problem occurs when all nodes are deployed on k8s, and all registered addresses are svc hostname.This problem occurs when worker pods have some problem and frequently restart. At this time, the underlying ip of the same svc host has changed, but the channel in the cache is still connected to the previous pods.So here only need to clean up when the sending fails and exceed the limit, next time you will go to the channel creation logic again, at this time, the svc will be parsed into the pods ip of the latest worker.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961682101
Kudos, SonarCloud Quality Gate passed! 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Bulls1986 commented on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
Bulls1986 commented on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961592284
At the same time, it seems that there is a legacy problem in TaskPriorityQueueConsumer. When the underlying dispatch throws an exception, it will only print the error log, but the task is not compensated and lost.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961866295
Kudos, SonarCloud Quality Gate passed! 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=CODE_SMELL) [1 Code Smell](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_coverage&view=list) [57.1% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter commented on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961679511
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Bulls1986 edited a comment on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
Bulls1986 edited a comment on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961592284
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961682101
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Bulls1986 commented on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
Bulls1986 commented on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961720632
Hi, I have modified the format and resubmitted.
@lenboo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Bulls1986 commented on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
Bulls1986 commented on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961592284
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] lenboo commented on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
lenboo commented on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961711031
there are some code style errors.
@Bulls1986
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter commented on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961679511
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6697?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#6697](https://codecov.io/gh/apache/dolphinscheduler/pull/6697?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (db93431) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/cbb623bfa0e44419e6f1e21ea64c86cd33df3560?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (cbb623b) will **decrease** coverage by `0.61%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/dolphinscheduler/pull/6697?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## dev #6697 +/- ##
============================================
- Coverage 38.42% 37.80% -0.62%
Complexity 3227 3227
============================================
Files 646 649 +3
Lines 25936 26356 +420
Branches 2817 2849 +32
============================================
- Hits 9966 9964 -2
- Misses 15055 15478 +423
+ Partials 915 914 -1
```
| [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/6697?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...master/dispatch/executor/NettyExecutorManager.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6697/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9kaXNwYXRjaC9leGVjdXRvci9OZXR0eUV4ZWN1dG9yTWFuYWdlci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...he/dolphinscheduler/common/enums/SqoopJobType.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6697/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL2VudW1zL1Nxb29wSm9iVHlwZS5qYXZh) | `0.00% <0.00%> (-88.89%)` | :arrow_down: |
| [...hinscheduler/dao/upgrade/ProcessDefinitionDao.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6697/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1kYW8vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvZGFvL3VwZ3JhZGUvUHJvY2Vzc0RlZmluaXRpb25EYW8uamF2YQ==) | `8.86% <0.00%> (-14.48%)` | :arrow_down: |
| [...pache/dolphinscheduler/dao/upgrade/UpgradeDao.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6697/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1kYW8vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvZGFvL3VwZ3JhZGUvVXBncmFkZURhby5qYXZh) | `2.77% <0.00%> (-1.45%)` | :arrow_down: |
| [...inscheduler/common/task/sqoop/SqoopParameters.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6697/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvY29tbW9uL3Rhc2svc3Fvb3AvU3Fvb3BQYXJhbWV0ZXJzLmphdmE=) | `74.00% <0.00%> (ø)` | |
| [...gin/registry/zookeeper/ZookeeperConfiguration.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6697/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1yZWdpc3RyeS1wbHVnaW4vZG9scGhpbnNjaGVkdWxlci1yZWdpc3RyeS16b29rZWVwZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcGx1Z2luL3JlZ2lzdHJ5L3pvb2tlZXBlci9ab29rZWVwZXJDb25maWd1cmF0aW9uLmphdmE=) | `100.00% <0.00%> (ø)` | |
| [...che/dolphinscheduler/dao/upgrade/JsonSplitDao.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6697/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1kYW8vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvZGFvL3VwZ3JhZGUvSnNvblNwbGl0RGFvLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...pache/dolphinscheduler/dao/upgrade/ProjectDao.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6697/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1kYW8vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvZGFvL3VwZ3JhZGUvUHJvamVjdERhby5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...ache/dolphinscheduler/dao/upgrade/ScheduleDao.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6697/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1kYW8vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvZGFvL3VwZ3JhZGUvU2NoZWR1bGVEYW8uamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...e/dolphinscheduler/remote/NettyRemotingClient.java](https://codecov.io/gh/apache/dolphinscheduler/pull/6697/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1yZW1vdGUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcmVtb3RlL05ldHR5UmVtb3RpbmdDbGllbnQuamF2YQ==) | `53.52% <0.00%> (+1.40%)` | :arrow_up: |
| ... and [3 more](https://codecov.io/gh/apache/dolphinscheduler/pull/6697/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6697?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/6697?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [cbb623b...db93431](https://codecov.io/gh/apache/dolphinscheduler/pull/6697?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-962417376
> This problem occurs when all nodes are deployed on k8s, and all registered addresses are svc hostname.This problem occurs when worker pods have some problem and frequently restart. At this time, the underlying ip of the same svc host has changed, but the channel in the cache is still connected to the previous pods.So here only need to clean up when the sending fails and exceed the limit, next time you will go to the channel creation logic again, at this time, the svc will be parsed into the pods ip of the latest worker.
Hi @Bulls1986, when the host change, it will get the new channel from channel cache `NettyRemotingClient.getChannel`. Why it still connected to the previous pod host?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Bulls1986 edited a comment on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
Bulls1986 edited a comment on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961592284
At the same time, it seems that there is a legacy problem in TaskPriorityQueueConsumer. When the task dispatch throws an exception, it will only print the error log, but the task is not compensated and lost.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] lenboo commented on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
lenboo commented on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961711031
there are some code style errors.
@Bulls1986
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Bulls1986 commented on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
Bulls1986 commented on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-962426641
On k8s, the worker always uses the same svc address as the host to register to zk. When the job restarts, although the zk listener is triggered, it will register the same address. At this time, the channel buffer of the communication layer has not been cleared. There are two places to actually clean up the corresponding host address. The first is that the getChannel method determines that it is not active. It will create a resolution ip based on the domain name, then establish a channel connection to this address, and finally overwrite the same key in the cache (the key here is the hostname); the second is to use the exceptionCaught method of the netty channel to clean up, because the channel is used here remoteAddress clears the cache, obtains the ip and port through remoteAddress, and then clears the above from the channel cache through the assembled address (note: the ip address may not be able to clear the channel with the hostname as the key), which causes the channel in the
channel cache to be invalid. Then it seems to be a more effective way to clean up invalid channels in the cache when the error exceeds the limit, so it can also prevent the next schedule from acquiring the problematic channel.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Bulls1986 closed pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
Bulls1986 closed pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Bulls1986 commented on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
Bulls1986 commented on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961594671
This problem occurs when all nodes are deployed on k8s, and all registered addresses are svc hostname。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961682101
Kudos, SonarCloud Quality Gate passed! 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] removed a comment on pull request #6697: [FIX-6692]Fix the problem that when the worker or the channel is dest…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #6697:
URL: https://github.com/apache/dolphinscheduler/pull/6697#issuecomment-961682101
Kudos, SonarCloud Quality Gate passed! 
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=6697&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=6697&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org