You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/05/11 03:35:18 UTC
[GitHub] [dolphinscheduler] netemadi opened a new pull request, #9980: Add the mechanism to increment the workerWaitingTaskCount in HeartBea…
netemadi opened a new pull request, #9980:
URL: https://github.com/apache/dolphinscheduler/pull/9980
<!--Thanks very much for contributing to Apache DolphinScheduler. Please review https://dolphinscheduler.apache.org/en-us/community/development/pull-request.html before opening a pull request.-->
## Purpose of the pull request
Add the mechanism to increment the workerWaitingTaskCount inside HeartBeat class to fix issue#9921
<!--(For example: This pull request adds checkstyle plugin).-->
## Brief change log
When HeartBeatTask is run priodically from heartBeatExecutor, then it will increment the workerWaitingTaskCount
<!--*(for example:)*
- *Add maven-checkstyle-plugin to root pom.xml*
-->
## Verify this pull request
<!--*(Please pick either of the following options)*-->
This pull request is code cleanup without any test coverage.
<!--*(example:)*
- *Manually verified the change by testing locally.* -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] ruanwenjun commented on a diff in pull request #9980: [Fix-9921] Add the mechanism to increment the workerWaitingTaskCount in HeartBea…
Posted by GitBox <gi...@apache.org>.
ruanwenjun commented on code in PR #9980:
URL: https://github.com/apache/dolphinscheduler/pull/9980#discussion_r934251266
##########
dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/registry/HeartBeatTask.java:
##########
@@ -67,6 +67,15 @@ public HeartBeatTask(long startupTime,
this.heartBeat = new HeartBeat(startupTime, maxCpuloadAvg, reservedMemory, hostWeight, workerThreadCount);
}
+ public void setWorkerWaitingTaskCount(int count){
+ this.workerWaitingTaskCount = count;
+ }
+
+ public void incrementWorkerWaitingTaskCount(){
+ this.workerWaitingTaskCount++;
Review Comment:
+1， this patch cannot solve this problem, you can use a Supplier to replace the workerWaitingTaskCount to solve this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #9980: Add the mechanism to increment the workerWaitingTaskCount in HeartBea…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #9980:
URL: https://github.com/apache/dolphinscheduler/pull/9980#issuecomment-1123169296
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=9980)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9980&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9980&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9980&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9980&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter commented on pull request #9980: Add the mechanism to increment the workerWaitingTaskCount in HeartBea…
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #9980:
URL: https://github.com/apache/dolphinscheduler/pull/9980#issuecomment-1123165973
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/9980?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#9980](https://codecov.io/gh/apache/dolphinscheduler/pull/9980?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (53b3b9b) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/e51d0fb59452178f204547d3d68545da81ca63d0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e51d0fb) will **decrease** coverage by `0.00%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## dev #9980 +/- ##
============================================
- Coverage 40.68% 40.67% -0.01%
- Complexity 4577 4578 +1
============================================
Files 834 834
Lines 33832 33837 +5
Branches 3745 3745
============================================
+ Hits 13763 13764 +1
- Misses 18747 18752 +5
+ Partials 1322 1321 -1
```
| [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/9980?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...olphinscheduler/server/registry/HeartBeatTask.java](https://codecov.io/gh/apache/dolphinscheduler/pull/9980/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL3JlZ2lzdHJ5L0hlYXJ0QmVhdFRhc2suamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...dolphinscheduler/remote/future/ResponseFuture.java](https://codecov.io/gh/apache/dolphinscheduler/pull/9980/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1yZW1vdGUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcmVtb3RlL2Z1dHVyZS9SZXNwb25zZUZ1dHVyZS5qYXZh) | `83.60% <0.00%> (+1.63%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/9980?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/9980?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [e51d0fb...53b3b9b](https://codecov.io/gh/apache/dolphinscheduler/pull/9980?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] JinyLeeChina commented on a diff in pull request #9980: [Fix-9921] Add the mechanism to increment the workerWaitingTaskCount in HeartBea…
Posted by GitBox <gi...@apache.org>.
JinyLeeChina commented on code in PR #9980:
URL: https://github.com/apache/dolphinscheduler/pull/9980#discussion_r873363096
##########
dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/registry/HeartBeatTask.java:
##########
@@ -67,6 +67,15 @@ public HeartBeatTask(long startupTime,
this.heartBeat = new HeartBeat(startupTime, maxCpuloadAvg, reservedMemory, hostWeight, workerThreadCount);
}
+ public void setWorkerWaitingTaskCount(int count){
+ this.workerWaitingTaskCount = count;
+ }
+
+ public void incrementWorkerWaitingTaskCount(){
+ this.workerWaitingTaskCount++;
Review Comment:
I don't think this modification can solve this problem, this value is the number of tasks to be run held by the current worker, not the count of heartbeat
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng closed pull request #9980: [Fix-9921] Add the mechanism to increment the workerWaitingTaskCount in HeartBea…
Posted by GitBox <gi...@apache.org>.
caishunfeng closed pull request #9980: [Fix-9921] Add the mechanism to increment the workerWaitingTaskCount in HeartBea…
URL: https://github.com/apache/dolphinscheduler/pull/9980
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on pull request #9980: [Fix-9921] Add the mechanism to increment the workerWaitingTaskCount in HeartBea…
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on PR #9980:
URL: https://github.com/apache/dolphinscheduler/pull/9980#issuecomment-1245029312
close due to #11232
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #9980: [Fix-9921] Add the mechanism to increment the workerWaitingTaskCount in HeartBea…
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #9980:
URL: https://github.com/apache/dolphinscheduler/pull/9980#issuecomment-1131017651
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=9980)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=9980&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9980&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9980&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9980&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=9980&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] JinyLeeChina commented on a diff in pull request #9980: [Fix-9921] Add the mechanism to increment the workerWaitingTaskCount in HeartBea…
Posted by GitBox <gi...@apache.org>.
JinyLeeChina commented on code in PR #9980:
URL: https://github.com/apache/dolphinscheduler/pull/9980#discussion_r873367014
##########
dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/registry/HeartBeatTask.java:
##########
@@ -67,6 +67,15 @@ public HeartBeatTask(long startupTime,
this.heartBeat = new HeartBeat(startupTime, maxCpuloadAvg, reservedMemory, hostWeight, workerThreadCount);
}
+ public void setWorkerWaitingTaskCount(int count){
+ this.workerWaitingTaskCount = count;
+ }
+
+ public void incrementWorkerWaitingTaskCount(){
+ this.workerWaitingTaskCount++;
Review Comment:
I suggest splitting this class into master and worker to obtain it separately. BTW, please test it fully
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on pull request #9980: [Fix-9921] Add the mechanism to increment the workerWaitingTaskCount in HeartBea…
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on PR #9980:
URL: https://github.com/apache/dolphinscheduler/pull/9980#issuecomment-1132396999
I restarted the failed CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org