You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "alei1206 (via GitHub)" <gi...@apache.org> on 2024/01/10 12:00:24 UTC
[PR] [Master][Bug] use workerNodeInfoWriteLock to update workerNodeInfo [dolphinscheduler]
alei1206 opened a new pull request, #15461:
URL: https://github.com/apache/dolphinscheduler/pull/15461
<!--Thanks very much for contributing to Apache DolphinScheduler, we are happy that you want to help us improve DolphinScheduler! -->
## Purpose of the pull request
Fix #15460 use workerNodeInfoWriteLock to update workerNodeInfo
<!--(For example: This pull request adds checkstyle plugin).-->
## Brief change log
<!--*(for example:)*
- *Add maven-checkstyle-plugin to root pom.xml*
-->
## Verify this pull request
<!--*(Please pick either of the following options)*-->
This pull request is code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
<!--*(example:)*
- *Added dolphinscheduler-dao tests for end-to-end.*
- *Added CronUtilsTest to verify the change.*
- *Manually verified the change by testing locally.* -->
(or)
If your pull request contain incompatible change, you should also add it to `docs/docs/en/guide/upgrede/incompatible.md`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Master][Bug] use workerNodeInfoWriteLock to update workerNodeInfo [dolphinscheduler]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #15461:
URL: https://github.com/apache/dolphinscheduler/pull/15461#issuecomment-2109044067
This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Master][Bug] use workerNodeInfoWriteLock to update workerNodeInfo [dolphinscheduler]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #15461: [Master][Bug] use workerNodeInfoWriteLock to update workerNodeInfo
URL: https://github.com/apache/dolphinscheduler/pull/15461
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Master][Bug] use workerNodeInfoWriteLock to update workerNodeInfo [dolphinscheduler]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #15461:
URL: https://github.com/apache/dolphinscheduler/pull/15461#issuecomment-1886351763
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=15461) **Quality Gate passed**
Kudos, no new issues were introduced!
[0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=15461&resolved=false&inNewCodePeriod=true)
[0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=15461&resolved=false&inNewCodePeriod=true)
[0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=15461&metric=new_coverage&view=list)
[0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=15461&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=15461)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Master][Bug] use workerNodeInfoWriteLock to update workerNodeInfo [dolphinscheduler]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #15461:
URL: https://github.com/apache/dolphinscheduler/pull/15461#issuecomment-1888671642
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=15461) **Quality Gate passed**
Kudos, no new issues were introduced!
[0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=15461&resolved=false&inNewCodePeriod=true)
[0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=15461&resolved=false&inNewCodePeriod=true)
[0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=15461&metric=new_coverage&view=list)
[0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=15461&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=15461)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Master][Bug] use workerNodeInfoWriteLock to update workerNodeInfo [dolphinscheduler]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #15461:
URL: https://github.com/apache/dolphinscheduler/pull/15461#issuecomment-1886343513
## [Codecov](https://app.codecov.io/gh/apache/dolphinscheduler/pull/15461?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: `2 lines` in your changes are missing coverage. Please review.
> Comparison is base [(`309c8c9`)](https://app.codecov.io/gh/apache/dolphinscheduler/commit/309c8c97a9b9003244496dedc7861a5026f9441a?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) 38.11% compared to head [(`6502707`)](https://app.codecov.io/gh/apache/dolphinscheduler/pull/15461?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) 38.11%.
> :exclamation: Current head 6502707 differs from pull request most recent head 4eda754. Consider uploading reports for the commit 4eda754 to get more accurate results
| [Files](https://app.codecov.io/gh/apache/dolphinscheduler/pull/15461?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...uler/server/master/registry/ServerNodeManager.java](https://app.codecov.io/gh/apache/dolphinscheduler/pull/15461?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZG9scGhpbnNjaGVkdWxlci1tYXN0ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvc2VydmVyL21hc3Rlci9yZWdpc3RyeS9TZXJ2ZXJOb2RlTWFuYWdlci5qYXZh) | 0.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/dolphinscheduler/pull/15461?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## dev #15461 +/- ##
=========================================
Coverage 38.11% 38.11%
Complexity 4697 4697
=========================================
Files 1299 1299
Lines 44774 44774
Branches 4797 4797
=========================================
Hits 17066 17066
Misses 25861 25861
Partials 1847 1847
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/dolphinscheduler/pull/15461?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Master][Bug] use workerNodeInfoWriteLock to update workerNodeInfo [dolphinscheduler]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #15461:
URL: https://github.com/apache/dolphinscheduler/pull/15461#issuecomment-1888670978
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=15461) **Quality Gate passed**
Kudos, no new issues were introduced!
[0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=15461&resolved=false&inNewCodePeriod=true)
[0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=15461&resolved=false&inNewCodePeriod=true)
[0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=15461&metric=new_coverage&view=list)
[0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=15461&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=15461)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Master][Bug] use workerNodeInfoWriteLock to update workerNodeInfo [dolphinscheduler]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #15461:
URL: https://github.com/apache/dolphinscheduler/pull/15461#issuecomment-2121475388
This pull request has been closed because it has not had recent activity. You could reopen it if you try to continue your work, and anyone who are interested in it are encouraged to continue work on this pull request.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Master][Bug] use workerNodeInfoWriteLock to update workerNodeInfo [dolphinscheduler]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #15461:
URL: https://github.com/apache/dolphinscheduler/pull/15461#issuecomment-1886351218
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=15461) **Quality Gate passed**
Kudos, no new issues were introduced!
[0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=15461&resolved=false&inNewCodePeriod=true)
[0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=15461&resolved=false&inNewCodePeriod=true)
[0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=15461&metric=new_coverage&view=list)
[0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=15461&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=15461)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Master][Bug] use workerNodeInfoWriteLock to update workerNodeInfo [dolphinscheduler]
Posted by "EricGao888 (via GitHub)" <gi...@apache.org>.
EricGao888 commented on code in PR #15461:
URL: https://github.com/apache/dolphinscheduler/pull/15461#discussion_r1450041515
##########
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/registry/ServerNodeManager.java:
##########
@@ -234,14 +234,14 @@ private void updateMasterNodes() {
}
private void updateWorkerNodes() {
- workerGroupWriteLock.lock();
Review Comment:
https://github.com/apache/dolphinscheduler/blob/bd48c991783b2e0ea0c602f6ef6c9a09c92e7b42/dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/registry/ServerNodeManager.java#L248-L277
I guess we need to do both. `updateWorkerGroupMappings` also accesses `workerNodeInfo`, correct?
cc @ruanwenjun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Master][Bug] use workerNodeInfoWriteLock to update workerNodeInfo [dolphinscheduler]
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on code in PR #15461:
URL: https://github.com/apache/dolphinscheduler/pull/15461#discussion_r1451743293
##########
dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/registry/ServerNodeManager.java:
##########
@@ -234,14 +234,14 @@ private void updateMasterNodes() {
}
private void updateWorkerNodes() {
- workerGroupWriteLock.lock();
Review Comment:
Yes, the logic in ServerNodeManager is complex, some lock is not needed.
It's better to split the ServerNodeManager into `MasterInstanceManager`, `WorkerInstanceManager`.
And the lock here is not needed, we don't need to use a lock here, each refresh we directly replace the whole map in memory is enough, right now we use read/write lock is overdesign.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org