You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by GitBox <gi...@apache.org> on 2020/03/20 22:50:13 UTC
[GitHub] [helix] jiajunwang opened a new pull request #904: Fix the
concurrent modification error happens during the HelixManager
initHandlers() call
jiajunwang opened a new pull request #904: Fix the concurrent modification error happens during the HelixManager initHandlers() call
URL: https://github.com/apache/helix/pull/904
### Issues
- [ ] My PR addresses the following Helix issues and references them in the PR description:
#903
### Description
- [ ] Here are some details about my PR, including screenshots of any UI changes:
Fix the concurrent modification error that happens during the HelixManager initHandlers() call.
### Tests
- [ ] The following tests are written for this issue:
TestHandleSession.testConcurrentInitCallbackHandlers()
- [ ] The following is the result of the "mvn test" command on the appropriate module:
Running
### Commits
- [ ] My commits all reference appropriate Apache Helix GitHub issues in their subject lines. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation (Optional)
- [ ] In case of new functionality, my PR adds documentation in the following wiki page:
(Link the GitHub wiki you added)
### Code Quality
- [ ] My diff has been formatted using helix-style.xml
(helix-style-intellij.xml if IntelliJ IDE is used)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] jiajunwang commented on issue #904: Fix the concurrent
modification error happens during the HelixManager initHandlers() call
Posted by GitBox <gi...@apache.org>.
jiajunwang commented on issue #904: Fix the concurrent modification error happens during the HelixManager initHandlers() call
URL: https://github.com/apache/helix/pull/904#issuecomment-601972438
This PR is ready to be merged, approved by @pkuwm
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] pkuwm commented on a change in pull request #904: Fix the
concurrent modification error happens during the HelixManager
initHandlers() call
Posted by GitBox <gi...@apache.org>.
pkuwm commented on a change in pull request #904: Fix the concurrent modification error happens during the HelixManager initHandlers() call
URL: https://github.com/apache/helix/pull/904#discussion_r395929866
##########
File path: helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixManager.java
##########
@@ -1004,7 +1004,10 @@ void waitUntilConnected() {
void initHandlers(List<CallbackHandler> handlers) {
synchronized (this) {
if (handlers != null) {
- for (CallbackHandler handler : handlers) {
+ // get a copy of the list and iterate over the copy list
+ // in case handler.init() modify the original handler list
Review comment:
Nit: modify -> modifies
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] pkuwm commented on a change in pull request #904: Fix the
concurrent modification error happens during the HelixManager
initHandlers() call
Posted by GitBox <gi...@apache.org>.
pkuwm commented on a change in pull request #904: Fix the concurrent modification error happens during the HelixManager initHandlers() call
URL: https://github.com/apache/helix/pull/904#discussion_r395940851
##########
File path: helix-core/src/test/java/org/apache/helix/manager/zk/TestHandleSession.java
##########
@@ -466,6 +468,79 @@ public void testSessionExpiredWhenResetHandlers() throws Exception {
deleteCluster(clusterName);
}
+ @Test
+ public void testConcurrentInitCallbackHandlers() throws Exception {
+ final String clusterName =
+ CLUSTER_PREFIX + "_" + _className + "_" + TestHelper.getTestMethodName();
+ TestHelper.setupEmptyCluster(_gZkClient, clusterName);
+ final String spectatorName = "testConcurrentHandlerChangeSpectator";
+ try {
+ BlockingHandleNewSessionZkHelixManager helixManager =
+ new BlockingHandleNewSessionZkHelixManager(clusterName, spectatorName,
+ InstanceType.SPECTATOR, _gZkClient.getServers());
+ helixManager.connect();
+ // Add two mock listeners that will add more callback handlers while handling INIT or CALLBACK event.
+ // Note that we have to test with 2 separate listeners so one of them has a chance to fail if
+ // there is a concurrent modification exception.
+ helixManager.addLiveInstanceChangeListener(
+ (LiveInstanceChangeListener) (liveInstances, changeContext) -> {
+ if (changeContext.getType() != NotificationContext.Type.FINALIZE) {
+ for (LiveInstance liveInstance : liveInstances) {
+ if (liveInstance.getInstanceName().equals("localhost_1")) {
+ try {
+ helixManager.addCurrentStateChangeListener(
+ (CurrentStateChangeListener) (instanceName, statesInfo, currentStateChangeContext) -> {
+ // empty callback
+ }, liveInstance.getInstanceName(), liveInstance.getEphemeralOwner());
+ } catch (Exception e) {
+ throw new HelixException(
+ "Unexpected exception in the testConcurrentHandlerProcessing.", e);
+ }
+ }
+ }
+ }
+ });
+ helixManager.addLiveInstanceChangeListener(
+ (LiveInstanceChangeListener) (liveInstances, changeContext) -> {
+ if (changeContext.getType() != NotificationContext.Type.FINALIZE) {
+ for (LiveInstance liveInstance : liveInstances) {
+ if (liveInstance.getInstanceName().equals("localhost_2")) {
+ try {
+ helixManager.addCurrentStateChangeListener(
+ (CurrentStateChangeListener) (instanceName, statesInfo, currentStateChangeContext) -> {
+ // empty callback
+ }, liveInstance.getInstanceName(), liveInstance.getEphemeralOwner());
+ } catch (Exception e) {
+ throw new HelixException(
+ "Unexpected exception in the testConcurrentHandlerProcessing.", e);
+ }
+ }
+ }
+ }
+ });
+ // Session expire will trigger all callbacks to be init. And the injected liveInstance
+ // listener will trigger more callbackhandlers to be registered during the init process.
+ ZkTestHelper.asyncExpireSession(helixManager.getZkClient());
+ // Create mock live instance znodes to trigger the internal callback handling logic which will
+ // modify the handler list.
+ setupLiveInstances(clusterName, new int[] { 1, 2 });
+ // Start new session handling so the manager will call the initHandler() for initializing all
+ // existing handlers.
+ helixManager.proceedNewSessionHandling();
+ // Ensure the new session has been processed.
+ TestHelper.verify(() -> helixManager.getHandleNewSessionEndTime() != 0, 3000);
+ // Verify that both new mock current state callback handlers have been initialized normally.
+ // Note that if there is concurrent modification that cause errors, one of the callback will
+ // not be initialized normally.
+ for (CallbackHandler handler : helixManager.getHandlers()) {
+ Assert.assertTrue(handler.isReady(),
+ "CallbackHandler is not initialized as expected. It might be caused by a ConcurrentModificationException");
Review comment:
Nit, wrap this long line by concatenation?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] jiajunwang merged pull request #904: Fix the concurrent
modification error happens during the HelixManager initHandlers() call
Posted by GitBox <gi...@apache.org>.
jiajunwang merged pull request #904: Fix the concurrent modification error happens during the HelixManager initHandlers() call
URL: https://github.com/apache/helix/pull/904
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] pkuwm commented on a change in pull request #904: Fix the
concurrent modification error happens during the HelixManager
initHandlers() call
Posted by GitBox <gi...@apache.org>.
pkuwm commented on a change in pull request #904: Fix the concurrent modification error happens during the HelixManager initHandlers() call
URL: https://github.com/apache/helix/pull/904#discussion_r395940379
##########
File path: helix-core/src/test/java/org/apache/helix/manager/zk/TestHandleSession.java
##########
@@ -466,6 +468,79 @@ public void testSessionExpiredWhenResetHandlers() throws Exception {
deleteCluster(clusterName);
}
+ @Test
+ public void testConcurrentInitCallbackHandlers() throws Exception {
+ final String clusterName =
+ CLUSTER_PREFIX + "_" + _className + "_" + TestHelper.getTestMethodName();
+ TestHelper.setupEmptyCluster(_gZkClient, clusterName);
+ final String spectatorName = "testConcurrentHandlerChangeSpectator";
+ try {
+ BlockingHandleNewSessionZkHelixManager helixManager =
+ new BlockingHandleNewSessionZkHelixManager(clusterName, spectatorName,
+ InstanceType.SPECTATOR, _gZkClient.getServers());
+ helixManager.connect();
+ // Add two mock listeners that will add more callback handlers while handling INIT or CALLBACK event.
+ // Note that we have to test with 2 separate listeners so one of them has a chance to fail if
+ // there is a concurrent modification exception.
+ helixManager.addLiveInstanceChangeListener(
+ (LiveInstanceChangeListener) (liveInstances, changeContext) -> {
+ if (changeContext.getType() != NotificationContext.Type.FINALIZE) {
+ for (LiveInstance liveInstance : liveInstances) {
+ if (liveInstance.getInstanceName().equals("localhost_1")) {
+ try {
+ helixManager.addCurrentStateChangeListener(
+ (CurrentStateChangeListener) (instanceName, statesInfo, currentStateChangeContext) -> {
+ // empty callback
+ }, liveInstance.getInstanceName(), liveInstance.getEphemeralOwner());
+ } catch (Exception e) {
+ throw new HelixException(
+ "Unexpected exception in the testConcurrentHandlerProcessing.", e);
Review comment:
testConcurrentHandlerProcessing -> testConcurrentInitCallbackHandlers.
May be better to just `getTestMethodName()`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] jiajunwang commented on a change in pull request #904: Fix
the concurrent modification error happens during the HelixManager
initHandlers() call
Posted by GitBox <gi...@apache.org>.
jiajunwang commented on a change in pull request #904: Fix the concurrent modification error happens during the HelixManager initHandlers() call
URL: https://github.com/apache/helix/pull/904#discussion_r395946837
##########
File path: helix-core/src/test/java/org/apache/helix/manager/zk/TestHandleSession.java
##########
@@ -466,6 +468,79 @@ public void testSessionExpiredWhenResetHandlers() throws Exception {
deleteCluster(clusterName);
}
+ @Test
+ public void testConcurrentInitCallbackHandlers() throws Exception {
+ final String clusterName =
+ CLUSTER_PREFIX + "_" + _className + "_" + TestHelper.getTestMethodName();
+ TestHelper.setupEmptyCluster(_gZkClient, clusterName);
+ final String spectatorName = "testConcurrentHandlerChangeSpectator";
+ try {
+ BlockingHandleNewSessionZkHelixManager helixManager =
+ new BlockingHandleNewSessionZkHelixManager(clusterName, spectatorName,
+ InstanceType.SPECTATOR, _gZkClient.getServers());
+ helixManager.connect();
+ // Add two mock listeners that will add more callback handlers while handling INIT or CALLBACK event.
+ // Note that we have to test with 2 separate listeners so one of them has a chance to fail if
+ // there is a concurrent modification exception.
+ helixManager.addLiveInstanceChangeListener(
+ (LiveInstanceChangeListener) (liveInstances, changeContext) -> {
+ if (changeContext.getType() != NotificationContext.Type.FINALIZE) {
+ for (LiveInstance liveInstance : liveInstances) {
+ if (liveInstance.getInstanceName().equals("localhost_1")) {
+ try {
+ helixManager.addCurrentStateChangeListener(
+ (CurrentStateChangeListener) (instanceName, statesInfo, currentStateChangeContext) -> {
+ // empty callback
+ }, liveInstance.getInstanceName(), liveInstance.getEphemeralOwner());
+ } catch (Exception e) {
+ throw new HelixException(
+ "Unexpected exception in the testConcurrentHandlerProcessing.", e);
+ }
+ }
+ }
+ }
+ });
+ helixManager.addLiveInstanceChangeListener(
+ (LiveInstanceChangeListener) (liveInstances, changeContext) -> {
+ if (changeContext.getType() != NotificationContext.Type.FINALIZE) {
+ for (LiveInstance liveInstance : liveInstances) {
+ if (liveInstance.getInstanceName().equals("localhost_2")) {
+ try {
+ helixManager.addCurrentStateChangeListener(
+ (CurrentStateChangeListener) (instanceName, statesInfo, currentStateChangeContext) -> {
+ // empty callback
+ }, liveInstance.getInstanceName(), liveInstance.getEphemeralOwner());
+ } catch (Exception e) {
+ throw new HelixException(
+ "Unexpected exception in the testConcurrentHandlerProcessing.", e);
+ }
+ }
+ }
+ }
+ });
+ // Session expire will trigger all callbacks to be init. And the injected liveInstance
+ // listener will trigger more callbackhandlers to be registered during the init process.
+ ZkTestHelper.asyncExpireSession(helixManager.getZkClient());
+ // Create mock live instance znodes to trigger the internal callback handling logic which will
+ // modify the handler list.
+ setupLiveInstances(clusterName, new int[] { 1, 2 });
+ // Start new session handling so the manager will call the initHandler() for initializing all
+ // existing handlers.
+ helixManager.proceedNewSessionHandling();
+ // Ensure the new session has been processed.
+ TestHelper.verify(() -> helixManager.getHandleNewSessionEndTime() != 0, 3000);
+ // Verify that both new mock current state callback handlers have been initialized normally.
+ // Note that if there is concurrent modification that cause errors, one of the callback will
+ // not be initialized normally.
+ for (CallbackHandler handler : helixManager.getHandlers()) {
+ Assert.assertTrue(handler.isReady(),
+ "CallbackHandler is not initialized as expected. It might be caused by a ConcurrentModificationException");
Review comment:
This is done by the auto format.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] pkuwm commented on a change in pull request #904: Fix the
concurrent modification error happens during the HelixManager
initHandlers() call
Posted by GitBox <gi...@apache.org>.
pkuwm commented on a change in pull request #904: Fix the concurrent modification error happens during the HelixManager initHandlers() call
URL: https://github.com/apache/helix/pull/904#discussion_r395941462
##########
File path: helix-core/src/test/java/org/apache/helix/manager/zk/TestHandleSession.java
##########
@@ -466,6 +468,79 @@ public void testSessionExpiredWhenResetHandlers() throws Exception {
deleteCluster(clusterName);
}
+ @Test
+ public void testConcurrentInitCallbackHandlers() throws Exception {
+ final String clusterName =
+ CLUSTER_PREFIX + "_" + _className + "_" + TestHelper.getTestMethodName();
+ TestHelper.setupEmptyCluster(_gZkClient, clusterName);
+ final String spectatorName = "testConcurrentHandlerChangeSpectator";
+ try {
+ BlockingHandleNewSessionZkHelixManager helixManager =
+ new BlockingHandleNewSessionZkHelixManager(clusterName, spectatorName,
+ InstanceType.SPECTATOR, _gZkClient.getServers());
+ helixManager.connect();
+ // Add two mock listeners that will add more callback handlers while handling INIT or CALLBACK event.
+ // Note that we have to test with 2 separate listeners so one of them has a chance to fail if
+ // there is a concurrent modification exception.
+ helixManager.addLiveInstanceChangeListener(
+ (LiveInstanceChangeListener) (liveInstances, changeContext) -> {
+ if (changeContext.getType() != NotificationContext.Type.FINALIZE) {
+ for (LiveInstance liveInstance : liveInstances) {
+ if (liveInstance.getInstanceName().equals("localhost_1")) {
+ try {
+ helixManager.addCurrentStateChangeListener(
+ (CurrentStateChangeListener) (instanceName, statesInfo, currentStateChangeContext) -> {
+ // empty callback
+ }, liveInstance.getInstanceName(), liveInstance.getEphemeralOwner());
+ } catch (Exception e) {
+ throw new HelixException(
+ "Unexpected exception in the testConcurrentHandlerProcessing.", e);
+ }
+ }
+ }
+ }
Review comment:
Can we wrap this block into a private method to avoid the duplicate code in these 2 listeners?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] pkuwm commented on a change in pull request #904: Fix the
concurrent modification error happens during the HelixManager
initHandlers() call
Posted by GitBox <gi...@apache.org>.
pkuwm commented on a change in pull request #904: Fix the concurrent modification error happens during the HelixManager initHandlers() call
URL: https://github.com/apache/helix/pull/904#discussion_r395940156
##########
File path: helix-core/src/test/java/org/apache/helix/manager/zk/TestHandleSession.java
##########
@@ -466,6 +468,79 @@ public void testSessionExpiredWhenResetHandlers() throws Exception {
deleteCluster(clusterName);
}
+ @Test
+ public void testConcurrentInitCallbackHandlers() throws Exception {
+ final String clusterName =
+ CLUSTER_PREFIX + "_" + _className + "_" + TestHelper.getTestMethodName();
+ TestHelper.setupEmptyCluster(_gZkClient, clusterName);
+ final String spectatorName = "testConcurrentHandlerChangeSpectator";
+ try {
+ BlockingHandleNewSessionZkHelixManager helixManager =
+ new BlockingHandleNewSessionZkHelixManager(clusterName, spectatorName,
+ InstanceType.SPECTATOR, _gZkClient.getServers());
+ helixManager.connect();
+ // Add two mock listeners that will add more callback handlers while handling INIT or CALLBACK event.
+ // Note that we have to test with 2 separate listeners so one of them has a chance to fail if
+ // there is a concurrent modification exception.
+ helixManager.addLiveInstanceChangeListener(
+ (LiveInstanceChangeListener) (liveInstances, changeContext) -> {
+ if (changeContext.getType() != NotificationContext.Type.FINALIZE) {
+ for (LiveInstance liveInstance : liveInstances) {
+ if (liveInstance.getInstanceName().equals("localhost_1")) {
+ try {
+ helixManager.addCurrentStateChangeListener(
+ (CurrentStateChangeListener) (instanceName, statesInfo, currentStateChangeContext) -> {
+ // empty callback
+ }, liveInstance.getInstanceName(), liveInstance.getEphemeralOwner());
+ } catch (Exception e) {
+ throw new HelixException(
+ "Unexpected exception in the testConcurrentHandlerProcessing.", e);
+ }
+ }
+ }
+ }
+ });
+ helixManager.addLiveInstanceChangeListener(
+ (LiveInstanceChangeListener) (liveInstances, changeContext) -> {
+ if (changeContext.getType() != NotificationContext.Type.FINALIZE) {
+ for (LiveInstance liveInstance : liveInstances) {
+ if (liveInstance.getInstanceName().equals("localhost_2")) {
+ try {
+ helixManager.addCurrentStateChangeListener(
+ (CurrentStateChangeListener) (instanceName, statesInfo, currentStateChangeContext) -> {
+ // empty callback
+ }, liveInstance.getInstanceName(), liveInstance.getEphemeralOwner());
+ } catch (Exception e) {
+ throw new HelixException(
+ "Unexpected exception in the testConcurrentHandlerProcessing.", e);
Review comment:
testConcurrentHandlerProcessing -> testConcurrentInitCallbackHandlers
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org