You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@zookeeper.apache.org by GitBox <gi...@apache.org> on 2020/02/21 04:46:30 UTC
[GitHub] [zookeeper] suhasdantkale opened a new pull request #1256:
ZOOKEEPER-2164: Quorum members can not rejoin after restart
suhasdantkale opened a new pull request #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256
(Ported from Mate Szalay-Beko's work in master branch.
Refer to https://github.com/apache/zookeeper/pull/1254)
Ever since ZOOKEEPER-107 (released in 3.5.0) the servers are sending
their addresses during initial connection requests. The receiving
server can potentially use these addresses to send back a new
connection request if the challenge is won by the receiver.
If the server config contains wildcard address (e.g. 0.0.0.0 in case
of IPv4) then the first connection request sent by A to B will contain
this address. If the ID of A is smaller than the ID of B, then A will
lose the challenge and the second connection request sent back by B
will never reach A, as B will send the initial message to 0.0.0.0.
So in any 3.5+ ZooKeeper, if wildcard addresses are used in the configs,
then there might be some servers never able to rejoin to the quorum
after they got restarted.
In 3.5+ for backward compatibility reasons (needed during rolling
upgrade) there is a version of the QuorumCnxManager.connectOne()
method that needs no election address but use the last known address
to initiate the connection. In this commit, we simply call this method
if the address is a wildcard address.
I also added a few restart realted tests, to make sure that restart
still works when we don't use no wildcard addresses. We can not test
the original error with unit tests, as it would require to start
the quorum on multiple hosts.
I also tested manually the rolling restart both with and without
wildcard addresses in the config.
Reviewer: Mate Szalay-Beko <sz...@gmail.com>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] symat commented on issue #1256: ZOOKEEPER-2164: Quorum
members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
symat commented on issue #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#issuecomment-591425847
@eolivelli what do you think about this issue / PR?
(also an other duplicate jira solved by this is: https://issues.apache.org/jira/browse/ZOOKEEPER-3725 )
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] symat commented on a change in pull request #1256:
ZOOKEEPER-2164: Quorum members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
symat commented on a change in pull request #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#discussion_r382449840
##########
File path: zookeeper-server/src/test/java/org/apache/zookeeper/test/QuorumUtil.java
##########
@@ -21,13 +21,7 @@
import java.io.File;
import java.io.IOException;
import java.net.InetSocketAddress;
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.HashMap;
-import java.util.LinkedHashSet;
-import java.util.List;
-import java.util.Map;
-import java.util.Set;
+import java.util.*;
Review comment:
please avoid the wildcard imports
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] suhasdantkale commented on a change in pull request
#1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
suhasdantkale commented on a change in pull request #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#discussion_r386202543
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumCnxManager.java
##########
@@ -25,11 +25,7 @@
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
-import java.net.InetSocketAddress;
-import java.net.ServerSocket;
-import java.net.Socket;
-import java.net.SocketException;
-import java.net.SocketTimeoutException;
+import java.net.*;
Review comment:
Done that. Yeah whenever I compile it with IntelliJ, it gets automatically added.
Thanks for pointing this. Sorry, I missed to check this before pushing the branch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] suhasdantkale commented on a change in pull request
#1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
suhasdantkale commented on a change in pull request #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#discussion_r386202543
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumCnxManager.java
##########
@@ -25,11 +25,7 @@
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
-import java.net.InetSocketAddress;
-import java.net.ServerSocket;
-import java.net.Socket;
-import java.net.SocketException;
-import java.net.SocketTimeoutException;
+import java.net.*;
Review comment:
Done that. Yeah whenever I compile it with I.telliJ, it gets automatically added.
Thanks for pointing this,
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] eolivelli closed pull request #1256: ZOOKEEPER-2164:
Quorum members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
eolivelli closed pull request #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] suhasdantkale commented on issue #1256: ZOOKEEPER-2164:
Quorum members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
suhasdantkale commented on issue #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#issuecomment-589496850
@symat,
This is the PR on 3.5
Please let me know your feedback.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] suhasdantkale commented on issue #1256: ZOOKEEPER-2164:
Quorum members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
suhasdantkale commented on issue #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#issuecomment-589761977
Thanks Mate for the review feedback. I've incorporated it here- [https://github.com/apache/zookeeper/pull/1256/commits/4d47f98e9c52b453e1b56bd78c5e848466690882]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] symat commented on issue #1256: ZOOKEEPER-2164: Quorum
members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
symat commented on issue #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#issuecomment-592421511
(I answered to this comment on the Jira ticket)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] symat commented on issue #1256: ZOOKEEPER-2164: Quorum
members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
symat commented on issue #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#issuecomment-591974372
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] MatthewKnight-NG commented on issue #1256:
ZOOKEEPER-2164: Quorum members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
MatthewKnight-NG commented on issue #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#issuecomment-592019991
Tried this with my NiFi cluster as a client. Leader election seems to succeed, but on the client side reconnecting to the cluster still never succeeds.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] symat commented on a change in pull request #1256:
ZOOKEEPER-2164: Quorum members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
symat commented on a change in pull request #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#discussion_r382448803
##########
File path: zookeeper-server/src/test/java/org/apache/zookeeper/ZKTestCase.java
##########
@@ -25,6 +25,8 @@
import org.junit.runner.Description;
import org.junit.runner.RunWith;
import org.junit.runners.model.FrameworkMethod;
+import java.time.LocalDateTime;
Review comment:
I think you will get a checkstyle error as the imports needs to be in alphabetical order.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] eolivelli commented on issue #1256: ZOOKEEPER-2164:
Quorum members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
eolivelli commented on issue #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#issuecomment-598276066
committed to branch-3.5
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] suhasdantkale commented on issue #1256: ZOOKEEPER-2164:
Quorum members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
suhasdantkale commented on issue #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#issuecomment-593228921
> I don't know why JenkinsAnt is failing, or how to re-trigger it...
I also am not sure. There are no tests failed. But the job was shown as failed.
I'm waiting for the JenkinsAnt run on my new patch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] symat commented on a change in pull request #1256:
ZOOKEEPER-2164: Quorum members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
symat commented on a change in pull request #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#discussion_r382449363
##########
File path: zookeeper-server/src/test/java/org/apache/zookeeper/ZKTestCase.java
##########
@@ -75,5 +77,30 @@ public void failed(Throwable e, Description method) {
}
};
+ public interface WaitForCondition {
+ /**
+ * @return true when success
+ */
+ boolean evaluate();
+ }
+
+ /**
+ * Wait for condition to be true; otherwise fail the test if it exceed
+ * timeout
+ * @param msg error message to print when fail
+ * @param condition condition to evaluate
+ * @param timeout timeout in seconds
+ * @throws InterruptedException
+ */
+ public void waitFor(String msg, WaitForCondition condition, int timeout) throws InterruptedException {
Review comment:
this is a very useful helper function, great that we will have it also for the 3.5 branch!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] eolivelli commented on issue #1256: ZOOKEEPER-2164:
Quorum members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
eolivelli commented on issue #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#issuecomment-591433146
@symat I am sorry I don't have time to check today.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] MatthewKnight-NG removed a comment on issue #1256:
ZOOKEEPER-2164: Quorum members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
MatthewKnight-NG removed a comment on issue #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#issuecomment-592019991
Tried this with my NiFi cluster as a client. Leader election seems to succeed, but on the client side reconnecting to the cluster still never succeeds.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] symat commented on issue #1256: ZOOKEEPER-2164: Quorum
members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
symat commented on issue #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#issuecomment-592482817
@suhasdantkale I got 1-2 comments from Enrico on the master branch PR (https://github.com/apache/zookeeper/pull/1254) about the tests. Can you push those changes here as well?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] MatthewKnight-NG commented on issue #1256:
ZOOKEEPER-2164: Quorum members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
MatthewKnight-NG commented on issue #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#issuecomment-592019528
Tried this with my NiFi cluster as a client. Leader election seems to succeed, but on the client side reconnecting to the cluster still never succeeds.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] symat commented on issue #1256: ZOOKEEPER-2164: Quorum
members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
symat commented on issue #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#issuecomment-593856272
@nkalmar @eolivelli this is the same PR you already reviewed for the master/branch-3.6 (https://github.com/apache/zookeeper/pull/1254), but due to the multi-address feature it had to be modified slightly for branch-3.5. Could you please take a look?
@suhasdantkale already incorporated the comments from @eolivelli from the other PR too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] symat commented on a change in pull request #1256:
ZOOKEEPER-2164: Quorum members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
symat commented on a change in pull request #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#discussion_r382440618
##########
File path: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/QuorumCnxManager.java
##########
@@ -25,11 +25,7 @@
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
-import java.net.InetSocketAddress;
-import java.net.ServerSocket;
-import java.net.Socket;
-import java.net.SocketException;
-import java.net.SocketTimeoutException;
+import java.net.*;
Review comment:
please avoid using * imports. I know this is in the default settings for IntelliJ, I always hit this problem when I use a new IntelliJ installation :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] MatthewKnight-NG edited a comment on issue #1256:
ZOOKEEPER-2164: Quorum members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
MatthewKnight-NG edited a comment on issue #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#issuecomment-592019528
Tried this with my NiFi cluster as a client. Leader election seems to succeed, but on the client side reconnecting to the cluster still never succeeds.
Have yall done a similar test to make sure clients stay functional? (want to figure out if it's just me or not)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zookeeper] symat commented on issue #1256: ZOOKEEPER-2164: Quorum
members can not rejoin after restart
Posted by GitBox <gi...@apache.org>.
symat commented on issue #1256: ZOOKEEPER-2164: Quorum members can not rejoin after restart
URL: https://github.com/apache/zookeeper/pull/1256#issuecomment-592483034
I don't know why JenkinsAnt is failing, or how to re-trigger it...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services