You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@zookeeper.apache.org by "jx (Jira)" <ji...@apache.org> on 2020/01/06 02:46:00 UTC
[jira] [Created] (ZOOKEEPER-3673) Getting a snapshot from leader
cause Connection reset shutdown Follower and repeated forever
jx created ZOOKEEPER-3673:
-----------------------------
Summary: Getting a snapshot from leader cause Connection reset shutdown Follower and repeated forever
Key: ZOOKEEPER-3673
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3673
Project: ZooKeeper
Issue Type: Bug
Affects Versions: 3.4.12
Reporter: jx
when one broker restart, zk repeated forever
1. Getting a snapshot from leader
2. Snapshotting to disk
3. cause Connection reset
4. shutdown Follower
Does get snapshot from leader or snapshot to disk cause synclimit timeout ?
{code:java}
// code placeholder
2020-01-05 22:56:31,168 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2020-01-05 22:56:31,169 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:Environment@100] - Server environment:java.io.tmpdir=/tmp
2020-01-05 22:56:31,169 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:Environment@100] - Server environment:java.compiler=<NA>
2020-01-05 22:56:31,169 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:Environment@100] - Server environment:os.name=Linux
2020-01-05 22:56:31,169 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:Environment@100] - Server environment:os.arch=amd64
2020-01-05 22:56:31,169 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:Environment@100] - Server environment:os.version=3.10.104-1-tlinux2_kvm_guest-0022.tl2
2020-01-05 22:56:31,169 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:Environment@100] - Server environment:user.name=user_00
2020-01-05 22:56:31,169 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:Environment@100] - Server environment:user.home=/home/user_00
2020-01-05 22:56:31,170 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:Environment@100] - Server environment:user.dir=/usr/local/services/zookeeper-3_4_12-V8-32-400-cluster-001-0.0
2020-01-05 22:56:31,171 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:ZooKeeperServer@173] - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir /data/zookeeper/version-2 snapdir /data/zookeeper/version-2
2020-01-05 22:56:31,183 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:Follower@65] - FOLLOWING - LEADER ELECTION TOOK - 81
2020-01-05 22:56:31,185 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:QuorumPeer$QuorumServer@184] - Resolved hostname: 100.94.122.151 to address: /100.94.122.151
2020-01-05 22:56:31,190 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:Learner@336] - Getting a snapshot from leader 0xb1a0a15a6
2020-01-05 22:57:19,023 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:FileTxnSnapLog@296] - Snapshotting: 0xb1a0a15a6 to /data/zookeeper/version-2/snapshot.b1a0a15a6
2020-01-05 22:57:53,554 [myid:3] - WARN [QuorumPeer[myid=3]/0.0.0.0:2181:Learner@387] - Got zxid 0xb1a0a15a7 expected 0x1
2020-01-05 22:57:53,596 [myid:3] - WARN [QuorumPeer[myid=3]/0.0.0.0:2181:Follower@90] - Exception when following the leader
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:210)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at java.io.DataInputStream.readFully(DataInputStream.java:195)
at java.io.DataInputStream.readFully(DataInputStream.java:169)
at org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:94)
at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:87)
at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99)
at org.apache.zookeeper.server.quorum.Learner.readPacket(Learner.java:153)
at org.apache.zookeeper.server.quorum.Learner.syncWithLeader(Learner.java:380)
at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:83)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:981)
2020-01-05 22:57:53,615 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:Follower@169] - shutdown called
java.lang.Exception: shutdown Follower
at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:169)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:985)
2020-01-05 22:57:53,615 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:FollowerZooKeeperServer@140] - Shutting down
2020-01-05 22:57:53,615 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:QuorumPeer@909] - LOOKING
2020-01-05 22:57:53,616 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:FastLeaderElection@813] - New election. My id = 3, proposed zxid=0xb1a0a15a6
2020-01-05 22:57:53,617 [myid:3] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@595] - Notification: 1 (message format version), 3 (n.leader), 0xb1a0a15a6 (n.zxid), 0x2 (n.round), LOOKING (n.state), 3 (n.sid), 0xb (n.peerEpoch) LOOKING (my state)
2020-01-05 22:57:53,618 [myid:3] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@595] - Notification: 1 (message format version), 2 (n.leader), 0xa0000001b (n.zxid), 0x1 (n.round), FOLLOWING (n.state), 1 (n.sid), 0xb (n.peerEpoch) LOOKING (my state)
2020-01-05 22:57:53,618 [myid:3] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@595] - Notification: 1 (message format version), 2 (n.leader), 0xa0000001b (n.zxid), 0x1 (n.round), LEADING (n.state), 2 (n.sid), 0xb (n.peerEpoch) LOOKING (my state)
2020-01-05 22:57:53,619 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:QuorumPeer@979] - FOLLOWING
2020-01-05 22:57:53,619 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:ZooKeeperServer@173] - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir /data/zookeeper/version-2 snapdir /data/zookeeper/version-2
2020-01-05 22:57:53,619 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:Follower@65] - FOLLOWING - LEADER ELECTION TOOK - 3
2020-01-05 22:57:53,619 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:QuorumPeer$QuorumServer@184] - Resolved hostname: 100.94.122.151 to address: /100.94.122.151
2020-01-05 22:57:53,628 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:Learner@336] - Getting a snapshot from leader 0xb1a0a4842
2020-01-05 22:58:34,196 [myid:3] - INFO [QuorumPeer[myid=3]/0.0.0.0:2181:FileTxnSnapLog@296] - Snapshotting: 0xb1a0a4842 to /data/zookeeper/version-2/snapshot.b1a0a4842
2020-01-05 22:59:03,670 [myid:3] - WARN [QuorumPeer[myid=3]/0.0.0.0:2181:Learner@387] - Got zxid 0xb1a0a4843 expected 0x1
2020-01-05 22:59:03,692 [myid:3] - WARN [QuorumPeer[myid=3]/0.0.0.0:2181:Follower@90] - Exception when following the leader
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:210)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)