Re: WARN [QuorumPeer[myid=1]/0.0.0.0:2181:Follower@90] - Exception when following the leader java.net.SocketTimeoutException: Read timed out

[Andor Molnar <an...@cloudera.com>]

Hi svanschalkwyk,

I'm not really sure.
It could be a network issue or a slow leader which is not able to answer
follower request in a timely fashion.
Does it happen on all followers or only one?

Regards,
Andor



On Tue, Feb 20, 2018 at 6:31 PM, svanschalkwyk <sv...@gmail.com>
wrote:

> Could this be a network issue? (I'm also seeing socket timeouts)...
>
> Periodically getting these.
>  /WARN  [QuorumPeer[myid=1]/0.0.0.0:2181:Follower@90] - Exception when
> following the leader
> java.net.SocketTimeoutException: Read timed out
>         at java.net.SocketInputStream.socketRead0(Native Method)
>         at java.net.SocketInputStream.socketRead(SocketInputStream.
> java:116)
>         at java.net.SocketInputStream.read(SocketInputStream.java:171)
>         at java.net.SocketInputStream.read(SocketInputStream.java:141)
>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
>         at java.io.DataInputStream.readInt(DataInputStream.java:387)
>         at
> org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
>         at
> org.apache.zookeeper.server.quorum.QuorumPacket.
> deserialize(QuorumPacket.java:85)
>         at
> org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99)
>         at
> org.apache.zookeeper.server.quorum.Learner.readPacket(Learner.java:153)
>         at
> org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:86)
>         at
> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:981)
> 2018-02-19 22:20:21,181 [myid:1] - INFO
> [QuorumPeer[myid=1]/0.0.0.0:2181:Follower@169] - shutdown called
> java.lang.Exception: shutdown Follower
>         at
> org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:169)
>         at
> org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:985)
> 2018-02-19 22:20:21,183 [myid:1] - INFO
> [QuorumPeer[myid=1]/0.0.0.0:2181:FollowerZooKeeperServer@140] - Shutting
> down
> 2018-02-19 22:20:21,183 [myid:1] - INFO
> [QuorumPeer[myid=1]/0.0.0.0:2181:ZooKeeperServer@501] - shutting down
> 2018-02-19 22:20:21,184 [myid:1] - INFO
> [QuorumPeer[myid=1]/0.0.0.0:2181:FollowerRequestProcessor@107] - Shutting
> down
> 2018-02-19 22:20:21,184 [myid:1] - INFO
> [FollowerRequestProcessor:1:FollowerRequestProcessor@97] -
> FollowerRequestProcessor exited loop!
> 2018-02-19 22:20:21,184 [myid:1] - INFO
> [QuorumPeer[myid=1]/0.0.0.0:2181:CommitProcessor@184] - Shutting down
> 2018-02-19 22:20:21,185 [myid:1] - INFO
> [QuorumPeer[myid=1]/0.0.0.0:2181:FinalRequestProcessor@403] - shutdown of
> request processor complete
> 2018-02-19 22:20:21,185 [myid:1] - INFO
> [CommitProcessor:1:CommitProcessor@153] - CommitProcessor exited loop!
> 2018-02-19 22:20:21,185 [myid:1] - INFO
> [QuorumPeer[myid=1]/0.0.0.0:2181:SyncRequestProcessor@208] - Shutting down
> 2018-02-19 22:20:21,185 [myid:1] - INFO
> [SyncThread:1:SyncRequestProcessor@186] - SyncRequestProcessor exited!
> 2018-02-19 22:20:21,186 [myid:1] - INFO
> [QuorumPeer[myid=1]/0.0.0.0:2181:QuorumPeer@909] - LOOKING
> 2018-02-19 22:20:21,189 [myid:1] - INFO
> [QuorumPeer[myid=1]/0.0.0.0:2181:FastLeaderElection@820] - New election.
> My
> id =  1, proposed zxid=0x38000004e9
> 2018-02-19 22:20:21,189 [myid:1] - INFO
> [WorkerReceiver[myid=1]:FastLeaderElection@602] - Notification: 1 (message
> fo/
>
> zoo.cfg has
>
> # The number of milliseconds of each tick
> tickTime=2000
> # The number of ticks that the initial
> # synchronization phase can take
> initLimit=100
> # The number of ticks that can pass between
> # sending a request and getting an acknowledgement
> syncLimit=10
> # the directory where the snapshot is stored.
> # do not use /tmp for storage, /tmp here is just
> # example sakes.
> dataDir=/data/zookeeper
> # the port at which the clients will connect
> clientPort=2181
> # the maximum number of client connections.
> # increase this if you need to handle more clients
> maxClientCnxns=200
> #
> # Be sure to read the maintenance section of the
> # administrator guide before turning on autopurge.
> #
> # http://zookeeper.apache.org/doc/current/zookeeperAdmin.
> html#sc_maintenance
> #
> # The number of snapshots to retain in dataDir
> autopurge.snapRetainCount=50
> # Purge task interval in hours
> # Set to "0" to disable auto purge feature
> autopurge.purgeInterval=3
> server.1=zk01:2888:3888
> server.2=zk02:2888:3888
> server.3=zk03:2888:3888
>
> I have increased initLimit and syncLimit.
>
> Is this a network issue?
>
>
>
>
> --
> Sent from: http://zookeeper-user.578899.n2.nabble.com/
>