You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@curator.apache.org by Benjamin Jaton <bj...@radiantlogic.com> on 2015/01/08 20:34:00 UTC
Curator retry strategy fails
Hello,
I came across a scenario where the connection state changes from SUSPENDED
to LOST within 17 seconds.
However the curator is configured with:
session timeout = 72000
connection timeout = 18000
Retry Strategy = RetryNTimes(3, 5000)
2015-01-04 16:18:22,387 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
*2015-01-04 16:18:22,531 INFO ConnectionStateManager:151 - State change:
SUSPENDED*
2015-01-04 16:18:23,114 INFO ClientCnxn:968 - Opening socket connection to
server host5/204.53.107.248:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:23,115 INFO ClientCnxn:849 - Socket connection
established to host5/204.53.107.248:2181, initiating session
2015-01-04 16:18:23,124 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:23,611 INFO ClientCnxn:968 - Opening socket connection to
server host6/204.53.107.249:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:23,611 INFO ClientCnxn:849 - Socket connection
established to host6/204.53.107.249:2181, initiating session
2015-01-04 16:18:23,623 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:25,507 INFO ClientCnxn:968 - Opening socket connection to
server host4/204.53.107.247:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:25,507 INFO ClientCnxn:849 - Socket connection
established to host4/204.53.107.247:2181, initiating session
2015-01-04 16:18:25,509 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:26,171 INFO ClientCnxn:968 - Opening socket connection to
server host5/204.53.107.248:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:26,171 INFO ClientCnxn:849 - Socket connection
established to host5/204.53.107.248:2181, initiating session
2015-01-04 16:18:26,172 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:26,817 INFO ClientCnxn:968 - Opening socket connection to
server host6/204.53.107.249:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:26,818 INFO ClientCnxn:849 - Socket connection
established to host6/204.53.107.249:2181, initiating session
2015-01-04 16:18:26,819 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:28,463 INFO ClientCnxn:968 - Opening socket connection to
server host4/204.53.107.247:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:28,463 INFO ClientCnxn:849 - Socket connection
established to host4/204.53.107.247:2181, initiating session
2015-01-04 16:18:28,465 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:28,904 INFO ClientCnxn:968 - Opening socket connection to
server host5/204.53.107.248:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:28,904 INFO ClientCnxn:849 - Socket connection
established to host5/204.53.107.248:2181, initiating session
2015-01-04 16:18:28,905 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:29,022 INFO ClientCnxn:968 - Opening socket connection to
server host6/204.53.107.249:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:29,023 INFO ClientCnxn:849 - Socket connection
established to host6/204.53.107.249:2181, initiating session
2015-01-04 16:18:29,024 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:30,359 INFO ClientCnxn:968 - Opening socket connection to
server host4/204.53.107.247:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:30,359 INFO ClientCnxn:849 - Socket connection
established to host4/204.53.107.247:2181, initiating session
2015-01-04 16:18:30,361 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:30,627 INFO ClientCnxn:968 - Opening socket connection to
server host5/204.53.107.248:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:30,628 INFO ClientCnxn:849 - Socket connection
established to host5/204.53.107.248:2181, initiating session
2015-01-04 16:18:30,628 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:31,332 INFO ClientCnxn:968 - Opening socket connection to
server host6/204.53.107.249:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:31,333 INFO ClientCnxn:849 - Socket connection
established to host6/204.53.107.249:2181, initiating session
2015-01-04 16:18:31,334 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:32,790 INFO ClientCnxn:968 - Opening socket connection to
server host4/204.53.107.247:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:32,791 INFO ClientCnxn:849 - Socket connection
established to host4/204.53.107.247:2181, initiating session
2015-01-04 16:18:33,118 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:33,297 INFO ClientCnxn:968 - Opening socket connection to
server host5/204.53.107.248:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:33,298 INFO ClientCnxn:849 - Socket connection
established to host5/204.53.107.248:2181, initiating session
2015-01-04 16:18:33,299 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:34,254 INFO ClientCnxn:968 - Opening socket connection to
server host6/204.53.107.249:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:34,254 INFO ClientCnxn:849 - Socket connection
established to host6/204.53.107.249:2181, initiating session
2015-01-04 16:18:34,255 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:35,562 INFO ClientCnxn:968 - Opening socket connection to
server host4/204.53.107.247:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:35,563 INFO ClientCnxn:849 - Socket connection
established to host4/204.53.107.247:2181, initiating session
2015-01-04 16:18:35,565 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:36,272 INFO ClientCnxn:968 - Opening socket connection to
server host5/204.53.107.248:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:36,272 INFO ClientCnxn:849 - Socket connection
established to host5/204.53.107.248:2181, initiating session
2015-01-04 16:18:36,273 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:37,120 INFO ClientCnxn:968 - Opening socket connection to
server host6/204.53.107.249:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:37,121 INFO ClientCnxn:849 - Socket connection
established to host6/204.53.107.249:2181, initiating session
2015-01-04 16:18:37,122 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:38,305 INFO ClientCnxn:968 - Opening socket connection to
server host4/204.53.107.247:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:38,306 INFO ClientCnxn:849 - Socket connection
established to host4/204.53.107.247:2181, initiating session
2015-01-04 16:18:38,307 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:39,194 INFO ClientCnxn:968 - Opening socket connection to
server host5/204.53.107.248:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:39,194 INFO ClientCnxn:849 - Socket connection
established to host5/204.53.107.248:2181, initiating session
2015-01-04 16:18:39,195 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:39,365 INFO ClientCnxn:968 - Opening socket connection to
server host6/204.53.107.249:2181. Will not attempt to authenticate using
SASL (unknown error)
2015-01-04 16:18:39,366 INFO ClientCnxn:849 - Socket connection
established to host6/204.53.107.249:2181, initiating session
2015-01-04 16:18:39,367 INFO ClientCnxn:1091 - Unable to read additional
data from server sessionid 0x34a7896d0ab0000, likely server has closed
socket, closing socket connection and attempting reconnect
2015-01-04 16:18:39,470 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.BackgroundSyncImpl$1.processResult(BackgroundSyncImpl.java:50)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:606)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
*2015-01-04 16:18:39,505 INFO ConnectionStateManager:151 - State change:
LOST*
2015-01-04 16:18:39,507 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,507 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,514 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,515 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,515 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,516 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,516 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,517 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,517 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,517 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,517 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,518 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,518 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,518 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,518 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,519 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,519 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,519 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,519 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,530 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,530 ERROR CuratorFrameworkImpl:530 - Background
operation retry gave up
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at
org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
at
org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
2015-01-04 16:18:39,613 INFO ZooKeeper:684 - Session: 0x34a7896d0ab0000
closed
2015-01-04 16:18:39,613 INFO ClientCnxn:509 - EventThread shut down
Any idea of what might be going on?
Thanks
Benjamin
Re: Curator retry strategy fails
Posted by Benjamin Jaton <bj...@radiantlogic.com>.
Hmm not sure but it could be the same.
This one happened in production so I am not sure it was caused for the same
reason.
On Thu, Jan 8, 2015 at 9:44 PM, Cameron McKenzie <mc...@gmail.com>
wrote:
> Is this the same issue as
> https://issues.apache.org/jira/browse/CURATOR-134?
>
> On Fri, Jan 9, 2015 at 6:34 AM, Benjamin Jaton <bj...@radiantlogic.com>
> wrote:
>
>> Hello,
>>
>> I came across a scenario where the connection state changes from
>> SUSPENDED to LOST within 17 seconds.
>>
>> However the curator is configured with:
>> session timeout = 72000
>> connection timeout = 18000
>> Retry Strategy = RetryNTimes(3, 5000)
>>
>> 2015-01-04 16:18:22,387 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> *2015-01-04 16:18:22,531 INFO ConnectionStateManager:151 - State change:
>> SUSPENDED*
>> 2015-01-04 16:18:23,114 INFO ClientCnxn:968 - Opening socket connection
>> to server host5/204.53.107.248:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:23,115 INFO ClientCnxn:849 - Socket connection
>> established to host5/204.53.107.248:2181, initiating session
>> 2015-01-04 16:18:23,124 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:23,611 INFO ClientCnxn:968 - Opening socket connection
>> to server host6/204.53.107.249:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:23,611 INFO ClientCnxn:849 - Socket connection
>> established to host6/204.53.107.249:2181, initiating session
>> 2015-01-04 16:18:23,623 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:25,507 INFO ClientCnxn:968 - Opening socket connection
>> to server host4/204.53.107.247:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:25,507 INFO ClientCnxn:849 - Socket connection
>> established to host4/204.53.107.247:2181, initiating session
>> 2015-01-04 16:18:25,509 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:26,171 INFO ClientCnxn:968 - Opening socket connection
>> to server host5/204.53.107.248:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:26,171 INFO ClientCnxn:849 - Socket connection
>> established to host5/204.53.107.248:2181, initiating session
>> 2015-01-04 16:18:26,172 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:26,817 INFO ClientCnxn:968 - Opening socket connection
>> to server host6/204.53.107.249:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:26,818 INFO ClientCnxn:849 - Socket connection
>> established to host6/204.53.107.249:2181, initiating session
>> 2015-01-04 16:18:26,819 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:28,463 INFO ClientCnxn:968 - Opening socket connection
>> to server host4/204.53.107.247:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:28,463 INFO ClientCnxn:849 - Socket connection
>> established to host4/204.53.107.247:2181, initiating session
>> 2015-01-04 16:18:28,465 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:28,904 INFO ClientCnxn:968 - Opening socket connection
>> to server host5/204.53.107.248:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:28,904 INFO ClientCnxn:849 - Socket connection
>> established to host5/204.53.107.248:2181, initiating session
>> 2015-01-04 16:18:28,905 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:29,022 INFO ClientCnxn:968 - Opening socket connection
>> to server host6/204.53.107.249:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:29,023 INFO ClientCnxn:849 - Socket connection
>> established to host6/204.53.107.249:2181, initiating session
>> 2015-01-04 16:18:29,024 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:30,359 INFO ClientCnxn:968 - Opening socket connection
>> to server host4/204.53.107.247:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:30,359 INFO ClientCnxn:849 - Socket connection
>> established to host4/204.53.107.247:2181, initiating session
>> 2015-01-04 16:18:30,361 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:30,627 INFO ClientCnxn:968 - Opening socket connection
>> to server host5/204.53.107.248:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:30,628 INFO ClientCnxn:849 - Socket connection
>> established to host5/204.53.107.248:2181, initiating session
>> 2015-01-04 16:18:30,628 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:31,332 INFO ClientCnxn:968 - Opening socket connection
>> to server host6/204.53.107.249:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:31,333 INFO ClientCnxn:849 - Socket connection
>> established to host6/204.53.107.249:2181, initiating session
>> 2015-01-04 16:18:31,334 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:32,790 INFO ClientCnxn:968 - Opening socket connection
>> to server host4/204.53.107.247:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:32,791 INFO ClientCnxn:849 - Socket connection
>> established to host4/204.53.107.247:2181, initiating session
>> 2015-01-04 16:18:33,118 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:33,297 INFO ClientCnxn:968 - Opening socket connection
>> to server host5/204.53.107.248:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:33,298 INFO ClientCnxn:849 - Socket connection
>> established to host5/204.53.107.248:2181, initiating session
>> 2015-01-04 16:18:33,299 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:34,254 INFO ClientCnxn:968 - Opening socket connection
>> to server host6/204.53.107.249:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:34,254 INFO ClientCnxn:849 - Socket connection
>> established to host6/204.53.107.249:2181, initiating session
>> 2015-01-04 16:18:34,255 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:35,562 INFO ClientCnxn:968 - Opening socket connection
>> to server host4/204.53.107.247:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:35,563 INFO ClientCnxn:849 - Socket connection
>> established to host4/204.53.107.247:2181, initiating session
>> 2015-01-04 16:18:35,565 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:36,272 INFO ClientCnxn:968 - Opening socket connection
>> to server host5/204.53.107.248:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:36,272 INFO ClientCnxn:849 - Socket connection
>> established to host5/204.53.107.248:2181, initiating session
>> 2015-01-04 16:18:36,273 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:37,120 INFO ClientCnxn:968 - Opening socket connection
>> to server host6/204.53.107.249:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:37,121 INFO ClientCnxn:849 - Socket connection
>> established to host6/204.53.107.249:2181, initiating session
>> 2015-01-04 16:18:37,122 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:38,305 INFO ClientCnxn:968 - Opening socket connection
>> to server host4/204.53.107.247:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:38,306 INFO ClientCnxn:849 - Socket connection
>> established to host4/204.53.107.247:2181, initiating session
>> 2015-01-04 16:18:38,307 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:39,194 INFO ClientCnxn:968 - Opening socket connection
>> to server host5/204.53.107.248:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:39,194 INFO ClientCnxn:849 - Socket connection
>> established to host5/204.53.107.248:2181, initiating session
>> 2015-01-04 16:18:39,195 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:39,365 INFO ClientCnxn:968 - Opening socket connection
>> to server host6/204.53.107.249:2181. Will not attempt to authenticate
>> using SASL (unknown error)
>> 2015-01-04 16:18:39,366 INFO ClientCnxn:849 - Socket connection
>> established to host6/204.53.107.249:2181, initiating session
>> 2015-01-04 16:18:39,367 INFO ClientCnxn:1091 - Unable to read additional
>> data from server sessionid 0x34a7896d0ab0000, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 2015-01-04 16:18:39,470 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.BackgroundSyncImpl$1.processResult(BackgroundSyncImpl.java:50)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:606)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> *2015-01-04 16:18:39,505 INFO ConnectionStateManager:151 - State change:
>> LOST*
>> 2015-01-04 16:18:39,507 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,507 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,514 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,515 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,515 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,516 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,516 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,517 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,517 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,517 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,517 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,518 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,518 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,518 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,518 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,519 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,519 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,519 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,519 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,530 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,530 ERROR CuratorFrameworkImpl:530 - Background
>> operation retry gave up
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss
>> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>> at
>> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
>> at
>> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
>> at
>> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
>> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
>> 2015-01-04 16:18:39,613 INFO ZooKeeper:684 - Session: 0x34a7896d0ab0000
>> closed
>> 2015-01-04 16:18:39,613 INFO ClientCnxn:509 - EventThread shut down
>>
>> Any idea of what might be going on?
>>
>> Thanks
>> Benjamin
>>
>>
>
Re: Curator retry strategy fails
Posted by Cameron McKenzie <mc...@gmail.com>.
Is this the same issue as https://issues.apache.org/jira/browse/CURATOR-134?
On Fri, Jan 9, 2015 at 6:34 AM, Benjamin Jaton <bj...@radiantlogic.com>
wrote:
> Hello,
>
> I came across a scenario where the connection state changes from SUSPENDED
> to LOST within 17 seconds.
>
> However the curator is configured with:
> session timeout = 72000
> connection timeout = 18000
> Retry Strategy = RetryNTimes(3, 5000)
>
> 2015-01-04 16:18:22,387 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> *2015-01-04 16:18:22,531 INFO ConnectionStateManager:151 - State change:
> SUSPENDED*
> 2015-01-04 16:18:23,114 INFO ClientCnxn:968 - Opening socket connection
> to server host5/204.53.107.248:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:23,115 INFO ClientCnxn:849 - Socket connection
> established to host5/204.53.107.248:2181, initiating session
> 2015-01-04 16:18:23,124 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:23,611 INFO ClientCnxn:968 - Opening socket connection
> to server host6/204.53.107.249:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:23,611 INFO ClientCnxn:849 - Socket connection
> established to host6/204.53.107.249:2181, initiating session
> 2015-01-04 16:18:23,623 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:25,507 INFO ClientCnxn:968 - Opening socket connection
> to server host4/204.53.107.247:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:25,507 INFO ClientCnxn:849 - Socket connection
> established to host4/204.53.107.247:2181, initiating session
> 2015-01-04 16:18:25,509 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:26,171 INFO ClientCnxn:968 - Opening socket connection
> to server host5/204.53.107.248:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:26,171 INFO ClientCnxn:849 - Socket connection
> established to host5/204.53.107.248:2181, initiating session
> 2015-01-04 16:18:26,172 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:26,817 INFO ClientCnxn:968 - Opening socket connection
> to server host6/204.53.107.249:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:26,818 INFO ClientCnxn:849 - Socket connection
> established to host6/204.53.107.249:2181, initiating session
> 2015-01-04 16:18:26,819 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:28,463 INFO ClientCnxn:968 - Opening socket connection
> to server host4/204.53.107.247:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:28,463 INFO ClientCnxn:849 - Socket connection
> established to host4/204.53.107.247:2181, initiating session
> 2015-01-04 16:18:28,465 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:28,904 INFO ClientCnxn:968 - Opening socket connection
> to server host5/204.53.107.248:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:28,904 INFO ClientCnxn:849 - Socket connection
> established to host5/204.53.107.248:2181, initiating session
> 2015-01-04 16:18:28,905 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:29,022 INFO ClientCnxn:968 - Opening socket connection
> to server host6/204.53.107.249:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:29,023 INFO ClientCnxn:849 - Socket connection
> established to host6/204.53.107.249:2181, initiating session
> 2015-01-04 16:18:29,024 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:30,359 INFO ClientCnxn:968 - Opening socket connection
> to server host4/204.53.107.247:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:30,359 INFO ClientCnxn:849 - Socket connection
> established to host4/204.53.107.247:2181, initiating session
> 2015-01-04 16:18:30,361 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:30,627 INFO ClientCnxn:968 - Opening socket connection
> to server host5/204.53.107.248:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:30,628 INFO ClientCnxn:849 - Socket connection
> established to host5/204.53.107.248:2181, initiating session
> 2015-01-04 16:18:30,628 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:31,332 INFO ClientCnxn:968 - Opening socket connection
> to server host6/204.53.107.249:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:31,333 INFO ClientCnxn:849 - Socket connection
> established to host6/204.53.107.249:2181, initiating session
> 2015-01-04 16:18:31,334 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:32,790 INFO ClientCnxn:968 - Opening socket connection
> to server host4/204.53.107.247:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:32,791 INFO ClientCnxn:849 - Socket connection
> established to host4/204.53.107.247:2181, initiating session
> 2015-01-04 16:18:33,118 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:33,297 INFO ClientCnxn:968 - Opening socket connection
> to server host5/204.53.107.248:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:33,298 INFO ClientCnxn:849 - Socket connection
> established to host5/204.53.107.248:2181, initiating session
> 2015-01-04 16:18:33,299 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:34,254 INFO ClientCnxn:968 - Opening socket connection
> to server host6/204.53.107.249:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:34,254 INFO ClientCnxn:849 - Socket connection
> established to host6/204.53.107.249:2181, initiating session
> 2015-01-04 16:18:34,255 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:35,562 INFO ClientCnxn:968 - Opening socket connection
> to server host4/204.53.107.247:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:35,563 INFO ClientCnxn:849 - Socket connection
> established to host4/204.53.107.247:2181, initiating session
> 2015-01-04 16:18:35,565 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:36,272 INFO ClientCnxn:968 - Opening socket connection
> to server host5/204.53.107.248:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:36,272 INFO ClientCnxn:849 - Socket connection
> established to host5/204.53.107.248:2181, initiating session
> 2015-01-04 16:18:36,273 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:37,120 INFO ClientCnxn:968 - Opening socket connection
> to server host6/204.53.107.249:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:37,121 INFO ClientCnxn:849 - Socket connection
> established to host6/204.53.107.249:2181, initiating session
> 2015-01-04 16:18:37,122 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:38,305 INFO ClientCnxn:968 - Opening socket connection
> to server host4/204.53.107.247:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:38,306 INFO ClientCnxn:849 - Socket connection
> established to host4/204.53.107.247:2181, initiating session
> 2015-01-04 16:18:38,307 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:39,194 INFO ClientCnxn:968 - Opening socket connection
> to server host5/204.53.107.248:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:39,194 INFO ClientCnxn:849 - Socket connection
> established to host5/204.53.107.248:2181, initiating session
> 2015-01-04 16:18:39,195 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:39,365 INFO ClientCnxn:968 - Opening socket connection
> to server host6/204.53.107.249:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2015-01-04 16:18:39,366 INFO ClientCnxn:849 - Socket connection
> established to host6/204.53.107.249:2181, initiating session
> 2015-01-04 16:18:39,367 INFO ClientCnxn:1091 - Unable to read additional
> data from server sessionid 0x34a7896d0ab0000, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2015-01-04 16:18:39,470 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.BackgroundSyncImpl$1.processResult(BackgroundSyncImpl.java:50)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:606)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> *2015-01-04 16:18:39,505 INFO ConnectionStateManager:151 - State change:
> LOST*
> 2015-01-04 16:18:39,507 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,507 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,514 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,515 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,515 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,516 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.ExistsBuilderImpl$1.processResult(ExistsBuilderImpl.java:123)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:552)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,516 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,517 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,517 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,517 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,517 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,518 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,518 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,518 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,518 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,519 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,519 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,519 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,519 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,530 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,530 ERROR CuratorFrameworkImpl:530 - Background
> operation retry gave up
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:487)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:590)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
> 2015-01-04 16:18:39,613 INFO ZooKeeper:684 - Session: 0x34a7896d0ab0000
> closed
> 2015-01-04 16:18:39,613 INFO ClientCnxn:509 - EventThread shut down
>
> Any idea of what might be going on?
>
> Thanks
> Benjamin
>
>