Zookeeper client is trying to connect to the invalid IP address

["Bae, Jae Hyeon" <me...@gmail.com>]

I am using Apache curator as zookeeper client with the static connection
string. When the client gets connection error, it couldn't recover the
connection. The logs were shown as the following. The problem is,
ClientCnxn was trying to connect to the invalid IP address, I couldn't find
its IP address from our connection string. At the last line of log, its
connection string looked OK.

2013-11-15 20:18:34,758 INFO [main-SendThread(...:2181)]
(org.apache.zookeeper.ClientCnxn) {locationInfo=org.apache.
log4j.spi.LocationInfo@27dfcdcc, netflix.appinfo.countryId=1} Opening
socket connection to server .../...:2181 <http://10.159.16.223:2181/>. Will
not attempt to authenticate using SASL (unknown error)
2013-11-15 20:18:34,760 INFO [main-SendThread(...:2181)]
(org.apache.zookeeper.ClientCnxn)
{locationInfo=org.apache.log4j.spi.LocationInfo@56402087,
netflix.appinfo.countryId=1} Socket connection established to
.../...:2181<http://10.159.16.223:2181/>,
initiating session
2013-11-15 20:18:34,773 INFO [main-SendThread(...:2181)]
(org.apache.zookeeper.ClientCnxn)
{locationInfo=org.apache.log4j.spi.LocationInfo@c17ab4e,
netflix.appinfo.countryId=1} Unable to read additional data from server
sessionid 0xaf4258ef230203bb, likely server has closed socket, closing
socket connection and attempting reconnect

2013-11-15 20:18:59,607 ERROR [http-0.0.0.0-7101-20]
(org.apache.curator.ConnectionState) {ami=ami-9398bdfa,
clientIPAddrFamily=UNKNOWN, netflix.appinfo.countryId=1, stack=testreg,
region=us-east-1, locationInfo=org.apache.log4j.spi.LocationInfo@56cc2889}
Connection timed out for connection string (...) and timeout (30000) /
elapsed (92006)
org.apache.curator.CuratorConnectionLossException: KeeperErrorCode =
ConnectionLoss
        at
org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:191)
        at
org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:86)
        at
org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:113)
        at
org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:456)
        at
org.apache.curator.framework.imps.CreateBuilderImpl$12.call(CreateBuilderImpl.java:722)
        at
org.apache.curator.framework.imps.CreateBuilderImpl$12.call(CreateBuilderImpl.java:714)
        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
        at
org.apache.curator.framework.imps.CreateBuilderImpl.findProtectedNodeInForeground(CreateBuilderImpl.java:710)
        at
org.apache.curator.framework.imps.CreateBuilderImpl.access$1000(CreateBuilderImpl.java:44)
        at
org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:669)
        at
org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:660)
        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
        at
org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:656)
        at
org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:441)
        at
org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:431)
        at
org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:411)
        at
org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:44)
        at
org.apache.curator.framework.recipes.locks.LockInternals.attemptLock(LockInternals.java:224)
        at
org.apache.curator.framework.recipes.locks.InterProcessMutex.internalLock(InterProcessMutex.java:221)
        at
org.apache.curator.framework.recipes.locks.InterProcessMutex.acquire(InterProcessMutex.java:96)