Where is ZK session ID stored?

[Rahil Bohra <ra...@gmail.com>]

Hi everyone,

I'm running a storm cluster in docker containers and a zookeeper also in a
docker container.

I had to make some changes to the zookeeper image, so I recreated (from
scratch) the zookeeper container (and all of my storm containers).

The problem is storm somehow still has a later ZK session ID so zookeeper
always refuses session establishment:

2016-04-25 14:56:58,310 [myid:] - INFO  [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection
from /10.64.77.29:39150
2016-04-25 14:56:58,310 [myid:] - INFO  [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:ZooKeeperServer@873] - Refusing session request for
client /10.64.77.29:39150 as it has seen zxid 0x64c24 our last zxid is
0x2eb client must try another server
2016-04-25 14:56:58,310 [myid:] - INFO  [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1008] - Closed socket connection for
client /10.64.77.29:39150 (no session established for client)
(log output repeated forever)

10.64.77.29 is the address of the nimbus node.

I've recreated my storm containers from scratch, so expect them to have
maintained no state, but somehow they always request a zk session with the
same latest id.

So I guess my question is, where does storm store the latest ZK session ID
that it has seen? (or if it doesn't store this information, how does it
decide what zxid to request a session with?)


Cheers,
Rahil