You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/01/03 13:41:42 UTC

[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.8.0-ea-b121) - Build # 8841 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8841/
Java: 64bit/jdk1.8.0-ea-b121 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.LeaderElectionTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.LeaderElectionTest:     1) Thread[id=1621, name=Timer-102, state=WAITING, group=TGRP-LeaderElectionTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at java.util.TimerThread.mainLoop(Timer.java:526)         at java.util.TimerThread.run(Timer.java:505)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.LeaderElectionTest: 
   1) Thread[id=1621, name=Timer-102, state=WAITING, group=TGRP-LeaderElectionTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at java.util.TimerThread.mainLoop(Timer.java:526)
        at java.util.TimerThread.run(Timer.java:505)
	at __randomizedtesting.SeedInfo.seed([BC00FE661FD95A17]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.LeaderElectionTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=1621, name=Timer-102, state=WAITING, group=TGRP-LeaderElectionTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at java.util.TimerThread.mainLoop(Timer.java:526)         at java.util.TimerThread.run(Timer.java:505)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=1621, name=Timer-102, state=WAITING, group=TGRP-LeaderElectionTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at java.util.TimerThread.mainLoop(Timer.java:526)
        at java.util.TimerThread.run(Timer.java:505)
	at __randomizedtesting.SeedInfo.seed([BC00FE661FD95A17]:0)




Build Log:
[...truncated 10330 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderElectionTest
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-LeaderElectionTest-1388752421184
   [junit4]   2> 307900 T1423 oas.SolrTestCaseJ4.setUp ###Starting testElection
   [junit4]   2> 307900 T1423 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 307915 T1424 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 308011 T1423 oasc.ZkTestServer.run start zk server on port:56211
   [junit4]   2> 308013 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308019 T1431 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b61c4d3 name:ZooKeeperConnection Watcher:127.0.0.1:56211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308019 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308030 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308031 T1434 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@49642eb9 name:ZooKeeperConnection Watcher:127.0.0.1:56211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308032 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308032 T1423 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 308036 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308038 T1437 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@347cdf54 name:ZooKeeperConnection Watcher:127.0.0.1:56211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308038 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308040 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308042 T1441 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@331c3f3c name:ZooKeeperConnection Watcher:127.0.0.1:56211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308042 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308043 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308046 T1445 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b65de00 name:ZooKeeperConnection Watcher:127.0.0.1:56211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308046 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308051 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308052 T1449 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9313c47 name:ZooKeeperConnection Watcher:127.0.0.1:56211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308052 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308053 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308054 T1453 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e5bfe0b name:ZooKeeperConnection Watcher:127.0.0.1:56211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308055 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308056 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308060 T1457 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7cecd123 name:ZooKeeperConnection Watcher:127.0.0.1:56211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308060 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308061 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308064 T1461 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@98c8560 name:ZooKeeperConnection Watcher:127.0.0.1:56211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308065 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308065 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308067 T1465 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9df5312 name:ZooKeeperConnection Watcher:127.0.0.1:56211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308067 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308068 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308069 T1469 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78054f84 name:ZooKeeperConnection Watcher:127.0.0.1:56211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308069 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308070 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308071 T1473 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62fafc6e name:ZooKeeperConnection Watcher:127.0.0.1:56211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308072 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308073 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308074 T1477 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f30d2d4 name:ZooKeeperConnection Watcher:127.0.0.1:56211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308074 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308075 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308081 T1481 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c0457e3 name:ZooKeeperConnection Watcher:127.0.0.1:56211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308081 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308082 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308083 T1485 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2eaf4e88 name:ZooKeeperConnection Watcher:127.0.0.1:56211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308084 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308085 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308086 T1489 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b03cebf name:ZooKeeperConnection Watcher:127.0.0.1:56211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308087 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308092 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308093 T1493 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@122c1979 name:ZooKeeperConnection Watcher:127.0.0.1:56211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308093 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308094 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 308095 T1497 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4cc9ff90 name:ZooKeeperConnection Watcher:127.0.0.1:56211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 308095 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 308097 T1438 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 308097 T1442 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 308097 T1446 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 308100 T1446 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 308100 T1442 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 308101 T1438 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 308102 T1450 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 308111 T1438 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 308118 T1438 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 308238 T1425 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x143581841660003, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 308241 T1445 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 308743 T1425 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x143581841660004, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 308744 T1425 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x143581841660007, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 308745 T1449 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 309250 T1425 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14358184166000a, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 309252 T1477 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 309756 T1489 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 309758 T1493 oasc.LeaderElector.checkIfIamLeader WARN Failed setting watch org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election/91013278677205007-12-n_0000000011
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1151)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:273)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:270)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:270)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:126)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 309758 T1493 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:93)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 309762 T1423 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:56211 56211
   [junit4]   2> 309891 T1423 oas.SolrTestCaseJ4.tearDown ###Ending testElection
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 309895 T1423 oas.SolrTestCaseJ4.setUp ###Starting testBasic
   [junit4]   2> 309895 T1423 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 309895 T1498 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 309995 T1423 oasc.ZkTestServer.run start zk server on port:33120
   [junit4]   2> 309996 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 310000 T1505 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@42d92f57 name:ZooKeeperConnection Watcher:127.0.0.1:33120 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 310000 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 310012 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 310013 T1508 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2fa3c6b7 name:ZooKeeperConnection Watcher:127.0.0.1:33120 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 310013 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 310014 T1423 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 310018 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 310020 T1511 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@691865e7 name:ZooKeeperConnection Watcher:127.0.0.1:33120/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 310020 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 310020 T1423 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 310022 T1423 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 310036 T1423 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 310043 T1423 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 310048 T1423 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:33120 33120
   [junit4]   2> 310142 T1423 oas.SolrTestCaseJ4.tearDown ###Ending testBasic
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 310150 T1423 oas.SolrTestCaseJ4.setUp ###Starting testStressElection
   [junit4]   2> 310150 T1423 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 310151 T1512 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 310251 T1423 oasc.ZkTestServer.run start zk server on port:40792
   [junit4]   2> 310257 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 310260 T1519 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5613e43e name:ZooKeeperConnection Watcher:127.0.0.1:40792 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 310261 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 310276 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 310278 T1522 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c8e9446 name:ZooKeeperConnection Watcher:127.0.0.1:40792 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 310278 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 310278 T1423 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 310281 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 310282 T1525 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7155e4bc name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 310282 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 310283 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 310284 T1529 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@36a6460f name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 310284 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 310285 T1530 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 310287 T1530 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 310290 T1530 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 310293 T1530 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 310749 T1469 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 310750 T1469 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:205)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:202)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:202)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:413)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:382)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:369)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:165)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:109)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 310751 T1465 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 310751 T1465 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:205)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:202)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:202)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:413)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:382)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:369)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:165)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:109)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 310751 T1473 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 310752 T1473 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:205)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:202)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:202)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:413)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:382)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:369)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:165)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:109)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 311254 T1481 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 311255 T1481 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:205)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:202)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:202)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:413)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:382)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:369)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:165)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:109)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 311257 T1489 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:205)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:202)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:202)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:413)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:382)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:369)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:165)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:109)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 311258 T1497 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:93)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:138)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 312288 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312289 T1537 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5cbb953c name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312289 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312290 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312295 T1542 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d869f07 name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312296 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312321 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312322 T1547 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c51e30c name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312322 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312325 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312326 T1552 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7da96048 name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312326 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312328 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312329 T1557 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e3e08ec name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312329 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312331 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312332 T1562 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@79e77950 name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312332 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312334 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312335 T1567 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ce1fc62 name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312335 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312348 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312348 T1513 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14358184a100007, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 312353 T1573 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56325c48 name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312353 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312359 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312361 T1578 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70ed1294 name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312363 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312364 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312365 T1583 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c0f15ee name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312365 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312367 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312367 T1588 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4085e9f8 name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312368 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312368 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312372 T1593 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2299f89b name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312372 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312375 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312376 T1598 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@346f071b name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312376 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312377 T1538 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 312387 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312388 T1603 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@26d3b127 name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312389 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312390 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312392 T1608 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6673f8b0 name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312392 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312393 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312394 T1612 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@33ea7d4b name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312394 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312395 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312396 T1616 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@88ba898 name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312396 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312397 T1531 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 312398 T1620 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6123451d name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 312399 T1531 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 312429 T1543 oasc.LeaderElectionTest$ClientThread.run ERROR setup failed org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.<init>(ElectionContext.java:101)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElectionTest$ClientThread.setupOnConnect(LeaderElectionTest.java:113)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElectionTest$ClientThread.run(LeaderElectionTest.java:124)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:222)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:219)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:219)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:100)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.<init>(ElectionContext.java:99)
   [junit4]   2> 	... 9 more
   [junit4]   2> 
   [junit4]   2> 312451 T1552 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7da96048 name:ZooKeeperConnection Watcher:127.0.0.1:40792/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
   [junit4]   2> 312452 T1552 oascc.ConnectionManager.process zkClient has disconnected
   [junit4]   2> 312518 T1513 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14358184a10000d, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 312539 T1558 oasc.LeaderElectionTest$ClientThread.run ERROR setup failed org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.<init>(ElectionContext.java:101)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElectionTest$ClientThread.setupOnConnect(LeaderElectionTest.java:113)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElectionTest$ClientThread.run(LeaderElectionTest.java:124)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:222)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:219)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:219)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:100)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.<init>(ElectionContext.java:99)
   [junit4]   2> 	... 9 more
   [junit4]   2> 
   [junit4]   2> 312545 T1574 oasc.LeaderElectionTest$ClientThread.run ERROR setup failed org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.<init>(ElectionContext.java:101)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElectionTest$ClientThread.setupOnConnect(LeaderElectionTest.java:113)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElectionTest$ClientThread.run(LeaderElectionTest.java:124)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:222)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:219)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:219)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:100)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.<init>(ElectionContext.java:99)
   [junit4]   2> 	... 9 more
   [junit4]   2> 
   [junit4]   2> 312559 T1589 oasc.LeaderElectionTest$ClientThread.run ERROR setup failed org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.<init>(ElectionContext.java:101)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElectionTest$ClientThread.setupOnConnect(LeaderElectionTest.java:113)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElectionTest$ClientThread.run(LeaderElectionTest.java:124)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:222)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:219)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:219)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:100)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.<init>(ElectionContext.java:99)
   [junit4]   2> 	... 9 more
   [junit4]   2> 
   [junit4]   2> 312616 T1599 oasc.LeaderElectionTest$ClientThread.run ERROR setup failed org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.<init>(ElectionContext.java:101)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElectionTest$ClientThread.setupOnConnect(LeaderElectionTest.java:113)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElectionTest$ClientThread.run(LeaderElectionTest.java:124)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:222)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:219)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:219)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:100)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.<init>(ElectionContext.java:99)
   [junit4]   2> 	... 9 more
   [junit4]   2> 
   [junit4]   2> 312666 T1604 oasc.LeaderElectionTest$ClientThread.run ERROR setup failed org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.<init>(ElectionContext.java:101)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElectionTest$ClientThread.setupOnConnect(LeaderElectionTest.java:113)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElectionTest$ClientThread.run(LeaderElectionTest.java:124)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:222)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:219)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:219)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:100)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.<init>(ElectionContext.java:99)
   [junit4]   2> 	... 9 more
   [junit4]   2> 
   [junit4]   2> 312835 T1543 oasc.LeaderElectionTest$ClientThread.run ERROR setup failed org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.<init>(ElectionContext.java:101)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElectionTest$ClientThread.setupOnConnect(LeaderElectionTest.java:113)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElectionTest$ClientThread.run(LeaderElectionTest.java:124)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:222)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:219)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:219)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:100)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.<init>(ElectionContext.java:99)
   [junit4]   2> 	... 9 more
   [junit4]   2> 
   [junit4]   2> 312865 T1558 oasc.LeaderElectionTest$ClientThread.run ERROR setup failed org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.<init>(ElectionContext.java:101)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElectionTest$ClientThread.setupOnConnect(LeaderElectionTest.java:113)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElectionTest$ClientThread.run(LeaderElectionTest.java:124)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:222)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:219)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:219)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:100)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.ensureExists(ZkCmdExecutor.java:94)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.<init>(ElectionContext.java:99)
   [junit4]   2> 	... 9 more
   [junit4]   2> 
   [junit4]   2> 312870 T1567 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 312911 T1620 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 313999 T1513 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14358184a100014, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 316392 T1423 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40792 40792
   [junit4]   2> 316499 T1423 oas.SolrTestCaseJ4.tearDown ###Ending testStressElection
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 316502 T1423 oas.SolrTestCaseJ4.setUp ###Starting testCancelElection
   [junit4]   2> 316503 T1423 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 316503 T1629 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 316603 T1423 oasc.ZkTestServer.run start zk server on port:43267
   [junit4]   2> 316604 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 316607 T1636 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14ea9e1f name:ZooKeeperConnection Watcher:127.0.0.1:43267 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 316607 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 316621 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 316622 T1639 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@101e6a30 name:ZooKeeperConnection Watcher:127.0.0.1:43267 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 316622 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 316622 T1423 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 316624 T1423 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 316625 T1642 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70fd548f name:ZooKeeperConnection Watcher:127.0.0.1:43267/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 316625 T1423 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 316626 T1423 oascc.SolrZkClient.makePath makePath: /collections/collection2
   [junit4]   2> 316628 T1423 oascc.SolrZkClient.makePath makePath: /collections/collection2/leader_elect/slice1/election
   [junit4]   2> 316632 T1423 oascc.SolrZkClient.makePath makePath: /collections/collection2/leaders/slice1
   [junit4]   2> 316634 T1423 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 318644 T1642 oascc.SolrZkClient.makePath makePath: /collections/collection2/leaders/slice1
   [junit4]   2> 319646 T1423 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:43267 43267
   [junit4]   2> 319765 T1423 oas.SolrTestCaseJ4.tearDown ###Ending testCancelElection
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 319766 T1423 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 11873 T1422 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> 31946 T1422 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1 thread leaked from SUITE scope at org.apache.solr.cloud.LeaderElectionTest: 
   [junit4]   2> 	   1) Thread[id=1621, name=Timer-102, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:526)
   [junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
   [junit4]   2> 31947 T1422 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
   [junit4]   2> 	   1) Thread[id=1621, name=Timer-102, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]   2> 34948 T1422 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
   [junit4]   2> 	   1) Thread[id=1621, name=Timer-102, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.lang.Object.wait(Object.java:502)
   [junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:526)
   [junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
   [junit4]   2> NOTE: test params are: codec=Lucene46: {}, docValues:{}, sim=DefaultSimilarity, locale=hr, timezone=Asia/Aqtau
   [junit4]   2> NOTE: Linux 3.8.0-34-generic amd64/Oracle Corporation 1.8.0-ea (64-bit)/cpus=8,threads=2,free=79489792,total=204451840
   [junit4]   2> NOTE: All tests run in this JVM: [TestRandomDVFaceting, TestDocBasedVersionConstraints, SolrIndexSplitterTest, ZkControllerTest, SOLR749Test, DeleteShardTest, ClusterStateUpdateTest, DirectUpdateHandlerOptimizeTest, TestConfig, TestSolrQueryParserResource, HdfsSyncSliceTest, BasicFunctionalityTest, TestWriterPerf, DocValuesTest, BasicDistributedZkTest, SuggesterFSTTest, AnalysisAfterCoreReloadTest, PrimitiveFieldTypeTest, PeerSyncTest, TestManagedSchema, BlockCacheTest, AddSchemaFieldsUpdateProcessorFactoryTest, SynonymTokenizerTest, SliceStateUpdateTest, CacheHeaderTest, TestCollapseQParserPlugin, TestReversedWildcardFilterFactory, TestLuceneMatchVersion, TestCustomSort, BJQParserTest, JsonLoaderTest, WordBreakSolrSpellCheckerTest, TestSort, DistributedSuggestComponentTest, TestGroupingSearch, TestPostingsSolrHighlighter, TestLMDirichletSimilarityFactory, DocumentBuilderTest, TestRemoteStreaming, TestSimpleQParserPlugin, TestSolrDeletionPolicy1, SuggesterTest, SchemaVersionSpecificBehaviorTest, TestSolrXmlPersistor, UpdateRequestProcessorFactoryTest, TestFaceting, TestAnalyzeInfixSuggestions, TestBadConfig, TestInfoStreamLogging, DOMUtilTest, ScriptEngineTest, BadComponentTest, HdfsCollectionsAPIDistributedZkTest, FastVectorHighlighterTest, QueryEqualityTest, TestCopyFieldCollectionResource, TestElisionMultitermQuery, TestDocSet, LeaderElectionTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderElectionTest -Dtests.seed=BC00FE661FD95A17 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=hr -Dtests.timezone=Asia/Aqtau -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J0 | LeaderElectionTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.LeaderElectionTest: 
   [junit4]    >    1) Thread[id=1621, name=Timer-102, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at java.lang.Object.wait(Object.java:502)
   [junit4]    >         at java.util.TimerThread.mainLoop(Timer.java:526)
   [junit4]    >         at java.util.TimerThread.run(Timer.java:505)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([BC00FE661FD95A17]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=1621, name=Timer-102, state=WAITING, group=TGRP-LeaderElectionTest]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at java.lang.Object.wait(Object.java:502)
   [junit4]    >         at java.util.TimerThread.mainLoop(Timer.java:526)
   [junit4]    >         at java.util.TimerThread.run(Timer.java:505)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([BC00FE661FD95A17]:0)
   [junit4] Completed on J0 in 35.00s, 4 tests, 2 errors <<< FAILURES!

[...truncated 801 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:459: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:439: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1307: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:940: There were test failures: 348 suites, 1522 tests, 2 suite-level errors, 821 ignored (5 assumptions)

Total time: 42 minutes 9 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0-ea-b121 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure