You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2014/02/27 05:14:03 UTC
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4631 - Still
Failing
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4631/
1 tests failed.
REGRESSION: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
Stack Trace:
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
at __randomizedtesting.SeedInfo.seed([743AB42D59411CE1:70323BDE4BE4F3C0]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:429)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:426)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:383)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:370)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:156)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:289)
at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:154)
at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:590)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
Build Log:
[...truncated 10692 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> 2036170 T4289 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
[junit4] 2> 2036171 T4289 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./solrtest-OverseerTest-1393472785045
[junit4] 2> 2036172 T4289 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 2036176 T4289 oas.SolrTestCaseJ4.setUp ###Starting testPlaceholders
[junit4] 2> 2036177 T4289 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 2036177 T4290 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 2036278 T4289 oasc.ZkTestServer.run start zk server on port:56988
[junit4] 2> 2036279 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2036283 T4296 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6968789f name:ZooKeeperConnection Watcher:127.0.0.1:56988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2036283 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2036284 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2036286 T4298 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@67b0f55c name:ZooKeeperConnection Watcher:127.0.0.1:56988 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2036286 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2036298 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2036299 T4300 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@47ea9efe name:ZooKeeperConnection Watcher:127.0.0.1:56988 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2036300 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2036300 T4289 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 2036310 T4289 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 2036313 T4289 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 2036315 T4289 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 2036323 T4289 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2036326 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2036328 T4302 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8a2eee0 name:ZooKeeperConnection Watcher:127.0.0.1:56988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2036329 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2036330 T4289 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2036332 T4289 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 2036335 T4296 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2036335 T4302 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2036335 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2036337 T4304 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48fcf186 name:ZooKeeperConnection Watcher:127.0.0.1:56988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2036337 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2036339 T4289 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 2036341 T4289 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 2036345 T4289 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56988_solr
[junit4] 2> 2036346 T4289 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 2036348 T4289 oasc.Overseer.start Overseer (id=91322632441561092-127.0.0.1:56988_solr-n_0000000000) starting
[junit4] 2> 2036350 T4289 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 2036353 T4289 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 2036357 T4289 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 2036359 T4306 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 2036360 T4306 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 2036360 T4305 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 2036363 T4305 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2036363 T4305 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=12 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"12",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2036364 T4305 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3, shard4, shard5, shard6, shard7, shard8, shard9, shard10, shard11, shard12]
[junit4] 2> 2036364 T4305 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard4
[junit4] 2> 2036367 T4304 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2036368 T4302 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2036368 T4296 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2036863 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard4/election
[junit4] 2> 2036885 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard4
[junit4] 2> 2036891 T4304 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2036891 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2036892 T4305 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2036895 T4296 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 2036895 T4302 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 2036897 T4289 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:56988 56988
[junit4] 2> 2037059 T4289 oas.SolrTestCaseJ4.tearDown ###Ending testPlaceholders
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2037066 T4289 oas.SolrTestCaseJ4.setUp ###Starting testShardLeaderChange
[junit4] 2> 2037067 T4289 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 2037068 T4307 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 2037168 T4289 oasc.ZkTestServer.run start zk server on port:56678
[junit4] 2> 2037169 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2037172 T4313 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55ffeb0b name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2037173 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2037174 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2037176 T4315 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@726f3d04 name:ZooKeeperConnection Watcher:127.0.0.1:56678 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2037176 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2037204 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2037206 T4317 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10b15e0 name:ZooKeeperConnection Watcher:127.0.0.1:56678 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2037206 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2037206 T4289 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 2037209 T4289 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 2037214 T4318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2037214 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2037215 T4321 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1045049d name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2037216 T4318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2037216 T4322 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2331e7c3 name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2037216 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2037217 T4318 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 2037217 T4289 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 2037219 T4318 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 2037220 T4289 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 2037222 T4289 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2037223 T4318 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56678_solr
[junit4] 2> 2037224 T4318 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 2037225 T4289 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 2037226 T4318 oasc.Overseer.start Overseer (id=91322632499822595-127.0.0.1:56678_solr-n_0000000000) starting
[junit4] 2> 2037227 T4322 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2037229 T4318 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 2037229 T4289 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 2037232 T4318 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 2037236 T4318 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 2037238 T4324 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 2037238 T4318 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
[junit4] 2> 2037239 T4324 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 2037239 T4323 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 2037240 T4323 oasc.Overseer$ClusterStateUpdater.amILeader org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1151)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:271)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:271)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.amILeader(Overseer.java:450)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:158)
[junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4] 2>
[junit4] 2> 2037240 T4318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2037240 T4323 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=91322632499822595-127.0.0.1:56678_solr-n_0000000000) am no longer a leader.
[junit4] 2> 2037242 T4326 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6437137a name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2037243 T4318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2037243 T4324 oasc.OverseerCollectionProcessor.run ERROR Unable to prioritize overseer java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:119)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:87)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:220)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor.prioritizeOverseerNodes(OverseerCollectionProcessor.java:231)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:182)
[junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4] 2>
[junit4] 2> 2037244 T4324 oasc.OverseerCollectionProcessor.amILeader org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1151)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:271)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:271)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor.amILeader(OverseerCollectionProcessor.java:359)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:189)
[junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4] 2>
[junit4] 2> 2037245 T4324 oasc.OverseerCollectionProcessor.amILeader According to ZK I (id=91322632499822595-127.0.0.1:56678_solr-n_0000000000) am no longer a leader.
[junit4] 2> 2037246 T4318 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56678_solr
[junit4] 2> 2037247 T4318 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 2037249 T4318 oasc.Overseer.start Overseer (id=91322632499822597-127.0.0.1:56678_solr-n_0000000001) starting
[junit4] 2> 2037254 T4328 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 2037254 T4328 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 2037254 T4327 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 2037256 T4327 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2037257 T4327 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2037257 T4327 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 2037257 T4327 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 2037260 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2037261 T4322 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2037734 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2037755 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 2037761 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2037763 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2037763 T4327 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2037763 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2037767 T4327 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2037767 T4327 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2037834 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2037835 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2037839 T4330 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ad27135 name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2037839 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2037842 T4289 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2037846 T4289 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 2037894 T4322 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2037895 T4330 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2037927 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2037927 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2037928 T4327 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2037928 T4327 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2037952 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2037958 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2038054 T4322 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2038054 T4330 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2038160 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2038164 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038164 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038164 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038165 T4327 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2038166 T4327 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 2038166 T4327 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4] 2> 2038166 T4327 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 2038182 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038295 T4330 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2038296 T4322 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2038665 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2038686 T4330 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2038686 T4322 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2038687 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2038688 T4330 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 2038700 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038700 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038700 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038702 T4327 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2038705 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038807 T4330 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2038889 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2038890 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2038892 T4332 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@60561276 name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2038893 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2038894 T4289 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2038897 T4289 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 2038899 T4330 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2038899 T4332 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2038901 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038901 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2038901 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038902 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038903 T4327 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2038905 T4327 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2038905 T4327 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2038917 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038918 T4330 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2038917 T4332 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2038927 T4332 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 2038927 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038927 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2038928 T4327 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2038928 T4327 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2038931 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038935 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038935 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2038937 T4334 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c51c528 name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2038937 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2038938 T4289 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2038940 T4289 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 2038943 T4332 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2038943 T4334 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2038945 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038945 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2038946 T4327 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2038947 T4327 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2038949 T4326 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2038951 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2038973 T4318 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
[junit4] 2> 2038974 T4318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2038976 T4336 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@37b761c4 name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2038976 T4318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2038980 T4318 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56678_solr
[junit4] 2> 2038980 T4318 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 2038982 T4318 oasc.Overseer.start Overseer (id=91322632499822601-127.0.0.1:56678_solr-n_0000000002) starting
[junit4] 2> 2038987 T4338 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 2038988 T4338 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 2038989 T4337 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2038990 T4337 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
[junit4] 2> 2038990 T4337 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2038990 T4337 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2038992 T4332 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2038992 T4334 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2038994 T4337 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2038994 T4337 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2038995 T4334 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2038995 T4332 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2038999 T4332 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2038999 T4334 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2039002 T4337 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2039002 T4337 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2039003 T4334 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2039003 T4332 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2039005 T4337 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 2039154 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2039157 T4336 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039158 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2039159 T4337 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2039160 T4337 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 2039160 T4337 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2039174 T4336 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039175 T4332 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2039175 T4334 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2039177 T4334 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2039177 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2039178 T4334 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 2039183 T4336 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039184 T4337 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2039187 T4336 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039289 T4334 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2039379 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2039381 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2039383 T4340 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5edd5be4 name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2039383 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2039385 T4289 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2039387 T4289 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 2039392 T4318 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
[junit4] 2> 2039409 T4334 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2039409 T4340 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2039409 T4318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2039437 T4342 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e6e662e name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2039437 T4318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2039439 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2039452 T4318 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56678_solr
[junit4] 2> 2039452 T4318 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 2039455 T4318 oasc.Overseer.start Overseer (id=91322632499822603-127.0.0.1:56678_solr-n_0000000003) starting
[junit4] 2> 2039468 T4340 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2039469 T4334 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2039471 T4340 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 2039472 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2039473 T4344 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 2039473 T4344 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 2039473 T4343 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 2039492 T4343 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2039493 T4342 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039493 T4343 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2039493 T4343 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2039495 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2039497 T4346 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c11c439 name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2039497 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2039498 T4340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2039499 T4289 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2039501 T4343 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2039501 T4289 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 2039502 T4343 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2039502 T4343 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2039504 T4340 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2039504 T4346 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2039505 T4342 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039507 T4342 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039507 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2039528 T4343 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2039528 T4343 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2039531 T4342 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039532 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2039638 T4340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2039638 T4346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2039734 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2039739 T4342 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039739 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2039739 T4342 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039740 T4342 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039740 T4343 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2039741 T4343 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 2039741 T4343 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2039752 T4342 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039763 T4346 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2039763 T4340 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2039764 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2039765 T4346 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 2039769 T4342 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039773 T4342 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039875 T4346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2039966 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2039968 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2039970 T4348 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@641e3b4e name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2039970 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2039972 T4289 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2039974 T4289 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 2039976 T4346 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2039977 T4348 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2039979 T4318 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
[junit4] 2> 2039987 T4342 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039987 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2039987 T4342 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039988 T4342 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2039988 T4343 oasc.Overseer$ClusterStateUpdater.run WARN Solr cannot talk to ZK, exiting Overseer main queue loop org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.orderedChildren(DistributedQueue.java:89)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:411)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:391)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:168)
[junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4] 2>
[junit4] 2> 2039988 T4318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2039990 T4350 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@321c4b77 name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2039990 T4318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2039994 T4318 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56678_solr
[junit4] 2> 2039994 T4318 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 2039995 T4348 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2039995 T4346 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2039997 T4318 oasc.Overseer.start Overseer (id=91322632499822606-127.0.0.1:56678_solr-n_0000000004) starting
[junit4] 2> 2039997 T4348 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 2039999 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2040003 T4352 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 2040003 T4352 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 2040004 T4351 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 2040006 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2040006 T4351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2040008 T4354 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@54711fa4 name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2040008 T4351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2040008 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2040008 T4351 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2040010 T4289 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2040012 T4350 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040013 T4354 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2040013 T4348 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2040013 T4289 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 2040016 T4348 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2040016 T4354 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2040017 T4351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2040017 T4351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2040018 T4351 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2040018 T4350 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040019 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2040024 T4350 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040025 T4351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2040025 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2040025 T4351 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2040028 T4350 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040132 T4348 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2040132 T4354 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2040188 T4322 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 2040188 T4322 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 2040227 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2040231 T4350 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040231 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2040231 T4350 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040232 T4350 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040233 T4351 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2040251 T4351 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 2040251 T4351 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2040278 T4350 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040278 T4348 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2040278 T4354 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2040279 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2040296 T4354 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 2040333 T4350 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040355 T4350 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040459 T4354 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2040498 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2040500 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2040502 T4356 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f00a295 name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2040502 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2040504 T4289 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2040506 T4289 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 2040509 T4354 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2040509 T4356 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2040510 T4318 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
[junit4] 2> 2040518 T4318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2040518 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2040537 T4358 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3ee06063 name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2040537 T4318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2040564 T4318 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56678_solr
[junit4] 2> 2040564 T4318 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 2040576 T4318 oasc.Overseer.start Overseer (id=91322632499822609-127.0.0.1:56678_solr-n_0000000005) starting
[junit4] 2> 2040577 T4356 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2040577 T4354 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2040604 T4356 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 2040605 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2040630 T4360 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 2040631 T4360 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 2040647 T4359 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 2040661 T4359 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2040661 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2040661 T4359 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2040662 T4359 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2040663 T4362 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@52040391 name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2040663 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2040665 T4289 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2040678 T4332 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 2040678 T4332 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 2040678 T4358 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040679 T4332 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 2040679 T4362 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 2040679 T4356 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2040681 T4289 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 2040693 T4359 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2040693 T4356 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2040693 T4362 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2040694 T4359 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2040694 T4359 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2040696 T4358 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040696 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2040700 T4358 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040701 T4359 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2040701 T4359 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2040703 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2040709 T4358 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040811 T4356 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2040811 T4362 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2040905 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2040916 T4358 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040916 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2040917 T4358 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040917 T4358 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040918 T4359 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2040918 T4359 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 2040919 T4359 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2040935 T4358 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040936 T4356 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2040937 T4362 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2040953 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2040954 T4362 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 2040972 T4358 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2040989 T4358 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2041093 T4362 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2041156 T4289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2041158 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2041160 T4364 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@29b29fc5 name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2041160 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2041162 T4289 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2041164 T4289 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 2041166 T4362 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2041166 T4364 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2041182 T4358 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2041182 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2041182 T4358 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2041183 T4358 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2041184 T4359 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2041184 T4359 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2041184 T4359 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2041201 T4358 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2041219 T4364 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2041219 T4362 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 2041231 T4364 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 2041232 T4358 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2041232 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2041233 T4359 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 2041233 T4359 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 2041236 T4358 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2041237 T4318 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
[junit4] 2> 2041238 T4318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2041239 T4289 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2041240 T4366 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@181252a name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2041241 T4318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2041241 T4368 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ddb2191 name:ZooKeeperConnection Watcher:127.0.0.1:56678/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2041242 T4289 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2041244 T4289 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2041245 T4318 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56678_solr
[junit4] 2> 2041245 T4318 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 2041246 T4289 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 2041247 T4318 oasc.Overseer.start Overseer (id=91322632499822612-127.0.0.1:56678_solr-n_0000000006) starting
[junit4] 2> 2041248 T4364 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2041248 T4368 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2041260 T4289 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2041262 T4370 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 2041263 T4370 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 2041264 T4340 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 2041264 T4340 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apach
[...truncated too long message...]
eption.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:206)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:203)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:203)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:414)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:383)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:370)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 2062319 T4669 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 2062324 T4675 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 2062326 T4677 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard4/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 2062326 T4679 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 2062326 T4679 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard4/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 2064686 T4289 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: test params are: codec=Lucene46: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=be, timezone=America/Miquelon
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=16,threads=1,free=250613896,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [TestZkChroot, TestSolrXml, TestNoOpRegenerator, CachingDirectoryFactoryTest, TestDefaultSimilarityFactory, TestUtils, EnumFieldTest, CurrencyFieldXmlFileTest, TestComponentsName, CoreContainerCoreInitFailuresTest, FullSolrCloudDistribCmdsTest, DefaultValueUpdateProcessorTest, HdfsChaosMonkeySafeLeaderTest, OutputWriterTest, DocumentAnalysisRequestHandlerTest, DistanceFunctionTest, TestSimpleQParserPlugin, HdfsBasicDistributedZkTest, TestFastOutputStream, TestDistributedSearch, TestCollationFieldDocValues, OpenCloseCoreStressTest, DebugComponentTest, TestStressRecovery, TestSerializedLuceneMatchVersion, TestAnalyzedSuggestions, DistributedQueryComponentOptimizationTest, TestSolr4Spatial, TestRecovery, NumericFieldsTest, TestRandomFaceting, FileUtilsTest, ShardRoutingCustomTest, PreAnalyzedUpdateProcessorTest, TestCoreDiscovery, TermVectorComponentDistributedTest, CollectionsAPIDistributedZkTest, TestCSVLoader, BlockDirectoryTest, TestNRTOpen, CurrencyFieldOpenExchangeTest, StatsComponentTest, TestFieldTypeCollectionResource, SolrCoreCheckLockOnStartupTest, TestDynamicFieldResource, TestSweetSpotSimilarityFactory, TestManagedSchemaFieldResource, DistributedQueryComponentCustomSortTest, SolrRequestParserTest, TestPartialUpdateDeduplication, TestIndexingPerformance, TestBinaryField, HdfsLockFactoryTest, HardAutoCommitTest, RequiredFieldsTest, TestIBSimilarityFactory, DeleteInactiveReplicaTest, BasicZkTest, SolrIndexConfigTest, AlternateDirectoryTest, TestClassNameShortening, HdfsBasicDistributedZk2Test, TestQuerySenderListener, TestMultiCoreConfBootstrap, TestXIncludeConfig, HdfsCollectionsAPIDistributedZkTest, FieldAnalysisRequestHandlerTest, TestSolrXmlPersistence, AddSchemaFieldsUpdateProcessorFactoryTest, PluginInfoTest, SolrCmdDistributorTest, DocValuesMultiTest, TestSolrCoreProperties, TestSolrQueryParserDefaultOperatorResource, TestDynamicFieldCollectionResource, HighlighterTest, TestLazyCores, TestOmitPositions, TestQuerySenderNoQuery, SuggestComponentTest, TestStressReorder, TestFieldResource, TestFieldTypeResource, TestUpdate, ShardSplitTest, DisMaxRequestHandlerTest, TestCodecSupport, NoCacheHeaderTest, DirectSolrSpellCheckerTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, OverseerTest]
[junit4] Completed on J1 in 28.70s, 8 tests, 1 error <<< FAILURES!
[...truncated 608 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:465: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:445: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:490: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1274: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:907: There were test failures: 373 suites, 1607 tests, 1 error, 47 ignored (19 assumptions)
Total time: 91 minutes 26 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure