You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/11/08 20:09:07 UTC
[JENKINS] Lucene-Solr-4.10-Linux (32bit/jdk1.7.0_67) - Build # 66 -
Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.10-Linux/66/
Java: 32bit/jdk1.7.0_67 -server -XX:+UseG1GC (asserts: false)
1 tests failed.
REGRESSION: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
Stack Trace:
org.apache.solr.common.SolrException: Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
at __randomizedtesting.SeedInfo.seed([A4A87ADA429D0299:A0A0F5295038EDB8]:0)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:150)
at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:157)
at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:662)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:40)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:137)
... 48 more
Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at org.apache.solr.common.cloud.SolrZkClient$11.execute(SolrZkClient.java:462)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:459)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:416)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:403)
at org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:142)
at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:34)
... 49 more
Build Log:
[...truncated 11813 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J0/./solr.cloud.OverseerTest-A4A87ADA429D0299-001/init-core-data-001
[junit4] 2> 1797207 T5038 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 1797207 T5038 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 1797207 T5038 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 1797210 T5038 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignmentBigger
[junit4] 2> 1797210 T5038 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1797211 T5039 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1797311 T5038 oasc.ZkTestServer.run start zk server on port:44553
[junit4] 2> 1797312 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797314 T5045 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c3db2c name:ZooKeeperConnection Watcher:127.0.0.1:44553 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797314 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797315 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797316 T5047 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fbd8e4 name:ZooKeeperConnection Watcher:127.0.0.1:44553 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797316 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797316 T5038 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1797319 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797320 T5049 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e2db2b name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797320 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797320 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1797321 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797322 T5051 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a4dda0 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797322 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797323 T5038 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1797324 T5038 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1797326 T5038 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1797327 T5038 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1797328 T5038 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:44553_solr
[junit4] 2> 1797328 T5038 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1797329 T5038 oasc.Overseer.start Overseer (id=92764408568020995-127.0.0.1:44553_solr-n_0000000000) starting
[junit4] 2> 1797331 T5038 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1797332 T5038 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1797334 T5038 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1797335 T5038 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1797337 T5038 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1797339 T5038 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1797342 T5038 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1797342 T5053 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1797343 T5052 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1797343 T5038 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1797345 T5038 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1797345 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797348 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797349 T5056 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@515bac name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797349 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797350 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797351 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node0
[junit4] 2> 1797353 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797354 T5060 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56b2f5 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797354 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797355 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797356 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 1797358 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797359 T5063 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a1c369 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797360 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797360 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797362 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 1797364 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797365 T5066 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19d7cbb name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797365 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797366 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797367 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node3
[junit4] 2> 1797370 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797370 T5069 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1012e6b name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797371 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797371 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797373 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node4
[junit4] 2> 1797375 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797376 T5072 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fe8a79 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797376 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797377 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797378 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node5
[junit4] 2> 1797381 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797383 T5075 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1629428 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797383 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797384 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797385 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node6
[junit4] 2> 1797388 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797389 T5078 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d7dd65 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797389 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797389 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797391 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node7
[junit4] 2> 1797393 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797394 T5081 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bf14f3 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797394 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797395 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797396 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node8
[junit4] 2> 1797399 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797400 T5084 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12d2629 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797400 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797407 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797409 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node9
[junit4] 2> 1797415 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797416 T5087 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e72211 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797416 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797417 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797419 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node10
[junit4] 2> 1797432 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797433 T5090 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6eb218 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797433 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797434 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797436 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node11
[junit4] 2> 1797442 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797446 T5093 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19de902 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797446 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797447 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797449 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node12
[junit4] 2> 1797463 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797471 T5096 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@171306d name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797472 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797473 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797474 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node13
[junit4] 2> 1797479 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797480 T5099 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b6e9b1 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797481 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797481 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797483 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node14
[junit4] 2> 1797491 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797493 T5102 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1664be1 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797493 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797494 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797495 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node15
[junit4] 2> 1797519 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797519 T5105 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e84b5e name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797520 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797520 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797522 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node16
[junit4] 2> 1797536 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797539 T5108 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ec295c name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797539 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797540 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797541 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node17
[junit4] 2> 1797545 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797546 T5111 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18d104f name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797546 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797547 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797548 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node18
[junit4] 2> 1797551 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797552 T5114 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fe4125 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797552 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797553 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797554 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node19
[junit4] 2> 1797558 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797560 T5117 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f0adc3 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797560 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797561 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797562 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node20
[junit4] 2> 1797574 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797576 T5120 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1da28fe name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797577 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797577 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797579 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node21
[junit4] 2> 1797583 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797584 T5123 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1091d30 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797584 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797585 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797587 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node22
[junit4] 2> 1797591 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797592 T5126 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@181cdfd name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797592 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797593 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797595 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node23
[junit4] 2> 1797598 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797599 T5129 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3c8049 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797599 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797600 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797602 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node24
[junit4] 2> 1797605 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797606 T5132 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ac558a name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797606 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797607 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797609 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node25
[junit4] 2> 1797613 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797614 T5135 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e5beca name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797614 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797615 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797617 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node26
[junit4] 2> 1797621 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797622 T5138 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b57269 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797622 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797623 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797624 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node27
[junit4] 2> 1797628 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797629 T5141 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@128296a name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797629 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797630 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797631 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node28
[junit4] 2> 1797635 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797636 T5144 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9578a6 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797637 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797637 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797639 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node29
[junit4] 2> 1797643 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797644 T5147 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ed239f name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797645 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797646 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797647 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node30
[junit4] 2> 1797651 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797652 T5150 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d8e471 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797653 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797653 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797655 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node31
[junit4] 2> 1797660 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797661 T5153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@109a5c4 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797661 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797662 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797663 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node32
[junit4] 2> 1797668 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797668 T5156 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1711c78 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797669 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797669 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797671 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node33
[junit4] 2> 1797676 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797676 T5159 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9956d6 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797677 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797677 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797679 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node34
[junit4] 2> 1797683 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797684 T5162 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@db3779 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797685 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797685 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797687 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node35
[junit4] 2> 1797693 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797694 T5165 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c15abf name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797694 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797695 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797696 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node36
[junit4] 2> 1797702 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797703 T5168 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16a50a5 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797703 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797704 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797705 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node37
[junit4] 2> 1797710 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797711 T5171 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14b7092 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797711 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797712 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797713 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node38
[junit4] 2> 1797718 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797719 T5173 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d73d27 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797719 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797720 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797721 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node39
[junit4] 2> 1797726 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797728 T5177 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16a817c name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797728 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797729 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797730 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node40
[junit4] 2> 1797735 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797737 T5180 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f72005 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797737 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797738 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797740 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node41
[junit4] 2> 1797745 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797746 T5183 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1da8965 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797746 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797747 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797748 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node42
[junit4] 2> 1797752 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797754 T5186 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e7f903 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797754 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797754 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797756 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node43
[junit4] 2> 1797762 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797763 T5189 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17b5297 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797763 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797764 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797765 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node44
[junit4] 2> 1797769 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797771 T5192 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fd423e name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797771 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797772 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797773 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node45
[junit4] 2> 1797778 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797779 T5195 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d96a25 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797779 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797780 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797781 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node46
[junit4] 2> 1797788 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797789 T5198 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2dc7ea name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797789 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797790 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797792 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node47
[junit4] 2> 1797798 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797799 T5201 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@166f55a name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797799 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797800 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797801 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node48
[junit4] 2> 1797807 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797808 T5204 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a9d665 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797808 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797808 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797809 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node49
[junit4] 2> 1797815 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797815 T5207 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@180af06 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797816 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797816 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797817 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node50
[junit4] 2> 1797825 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797826 T5210 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@85d7a4 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797826 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797827 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797829 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node51
[junit4] 2> 1797837 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797839 T5213 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@df5ff4 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797839 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797840 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797841 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node52
[junit4] 2> 1797850 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797851 T5216 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fc9165 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797851 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797852 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797854 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node53
[junit4] 2> 1797860 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797861 T5219 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d777fa name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797862 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797862 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797864 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node54
[junit4] 2> 1797872 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797873 T5222 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f07965 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797873 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797874 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797876 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node55
[junit4] 2> 1797882 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797884 T5225 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@194aa14 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797884 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797884 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797886 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node56
[junit4] 2> 1797893 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797894 T5228 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19a4f78 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797894 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797895 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797896 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node57
[junit4] 2> 1797904 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797905 T5231 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a04536 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797905 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797906 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797907 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node58
[junit4] 2> 1797925 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797925 T5234 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ec37eb name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797926 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797926 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797928 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node59
[junit4] 2> 1797935 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797936 T5237 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c769a7 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797939 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797941 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797943 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node60
[junit4] 2> 1797950 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797951 T5240 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8288c1 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797951 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797952 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797953 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node61
[junit4] 2> 1797961 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797961 T5243 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1391927 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797962 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797962 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797964 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node62
[junit4] 2> 1797969 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797971 T5245 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f5e126 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797972 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797973 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797974 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node63
[junit4] 2> 1797982 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797983 T5249 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1930d07 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797983 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797984 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797985 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node64
[junit4] 2> 1797994 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1797995 T5252 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13ae22c name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1797995 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1797996 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1797997 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node65
[junit4] 2> 1798005 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1798006 T5255 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2bd371 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1798006 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1798007 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1798008 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node66
[junit4] 2> 1798016 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1798016 T5258 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7fbe0a name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1798017 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1798017 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1798019 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node67
[junit4] 2> 1798026 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1798027 T5261 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11c414e name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1798027 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1798028 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1798030 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node68
[junit4] 2> 1798036 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1798038 T5264 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14dd93d name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1798038 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1798039 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1798040 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node69
[junit4] 2> 1798049 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1798050 T5267 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c7b9c6 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1798050 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1798051 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1798052 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node70
[junit4] 2> 1798060 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1798061 T5270 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78529c name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1798061 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1798062 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1798063 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node71
[junit4] 2> 1798072 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1798073 T5273 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1abbc8e name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1798073 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1798074 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1798075 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node72
[junit4] 2> 1798087 T5038 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1798088 T5276 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7123b7 name:ZooKeeperConnection Watcher:127.0.0.1:44553/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1798088 T5038 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1798089 T5038 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1798090 T5038 oascc.SolrZkClient.makePath makePath: /live_nodes/node73
[junit4] 2> 1798107 T5290 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1798108 T5052 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=20 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"20",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1798108 T5052 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3, shard4, shard5, shard6, shard7, shard8, shard9, shard10, shard11, shard12, shard13, shard14, shard15, shard16, shard17, shard18, shard19, shard20]
[junit4] 2> 1798109 T5052 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard8
[junit4] 2> 1798112 T5277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798112 T5174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798112 T5103 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5250 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5246 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5187 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798112 T5238 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5202 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798115 T5118 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798112 T5199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798112 T5154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798112 T5232 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798112 T5247 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798115 T5079 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798115 T5073 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798115 T5148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798115 T5136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798115 T5064 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798115 T5133 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798115 T5229 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798115 T5205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798115 T5085 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798115 T5070 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5088 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5097 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5211 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5112 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5127 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5145 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5193 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5067 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5178 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5181 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5274 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5100 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798114 T5259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5268 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5271 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5184 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5094 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5142 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5076 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5115 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5241 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5139 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5196 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5223 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5256 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5262 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5217 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5253 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5082 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5091 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798113 T5235 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798112 T5057 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798112 T5220 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798112 T5058 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798134 T5265 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 74)
[junit4] 2> 1798136 T5290 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1798137 T5052 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=20 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node6",
[junit4] 2> "core":"core6",
[junit4] 2> "core_node_name":"node6",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"20",
[junit4] 2> "base_url":"http://node6/solr/"}
[junit4] 2> 1798137 T5052 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=20
[junit4] 2> 1798137 T5052 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1798139 T5052 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=20 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node0",
[junit4] 2> "core":"core0",
[junit4] 2> "core_node_name":"node0",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"20",
[junit4] 2> "base_url":"http://node0/solr/"}
[junit4] 2> 1798139 T5052 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=20
[junit4] 2> 1798139 T5052 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard5
[junit4] 2> 1798140 T5290 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1798141 T5052 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=20 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"20",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 1798141 T5052 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=20
[junit4] 2> 1798141 T5052 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 1798143 T5052 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=20 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node4",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node4",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"20",
[junit4] 2> "base_url":"http://node4/solr/"}
[junit4] 2> 1798143 T5052 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=20
[junit4] 2> 1798144 T5052 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard11
[junit4] 2> 1798145 T5290 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1798145 T5052 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=20 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node7",
[junit4] 2> "core":"core7",
[junit4] 2> "core_node_name":"node7",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"20",
[junit4] 2> "base_url":"http://node7/solr/"}
[junit4] 2> 1798146 T5052 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=20
[junit4] 2> 1798146 T5052 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard13
[junit4] 2> 1798148 T5052 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=20 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node5",
[junit4] 2> "core":"core5",
[junit4] 2> "core_node_name":"node5",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"20",
[junit4] 2> "base_url":"http://node5/solr/"}
[junit4] 2> 1798148 T5052 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=20
[junit4] 2> 1798148 T5052 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard15
[junit4] 2> 1798150 T5290 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1798150 T5052 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=20 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node8",
[junit4] 2> "core":"core8",
[junit4] 2> "core_node_name":"node8",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"20",
[junit4] 2> "base_url":"http://node8/solr/"}
[junit4] 2> 1798151 T5052 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=20
[junit4] 2> 1798151 T5052 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard20
[junit4] 2> 1798153 T5052 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=20 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node10",
[junit4] 2> "core":"core10",
[junit4] 2> "core_node_name":"node10",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"20",
[junit4] 2> "base_url":"http://node10/solr/"}
[junit4] 2> 1798153 T5052 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=20
[junit4] 2> 1798153 T5052 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard19
[junit4] 2> 1798155 T5290 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1798156 T5052 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=20 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node3",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"20",
[junit4] 2> "base_url":"http://node3/solr/"}
[junit4] 2> 1798156 T5052 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=20
[junit4] 2> 1798156 T5052 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard6
[junit4] 2> 1798158 T5052 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=20 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node11",
[junit4] 2> "core":"core11",
[junit4] 2> "core_node_name":"node11",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"20",
[junit4] 2> "base_url":"http://node11/solr/"}
[junit4] 2> 1798158 T5052 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=20
[junit4] 2> 1798159 T5052 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard7
[junit4] 2> 1798160 T5290 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1798160 T5052 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=20 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node9",
[junit4] 2> "core":"core9",
[junit4] 2> "core_node_name":"node9",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"20",
[junit4] 2> "base_url":"http://node9/solr/"}
[junit4] 2> 1798161 T5052 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=20
[junit4] 2> 1798161 T5052 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard18
[junit4] 2> 1798163 T5052 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=20 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node13",
[junit4] 2> "core":"core13",
[junit4] 2> "core_node_name":"node13",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"20",
[junit4] 2> "base_url":"http://node13/solr/"}
[junit4] 2> 1798163 T5052 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=20
[junit4] 2> 1798163 T5052 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard14
[junit4] 2> 1798165 T5290 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1798165 T
[...truncated too long message...]
38 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1880688 T5038 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1880689 T5038 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:33514_solr
[junit4] 2> 1880690 T5038 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1880691 T5038 oasc.Overseer.start Overseer (id=92764414030577667-127.0.0.1:33514_solr-n_0000000000) starting
[junit4] 2> 1880693 T5038 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1880695 T5038 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1880697 T5038 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1880698 T5038 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1880700 T5038 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1880703 T5038 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1880703 T5808 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1880705 T5807 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
[junit4] 2> 1880705 T5807 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1880706 T5810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1880707 T5807 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core2",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1880707 T5810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1880708 T5807 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1880709 T5807 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core3",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1880710 T5811 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1880710 T5810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1880806 T5038 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:33514 33514
[junit4] 2> 1881511 T5794 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:33514 33514
[junit4] 2> 1881512 T5038 oas.SolrTestCaseJ4.tearDown ###Ending testReplay
[junit4] 2> 1881512 T5038 oasc.Overseer.close Overseer (id=92764414030577667-127.0.0.1:33514_solr-n_0000000000) closing
[junit4] 2> 1881513 T5807 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:33514_solr
[junit4] 2> 1881514 T5812 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:320)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:292)
[junit4] 2>
[junit4] 2> 1884515 T5038 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J0/./solr.cloud.OverseerTest-A4A87ADA429D0299-001
[junit4] 2> NOTE: test params are: codec=Lucene45, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=pt_BR, timezone=America/Blanc-Sablon
[junit4] 2> NOTE: Linux 3.13.0-39-generic i386/Oracle Corporation 1.7.0_67 (32-bit)/cpus=8,threads=1,free=354691784,total=534773760
[junit4] 2> NOTE: All tests run in this JVM: [TestCollapseQParserPlugin, PathHierarchyTokenizerFactoryTest, MBeansHandlerTest, SolrIndexSplitterTest, TestRemoteStreaming, TestStressRecovery, TestManagedStopFilterFactory, IndexSchemaTest, CustomCollectionTest, DocumentAnalysisRequestHandlerTest, AnalyticsMergeStrategyTest, TestConfig, TermsComponentTest, TestFaceting, JSONWriterTest, SuggesterWFSTTest, TestRandomDVFaceting, HdfsBasicDistributedZk2Test, TestMissingGroups, ClusterStateUpdateTest, SimpleFacetsTest, TestManagedSynonymFilterFactory, DistribCursorPagingTest, PrimitiveFieldTypeTest, PingRequestHandlerTest, DirectSolrSpellCheckerTest, TestRTGBase, TestDocBasedVersionConstraints, TestJmxMonitoredMap, TestExceedMaxTermLength, TestSolr4Spatial2, DistributedIntervalFacetingTest, DateMathParserTest, UnloadDistributedZkTest, InfoHandlerTest, SignatureUpdateProcessorFactoryTest, TestFieldCollectionResource, TestSurroundQueryParser, TestInitQParser, BasicFunctionalityTest, DocumentBuilderTest, TestInfoStreamLogging, MoreLikeThisHandlerTest, TestSchemaSimilarityResource, ZkCLITest, SuggesterTSTTest, WordBreakSolrSpellCheckerTest, PeerSyncTest, TestSort, BadIndexSchemaTest, TestIntervalFaceting, DistributedSpellCheckComponentTest, DistributedSuggestComponentTest, TestReplicationHandler, HighlighterMaxOffsetTest, QueryResultKeyTest, TestCursorMarkWithoutUniqueKey, ResourceLoaderTest, PreAnalyzedFieldTest, TestMiniSolrCloudCluster, HighlighterTest, XmlUpdateRequestHandlerTest, TestDefaultSimilarityFactory, TestSimpleQParserPlugin, CoreAdminRequestStatusTest, SolrCoreCheckLockOnStartupTest, TestLeaderElectionZkExpiry, SuggestComponentTest, DocValuesMultiTest, TestDynamicFieldResource, DirectUpdateHandlerTest, TestSchemaResource, EchoParamsTest, JsonLoaderTest, TestAnalyzedSuggestions, NoCacheHeaderTest, TestFoldingMultitermQuery, DisMaxRequestHandlerTest, TestReloadAndDeleteDocs, XsltUpdateRequestHandlerTest, TestLuceneMatchVersion, TestSolrJ, TestTrie, AliasIntegrationTest, TestOverriddenPrefixQueryForCustomFieldType, TestCopyFieldCollectionResource, TestSolrDeletionPolicy2, SyncSliceTest, TestComponentsName, TestSolrXmlPersistor, RequiredFieldsTest, IndexBasedSpellCheckerTest, CoreAdminHandlerTest, RegexBoostProcessorTest, SpellPossibilityIteratorTest, DocExpirationUpdateProcessorFactoryTest, DeleteReplicaTest, TestNumberUtils, TestStressVersions, BadCopyFieldTest, FileUtilsTest, CursorPagingTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, OverseerTest]
[junit4] Completed on J0 in 87.33s, 9 tests, 1 error <<< FAILURES!
[...truncated 644 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/build.xml:474: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/build.xml:454: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/common-build.xml:494: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/lucene/common-build.xml:1350: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/lucene/common-build.xml:970: There were test failures: 421 suites, 1742 tests, 1 error, 38 ignored (17 assumptions)
Total time: 98 minutes 9 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.7.0_67 -server -XX:+UseG1GC (asserts: false)
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-4.10-Linux (64bit/jdk1.8.0_40-ea-b09) - Build
# 68 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.10-Linux/68/
Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseSerialGC (asserts: false)
1 tests failed.
REGRESSION: org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings
Error Message:
startOffset must be non-negative, and endOffset must be >= startOffset, startOffset=39,endOffset=34
Stack Trace:
java.lang.IllegalArgumentException: startOffset must be non-negative, and endOffset must be >= startOffset, startOffset=39,endOffset=34
at __randomizedtesting.SeedInfo.seed([4400A1EEFBBE1D5:6E1BB50FB6F5C126]:0)
at org.apache.lucene.analysis.tokenattributes.PackedTokenAttributeImpl.setOffset(PackedTokenAttributeImpl.java:107)
at org.apache.lucene.analysis.shingle.ShingleFilter.incrementToken(ShingleFilter.java:345)
at org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:68)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:703)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:614)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:513)
at org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:946)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 5795 lines...]
[junit4] Suite: org.apache.lucene.analysis.core.TestRandomChains
[junit4] 2> TEST FAIL: useCharFilter=false text='\u4557 < pfgxf \uf122\udab4\udec9\u0224\u55fe pweucpeb h � s'
[junit4] 2> Exception from random analyzer:
[junit4] 2> charfilters=
[junit4] 2> tokenizer=
[junit4] 2> org.apache.lucene.analysis.core.WhitespaceTokenizer(org.apache.lucene.analysis.core.TestRandomChains$CheckThatYouDidntReadAnythingReaderWrapper@46169795)
[junit4] 2> filters=
[junit4] 2> org.apache.lucene.analysis.miscellaneous.WordDelimiterFilter(ValidatingTokenFilter@50d8cbb4 term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word, [-62, -33, 47, -114, 15, -104, 79, 27, 69, -109, 64, 54, 109, -52, -85, -103, -78, -126, -34, 9, -89, 13, 100, 47, 54, 70, 57, -23, -54, -7, 118, 26, -8, 71, -64, -43, 53, 68, 76, 60, -24, 40, -9, -16, -46, 56, -88, 78, 23, -103, 111, -55, 23, -122, 40, -22, -3, 28, -126, -69, -64, 60, 51, 93, -118, -24, 119, -93, 113, 18, 55, -79, 3, 90, -48, 15, -100, -36, 60, 36, -11, 1, -7, 35, 84, -5, 91, -13, -60, 90, 100, -1, 4, -104, 2, -98, -99, -87, 50, 97, 28, 66, -58, 3, -37, -53, 75, 99, 46, 36, 47, 108, 26, 63, -26, -120, 24, -42, -97, 75, -118, 79, 6, -111, 44, 69, 124, -111, 74, 87, 35, -106, 112, 89, 108, 9, -53, 42, -12, 52, 114, 119, -62, -56, -119, -127, -17, 50, 67, -48, -108, 4, -92, 47, 90, -15, -107, 45, -101, -101, 30, -84, -120, 7, 15], -10, [c, wavyix, lniio, llxpkmwq])
[junit4] 2> org.apache.lucene.analysis.ckb.SoraniNormalizationFilter(ValidatingTokenFilter@6af53ce0 term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word)
[junit4] 2> org.apache.lucene.analysis.miscellaneous.WordDelimiterFilter(ValidatingTokenFilter@77ca5df4 term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word, -47, [x, dygonab, ruzoopid, cea, bptab])
[junit4] 2> org.apache.lucene.analysis.shingle.ShingleFilter(ValidatingTokenFilter@41915004 term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word, <ALPHANUM>)
[junit4] 2> offsetsAreCorrect=false
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestRandomChains -Dtests.method=testRandomChainsWithLargeStrings -Dtests.seed=4400A1EEFBBE1D5 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=sr_BA -Dtests.timezone=US/Mountain -Dtests.asserts=false -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.83s J1 | TestRandomChains.testRandomChainsWithLargeStrings <<<
[junit4] > Throwable #1: java.lang.IllegalArgumentException: startOffset must be non-negative, and endOffset must be >= startOffset, startOffset=39,endOffset=34
[junit4] > at __randomizedtesting.SeedInfo.seed([4400A1EEFBBE1D5:6E1BB50FB6F5C126]:0)
[junit4] > at org.apache.lucene.analysis.tokenattributes.PackedTokenAttributeImpl.setOffset(PackedTokenAttributeImpl.java:107)
[junit4] > at org.apache.lucene.analysis.shingle.ShingleFilter.incrementToken(ShingleFilter.java:345)
[junit4] > at org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:68)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:703)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:614)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:513)
[junit4] > at org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:946)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/lucene/build/analysis/common/test/J1/./lucene.analysis.core.TestRandomChains-4400A1EEFBBE1D5-001
[junit4] 2> NOTE: test params are: codec=Lucene46, sim=DefaultSimilarity, locale=sr_BA, timezone=US/Mountain
[junit4] 2> NOTE: Linux 3.13.0-39-generic amd64/Oracle Corporation 1.8.0_40-ea (64-bit)/cpus=8,threads=1,free=90999944,total=127795200
[junit4] 2> NOTE: All tests run in this JVM: [TestScandinavianFoldingFilterFactory, TestSolrSynonymParser, TestNorwegianMinimalStemFilter, TestGreekLowerCaseFilterFactory, TestRollingCharBuffer, TestKStemmer, TestCaseInsensitive, TokenTypeSinkTokenizerTest, TestCzechStemmer, TestCapitalizationFilter, TestDictionaryCompoundWordTokenFilterFactory, TestPortugueseLightStemFilterFactory, TestKeepWordFilter, TestTurkishLowerCaseFilter, TestGalicianAnalyzer, TestFrenchAnalyzer, TestCJKWidthFilter, TestHungarianLightStemFilter, TestStopFilter, TestPersianAnalyzer, TestSynonymMap, TestFlagNum, TestStandardFactories, TestCondition, HTMLStripCharFilterTest, CommonGramsFilterTest, TestRandomChains]
[junit4] Completed on J1 in 38.56s, 2 tests, 1 error <<< FAILURES!
[...truncated 593 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/build.xml:474: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/build.xml:454: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/lucene/build.xml:540: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/lucene/common-build.xml:2107: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/lucene/analysis/build.xml:106: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/lucene/analysis/build.xml:38: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/lucene/module-build.xml:60: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/lucene/common-build.xml:1350: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/lucene/common-build.xml:970: There were test failures: 281 suites, 1470 tests, 1 error, 1 ignored
Total time: 25 minutes 30 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseSerialGC (asserts: false)
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-4.10-Linux (64bit/jdk1.7.0_67) - Build # 67 -
Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.10-Linux/67/
Java: 64bit/jdk1.7.0_67 -XX:-UseCompressedOops -XX:+UseParallelGC (asserts: true)
1 tests failed.
FAILED: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
Stack Trace:
org.apache.solr.common.SolrException: Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
at __randomizedtesting.SeedInfo.seed([5521FA015B1A02AA:512975F249BFED8B]:0)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:150)
at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:157)
at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:662)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:40)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:137)
... 48 more
Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at org.apache.solr.common.cloud.SolrZkClient$11.execute(SolrZkClient.java:462)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:459)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:416)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:403)
at org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:142)
at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:34)
... 49 more
Build Log:
[...truncated 11631 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.OverseerTest-5521FA015B1A02AA-001/init-core-data-001
[junit4] 2> 1620465 T4352 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
[junit4] 2> 1620465 T4352 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 1620465 T4352 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 1620468 T4352 oas.SolrTestCaseJ4.setUp ###Starting testOverseerFailure
[junit4] 2> 1620468 T4352 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1620469 T4353 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1620569 T4352 oasc.ZkTestServer.run start zk server on port:32998
[junit4] 2> 1620570 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1620575 T4359 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@39bebd5f name:ZooKeeperConnection Watcher:127.0.0.1:32998 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1620575 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1620579 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1620581 T4361 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@36b98a2f name:ZooKeeperConnection Watcher:127.0.0.1:32998 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1620582 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1620582 T4352 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1620589 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1620591 T4363 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4fbb50df name:ZooKeeperConnection Watcher:127.0.0.1:32998/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1620591 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1620591 T4352 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1620595 T4352 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1620598 T4352 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1620600 T4352 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1620605 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1620607 T4365 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@339416c9 name:ZooKeeperConnection Watcher:127.0.0.1:32998/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1620607 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1620609 T4352 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1620613 T4352 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 1620617 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1620619 T4369 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@74e0321d name:ZooKeeperConnection Watcher:127.0.0.1:32998/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1620619 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1620620 T4352 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1620622 T4352 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1620624 T4352 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1620627 T4352 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1620629 T4352 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:32998_solr
[junit4] 2> 1620630 T4352 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1620632 T4352 oasc.Overseer.start Overseer (id=92765866147905540-127.0.0.1:32998_solr-n_0000000000) starting
[junit4] 2> 1620633 T4352 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1620636 T4352 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1620639 T4352 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1620643 T4352 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1620647 T4352 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1620651 T4352 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1620656 T4352 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1620657 T4371 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1620658 T4370 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1621659 T4373 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1621661 T4370 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1621662 T4370 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 1621662 T4370 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1621664 T4367 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1621664 T4366 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1622159 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1622165 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1622168 T4373 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1622169 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1622170 T4370 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1622170 T4370 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1622171 T4373 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1622172 T4366 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1622172 T4367 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1622660 T4371 oasc.OverseerCollectionProcessor.amILeader org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor.amILeader(OverseerCollectionProcessor.java:514)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:272)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1622660 T4371 oasc.OverseerCollectionProcessor.amILeader According to ZK I (id=92765866147905540-127.0.0.1:32998_solr-n_0000000000) am no longer a leader.
[junit4] 2> 1623226 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1623230 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1623231 T4375 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@24384e8c name:ZooKeeperConnection Watcher:127.0.0.1:32998/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1623231 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1623231 T4352 oasc.Overseer.close Overseer (id=92765866147905540-127.0.0.1:32998_solr-n_0000000000) closing
[junit4] 2> 1623232 T4370 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:32998_solr
[junit4] 2> 1623233 T4352 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1623234 T4376 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:320)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:292)
[junit4] 2>
[junit4] 2> 1623235 T4352 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1623236 T4352 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:32998_solr
[junit4] 2> 1623236 T4352 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1623237 T4352 oasc.Overseer.start Overseer (id=92765866147905541-127.0.0.1:32998_solr-n_0000000001) starting
[junit4] 2> 1623242 T4352 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1623243 T4378 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1623243 T4377 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1623245 T4377 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1623245 T4377 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1623246 T4380 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1623246 T4367 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1623246 T4366 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1623299 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1623299 T4380 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1623304 T4352 oasc.LeaderElector.checkIfIamLeader WARN Failed setting watch org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election/92765866147905539-node1_core1-n_0000000002
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:148)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
[junit4] 2> at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:157)
[junit4] 2> at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:662)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:606)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4] 2> at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1623306 T4377 oasc.SolrException.log ERROR Problem cleaning up collection in zk:collection1:org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:125)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:189)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:186)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:186)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:670)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:665)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:665)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:665)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.removeCore(Overseer.java:1104)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:362)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:244)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1623306 T4381 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1623305 T4366 oasc.LeaderElector.checkIfIamLeader WARN Our node is no longer in line to be leader
[junit4] 2> 1623306 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1623309 T4380 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1623310 T4377 oasc.Overseer$ClusterStateUpdater.setShardLeader ERROR Could not mark shard leader for non existing collection:collection1
[junit4] 2> 1623413 T4367 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1623413 T4381 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1624309 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1625312 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1626314 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1627316 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1628320 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1629322 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1630326 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1631328 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1632331 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1633333 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1634336 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1635338 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1636340 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1637342 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1638345 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1638352 T4352 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:32998 32998
[junit4] 2> 1639026 T4353 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:32998 32998
[junit4] 2> 1639028 T4352 oas.SolrTestCaseJ4.tearDown ###Ending testOverseerFailure
[junit4] 2> 1639028 T4352 oasc.Overseer.close Overseer (id=92765866147905541-127.0.0.1:32998_solr-n_0000000001) closing
[junit4] 2> 1639028 T4377 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:32998_solr
[junit4] 2> 1639030 T4382 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:320)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:292)
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerFailure -Dtests.seed=5521FA015B1A02AA -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_PR -Dtests.timezone=Europe/Tiraspol -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 18.6s J1 | OverseerTest.testOverseerFailure <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
[junit4] > at __randomizedtesting.SeedInfo.seed([5521FA015B1A02AA:512975F249BFED8B]:0)
[junit4] > at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:150)
[junit4] > at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
[junit4] > at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
[junit4] > at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
[junit4] > at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
[junit4] > at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
[junit4] > at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:157)
[junit4] > at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:662)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > Caused by: org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
[junit4] > at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:40)
[junit4] > at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:137)
[junit4] > ... 48 more
[junit4] > Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] > at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$11.execute(SolrZkClient.java:462)
[junit4] > at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:459)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:416)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:403)
[junit4] > at org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:142)
[junit4] > at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:34)
[junit4] > ... 49 more
[junit4] 2> 1639043 T4352 oas.SolrTestCaseJ4.setUp ###Starting testStateChange
[junit4] 2> 1639044 T4352 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1639045 T4383 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1639144 T4352 oasc.ZkTestServer.run start zk server on port:44886
[junit4] 2> 1639145 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1639148 T4389 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2a59f3ee name:ZooKeeperConnection Watcher:127.0.0.1:44886/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1639148 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1639149 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1639150 T4391 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32ebfc0a name:ZooKeeperConnection Watcher:127.0.0.1:44886 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1639150 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1639152 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1639153 T4393 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4ceb9b64 name:ZooKeeperConnection Watcher:127.0.0.1:44886 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1639153 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1639154 T4352 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1639157 T4352 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1639159 T4352 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1639161 T4352 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1639162 T4352 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1639163 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1639164 T4395 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19043c85 name:ZooKeeperConnection Watcher:127.0.0.1:44886/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1639164 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1639165 T4352 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1639166 T4352 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1639167 T4352 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1639168 T4352 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1639169 T4352 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:44886_solr
[junit4] 2> 1639169 T4352 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1639170 T4352 oasc.Overseer.start Overseer (id=92765867365302275-127.0.0.1:44886_solr-n_0000000000) starting
[junit4] 2> 1639171 T4352 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1639172 T4352 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1639174 T4352 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1639175 T4352 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1639176 T4352 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1639178 T4352 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1639180 T4352 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1639181 T4397 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1639181 T4396 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1639183 T4396 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1639183 T4396 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1639184 T4399 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1639184 T4400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1639234 T4399 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1639235 T4396 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"active"}
[junit4] 2> 1639236 T4396 oasc.Overseer$ClusterStateUpdater.updateState node=core_node1 is already registered
[junit4] 2> 1639236 T4396 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1639339 T4400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1639387 T4352 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:44886 44886
[junit4] 2> 1639955 T4383 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:44886 44886
[junit4] 2> 1639956 T4352 oas.SolrTestCaseJ4.tearDown ###Ending testStateChange
[junit4] 2> 1639956 T4352 oasc.Overseer.close Overseer (id=92765867365302275-127.0.0.1:44886_solr-n_0000000000) closing
[junit4] 2> 1639957 T4396 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:44886_solr
[junit4] 2> 1639959 T4401 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:320)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:292)
[junit4] 2>
[junit4] 2> 1639966 T4352 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignment
[junit4] 2> 1639967 T4352 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1639968 T4402 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1640067 T4352 oasc.ZkTestServer.run start zk server on port:40082
[junit4] 2> 1640069 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1640071 T4408 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4becc9ef name:ZooKeeperConnection Watcher:127.0.0.1:40082 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1640071 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1640073 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1640074 T4410 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a26d9a7 name:ZooKeeperConnection Watcher:127.0.0.1:40082 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1640074 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1640075 T4352 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1640079 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1640080 T4412 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3dd5dc8b name:ZooKeeperConnection Watcher:127.0.0.1:40082/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1640080 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1640080 T4352 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1640083 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1640084 T4414 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@26686864 name:ZooKeeperConnection Watcher:127.0.0.1:40082/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1640084 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1640085 T4352 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1640087 T4352 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1640090 T4352 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1640091 T4352 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1640093 T4352 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:40082_solr
[junit4] 2> 1640093 T4352 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1640095 T4352 oasc.Overseer.start Overseer (id=92765867425857539-127.0.0.1:40082_solr-n_0000000000) starting
[junit4] 2> 1640097 T4352 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1640100 T4352 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1640103 T4352 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1640105 T4352 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1640108 T4352 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1640111 T4352 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1640114 T4352 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1640115 T4416 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1640116 T4352 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1640116 T4415 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1640117 T4352 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1640118 T4352 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1640119 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1640120 T4419 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@522c03b7 name:ZooKeeperConnection Watcher:127.0.0.1:40082/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1640120 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1640122 T4352 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1640123 T4352 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1
[junit4] 2> 1640127 T4422 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1640128 T4415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1640128 T4415 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 1640128 T4415 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1640131 T4421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1640131 T4420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1640627 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1640632 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1640634 T4422 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1640636 T4415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1640637 T4415 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1640637 T4415 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 1640638 T4422 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1640638 T4421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1640639 T4420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1641135 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 1641140 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 1641142 T4422 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1641145 T4415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1641145 T4415 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1641145 T4415 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1641146 T4422 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1641147 T4420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1641147 T4421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1641643 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 1641647 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 1641649 T4422 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1641652 T4415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node4",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1641652 T4415 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1641652 T4415 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1641653 T4422 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1641654 T4421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1641654 T4420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1642150 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1642155 T4422 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1642156 T4415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core5",
[junit4] 2> "core_node_name":"node5",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1642156 T4415 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1642156 T4415 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 1642259 T4421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1642259 T4420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1642655 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 1642659 T4422 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1642660 T4415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core6",
[junit4] 2> "core_node_name":"node6",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1642660 T4415 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1642661 T4415 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1642763 T4420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1642763 T4421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1643159 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 1643164 T4423 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:290)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:287)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:287)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:209)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1643164 T4419 oaz.ClientCnxn$EventThread.processEvent ERROR Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@78c2a1bb rejected from java.util.concurrent.ThreadPoolExecutor@320839ed[Shutting down, pool size = 2, active threads = 2, queued tasks = 0, completed tasks = 8]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
[junit4] 2> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:206)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 1644664 T4421 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard3/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:290)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:287)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:287)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:209)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1644667 T4352 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40082 40082
[junit4] 2> 1645376 T4402 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40082 40082
[junit4] 2> 1645377 T4352 oas.SolrTestCaseJ4.tearDown ###Ending testShardAssignment
[junit4] 2> 1645378 T4352 oasc.Overseer.close Overseer (id=92765867425857539-127.0.0.1:40082_solr-n_0000000000) closing
[junit4] 2> 1645378 T4415 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:40082_solr
[junit4] 2> 1645381 T4425 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:320)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:292)
[junit4] 2>
[junit4] 2> 1645385 T4352 oas.SolrTestCaseJ4.setUp ###Starting testBadQueueItem
[junit4] 2> 1645386 T4352 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1645387 T4426 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1645487 T4352 oasc.ZkTestServer.run start zk server on port:51839
[junit4] 2> 1645488 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1645490 T4432 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ff114d5 name:ZooKeeperConnection Watcher:127.0.0.1:51839 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1645490 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1645492 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1645493 T4434 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3710c3f5 name:ZooKeeperConnection Watcher:127.0.0.1:51839 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1645493 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1645493 T4352 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1645497 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1645498 T4436 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@75b2f026 name:ZooKeeperConnection Watcher:127.0.0.1:51839/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1645498 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1645498 T4352 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1645500 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1645501 T4438 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2ff1aa75 name:ZooKeeperConnection Watcher:127.0.0.1:51839/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1645501 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1645501 T4352 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1645502 T4352 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1645504 T4352 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1645505 T4352 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1645507 T4352 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:51839_solr
[junit4] 2> 1645507 T4352 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1645508 T4352 oasc.Overseer.start Overseer (id=92765867780931587-127.0.0.1:51839_solr-n_0000000000) starting
[junit4] 2> 1645510 T4352 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1645511 T4352 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1645513 T4352 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1645515 T4352 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1645516 T4352 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1645518 T4352 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1645520 T4352 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1645520 T4440 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1645521 T4352 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1645521 T4439 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1645522 T4352 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1645523 T4352 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1645525 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1645525 T4443 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10dfd41c name:ZooKeeperConnection Watcher:127.0.0.1:51839/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1645526 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1645527 T4352 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1645529 T4352 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1
[junit4] 2> 1645531 T4446 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1645532 T4439 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1645532 T4439 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 1645532 T4439 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1645534 T4445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1645534 T4444 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1646031 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1646044 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1646051 T4446 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1646057 T4439 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1646057 T4439 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1646058 T4439 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 1646061 T4446 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1646062 T4445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1646062 T4444 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1646553 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 1646557 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 1646560 T4446 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1646565 T4439 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1646565 T4439 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1646565 T4439 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1646568 T4446 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1646569 T4445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1646569 T4444 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1647062 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 1647066 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 1647069 T4446 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1647172 T4444 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1647172 T4445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1647222 T4446 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1647223 T4439 oasc.Overseer$ClusterStateUpdater.checkKeyExistence ERROR Skipping invalid Overseer message because it has no collection specified: {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core0",
[junit4] 2> "core_node_name":"node0",
[junit4] 2> "collection":"",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1647226 T4439 oasc.Overseer$ClusterStateUpdater.checkKeyExistence ERROR Skipping invalid Overseer message because it has no collection specified: {
[junit4] 2> "operation":"deletecore",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core0",
[junit4] 2> "core_node_name":"node0",
[junit4] 2> "collection":""}
[junit4] 2> 1647229 T4446 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1647229 T4439 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1647229 T4439 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection2 with shards [shard1, shard2, shard3]
[junit4] 2> 1647230 T4439 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1647333 T4444 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1647333 T4445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1647724 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection2/leader_elect/shard1/election
[junit4] 2> 1647729 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection2/leaders/shard1
[junit4] 2> 1647732 T4446 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1647734 T4439 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1647734 T4439 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1647735 T4439 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 1647736 T4446 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1647736 T4445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1647736 T4444 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1648233 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection2/leader_elect/shard3/election
[junit4] 2> 1648237 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection2/leaders/shard3
[junit4] 2> 1648239 T4446 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1648241 T4439 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1648241 T4439 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1648241 T4439 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1648243 T4446 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1648243 T4444 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1648243 T4445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1648740 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection2/leader_elect/shard2/election
[junit4] 2> 1648744 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection2/leaders/shard2
[junit4] 2> 1648746 T4446 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1648849 T4445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1648849 T4444 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1650399 T4445 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 1650401 T4352 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51839 51839
[junit4] 2> 1651055 T4426 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51839 51839
[junit4] 2> 1651056 T4352 oas.SolrTestCaseJ4.tearDown ###Ending testBadQueueItem
[junit4] 2> 1651056 T4352 oasc.Overseer.close Overseer (id=92765867780931587-127.0.0.1:51839_solr-n_0000000000) closing
[junit4] 2> 1651057 T4439 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:51839_solr
[junit4] 2> 1651059 T4447 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:320)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:292)
[junit4] 2>
[junit4] 2> 1651062 T4352 oas.SolrTestCaseJ4.setUp ###Starting testShardLeaderChange
[junit4] 2> 1651063 T4352 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1651063 T4448 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1651163 T4352 oasc.ZkTestServer.run start zk server on port:45641
[junit4] 2> 1651164 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1651173 T4454 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@490e490e name:ZooKeeperConnection Watcher:127.0.0.1:45641/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1651174 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1651174 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1651176 T4456 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7bde31f name:ZooKeeperConnection Watcher:127.0.0.1:45641 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1651176 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1651178 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1651179 T4458 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3bc24107 name:ZooKeeperConnection Watcher:127.0.0.1:45641 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1651179 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1651179 T4352 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1651182 T4352 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1651186 T4459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1651186 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1651187 T4462 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4e28e718 name:ZooKeeperConnection Watcher:127.0.0.1:45641/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1651187 T4459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1651187 T4463 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@344d8833 name:ZooKeeperConnection Watcher:127.0.0.1:45641/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1651187 T4459 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1651187 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1651188 T4459 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1651188 T4352 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1651189 T4459 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1651190 T4352 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1651190 T4352 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1651190 T4459 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1651192 T4459 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:45641_solr
[junit4] 2> 1651192 T4459 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1651192 T4352 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 1651193 T4459 oasc.Overseer.start Overseer (id=92765868152913923-127.0.0.1:45641_solr-n_0000000000) starting
[junit4] 2> 1651194 T4352 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1651194 T4459 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1651196 T4459 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1651197 T4459 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1651198 T4459 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1651199 T4459 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1651201 T4459 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1651203 T4459 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpir
[...truncated too long message...]
] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1716536 T5166 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 1716536 T5166 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1716537 T5169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1716537 T5163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1716538 T5162 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1717035 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1717040 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1717042 T5169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1717044 T4352 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1717045 T5171 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9764032 name:ZooKeeperConnection Watcher:127.0.0.1:40514/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1717045 T4352 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1717046 T4352 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1717047 T4352 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 1717048 T5169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1717048 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1717049 T5166 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1717049 T5166 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1717051 T4352 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1717053 T5169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1717155 T5172 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1717155 T5162 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1717158 T4352 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40514 40514
[junit4] 2> 1717587 T5149 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40514 40514
[junit4] 2> 1717588 T4352 oas.SolrTestCaseJ4.tearDown ###Ending testDoubleAssignment
[junit4] 2> 1717588 T4352 oasc.Overseer.close Overseer (id=92765872434970628-127.0.0.1:40514_solr-n_0000000000) closing
[junit4] 2> 1717588 T5166 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:40514_solr
[junit4] 2> 1717589 T5173 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:74)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:320)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:292)
[junit4] 2>
[junit4] 2> 1720589 T4352 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build/solr-core/test/J1/./solr.cloud.OverseerTest-5521FA015B1A02AA-001
[junit4] 2> NOTE: test params are: codec=Lucene410: {}, docValues:{}, sim=DefaultSimilarity, locale=es_PR, timezone=Europe/Tiraspol
[junit4] 2> NOTE: Linux 3.13.0-39-generic amd64/Oracle Corporation 1.7.0_67 (64-bit)/cpus=8,threads=1,free=124961328,total=425721856
[junit4] 2> NOTE: All tests run in this JVM: [HdfsChaosMonkeySafeLeaderTest, CollectionsAPIDistributedZkTest, SampleTest, TimeZoneUtilsTest, HdfsLockFactoryTest, DistribDocExpirationUpdateProcessorTest, TestRecovery, TestQuerySenderNoQuery, QueryParsingTest, SolrInfoMBeanTest, StressHdfsTest, SearchHandlerTest, FieldAnalysisRequestHandlerTest, DistributedDebugComponentTest, AtomicUpdatesTest, LukeRequestHandlerTest, OverseerRolesTest, CachingDirectoryFactoryTest, PrimUtilsTest, ScriptEngineTest, AssignTest, TestConfigSets, URLClassifyProcessorTest, OverseerStatusTest, TestDistributedMissingSort, CircularListTest, DistributedQueryComponentCustomSortTest, RollingRestartTest, DateFieldTest, SolrRequestParserTest, DOMUtilTest, TestSolrQueryParserResource, TestExpandComponent, TestFunctionQuery, BasicZkTest, AlternateDirectoryTest, AnalysisAfterCoreReloadTest, TestReplicationHandlerBackup, AutoCommitTest, TestSweetSpotSimilarityFactory, TestJoin, TestClassNameShortening, TestBinaryField, TestRecoveryHdfs, BlockCacheTest, SystemInfoHandlerTest, DefaultValueUpdateProcessorTest, FullSolrCloudDistribCmdsTest, DebugComponentTest, SchemaVersionSpecificBehaviorTest, TestExtendedDismaxParser, SolrCoreTest, TestFileDictionaryLookup, StatsComponentTest, TestLRUCache, TestManagedResourceStorage, SynonymTokenizerTest, TestHighFrequencyDictionaryFactory, TestFastLRUCache, TestReRankQParserPlugin, LegacyHTMLStripCharFilterTest, UpdateRequestProcessorFactoryTest, HdfsUnloadDistributedZkTest, SpatialFilterTest, TestUpdate, HardAutoCommitTest, MultiTermTest, TestDistributedGrouping, HdfsWriteToMultipleCollectionsTest, ChaosMonkeySafeLeaderTest, RemoteQueryErrorTest, TestStandardQParsers, AsyncMigrateRouteKeyTest, DistributedFacetPivotLargeTest, CurrencyFieldXmlFileTest, TestIndexingPerformance, BinaryUpdateRequestHandlerTest, FileBasedSpellCheckerTest, TestCSVLoader, TestSolrQueryParser, HdfsDirectoryTest, TestShortCircuitedRequests, TestCharFilters, SuggesterFSTTest, TestOmitPositions, CacheHeaderTest, TestManagedResource, TestFieldTypeCollectionResource, TestNonNRTOpen, TestMaxScoreQueryParser, CopyFieldTest, TestDynamicFieldCollectionResource, IndexSchemaRuntimeFieldTest, DistanceFunctionTest, ResponseHeaderTest, TestElisionMultitermQuery, DistributedFacetPivotSmallTest, SSLMigrationTest, TestQueryTypes, TestPostingsSolrHighlighter, NumericFieldsTest, TestLMDirichletSimilarityFactory, BasicDistributedZk2Test, OpenCloseCoreStressTest, OverseerTest]
[junit4] Completed on J1 in 100.14s, 9 tests, 1 error <<< FAILURES!
[...truncated 682 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/build.xml:474: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/build.xml:454: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/solr/common-build.xml:494: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/lucene/common-build.xml:1350: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.10-Linux/lucene/common-build.xml:970: There were test failures: 421 suites, 1742 tests, 1 error, 38 ignored (17 assumptions)
Total time: 95 minutes 42 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.7.0_67 -XX:-UseCompressedOops -XX:+UseParallelGC (asserts: true)
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any