You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/08/13 17:01:35 UTC
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.7.0_65) - Build # 10896
- Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/10896/
Java: 32bit/jdk1.7.0_65 -client -XX:+UseParallelGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
Stack Trace:
org.apache.solr.common.SolrException: Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
at __randomizedtesting.SeedInfo.seed([18BE4D4FEE2A84CF:1CB6C2BCFC8F6BEE]:0)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:144)
at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:155)
at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:660)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:40)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:131)
... 48 more
Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at org.apache.solr.common.cloud.SolrZkClient$11.execute(SolrZkClient.java:457)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:454)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:411)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:398)
at org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:136)
at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:34)
... 49 more
Build Log:
[...truncated 11449 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solr.cloud.OverseerTest-18BE4D4FEE2A84CF-001/init-core-data-001
[junit4] 2> 508208 T1560 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
[junit4] 2> 508209 T1560 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 508209 T1560 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 508212 T1560 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignment
[junit4] 2> 508212 T1560 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 508213 T1561 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 508313 T1560 oasc.ZkTestServer.run start zk server on port:60357
[junit4] 2> 508314 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 508316 T1567 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12a1bfd name:ZooKeeperConnection Watcher:127.0.0.1:60357 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 508316 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 508318 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 508318 T1569 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@66d841 name:ZooKeeperConnection Watcher:127.0.0.1:60357 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 508319 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 508319 T1560 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 508321 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 508322 T1571 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@128350d name:ZooKeeperConnection Watcher:127.0.0.1:60357/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 508322 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 508323 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 508324 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 508325 T1573 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b5a367 name:ZooKeeperConnection Watcher:127.0.0.1:60357/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 508325 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 508326 T1560 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 508327 T1560 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 508329 T1560 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:60357_solr
[junit4] 2> 508330 T1560 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 508331 T1560 oasc.Overseer.start Overseer (id=92270738994954243-127.0.0.1:60357_solr-n_0000000000) starting
[junit4] 2> 508333 T1560 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 508335 T1560 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 508338 T1560 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 508340 T1560 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 508342 T1560 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 508344 T1560 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 508346 T1575 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 508347 T1560 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 508347 T1574 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 508348 T1560 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 508349 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 508351 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 508352 T1577 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ca600f name:ZooKeeperConnection Watcher:127.0.0.1:60357/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 508352 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 508353 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 508354 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1
[junit4] 2> 508358 T1579 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 508358 T1578 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 508360 T1580 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 508362 T1574 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 508363 T1574 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 508363 T1574 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 508364 T1574 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 508368 T1579 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 508368 T1578 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 508860 T1560 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 508867 T1560 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 508871 T1580 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 508872 T1574 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 508874 T1574 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 508874 T1574 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 508875 T1574 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 508876 T1580 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 508877 T1578 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 508877 T1579 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 509372 T1560 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 509376 T1560 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 509378 T1580 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 509379 T1574 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 509381 T1574 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 509381 T1574 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 509381 T1574 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 509382 T1580 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 509383 T1578 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 509383 T1579 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 509879 T1560 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 509889 T1560 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 509894 T1580 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 509895 T1574 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 509898 T1574 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node4",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 509898 T1574 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 509898 T1574 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 509899 T1580 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 509900 T1578 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 509900 T1579 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 510396 T1560 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 510404 T1580 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 510405 T1574 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 510405 T1574 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core5",
[junit4] 2> "core_node_name":"node5",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 510406 T1574 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 510406 T1574 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 510508 T1578 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 510508 T1579 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 510904 T1560 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 510914 T1580 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 510916 T1574 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 510916 T1574 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core6",
[junit4] 2> "core_node_name":"node6",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 510917 T1574 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 510917 T1574 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 511020 T1578 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 511020 T1579 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 511414 T1560 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 511419 T1579 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 511420 T1581 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 511420 T1577 oaz.ClientCnxn$EventThread.processEvent ERROR Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@10359e6 rejected from java.util.concurrent.ThreadPoolExecutor@1033183[Shutting down, pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 8]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
[junit4] 2> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:201)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 511422 T1577 oaz.ClientCnxn$EventThread.processEvent ERROR Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@c92656 rejected from java.util.concurrent.ThreadPoolExecutor@1033183[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
[junit4] 2> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:201)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 511422 T1560 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:60357 60357
[junit4] 2> 511999 T1561 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:60357 60357
[junit4] 2> 512000 T1560 oas.SolrTestCaseJ4.tearDown ###Ending testShardAssignment
[junit4] 2> 512001 T1574 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:60357_solr
[junit4] 2> 512002 T1582 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:316)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:288)
[junit4] 2>
[junit4] 2> 512005 T1560 oas.SolrTestCaseJ4.setUp ###Starting testDoubleAssignment
[junit4] 2> 512005 T1560 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 512006 T1583 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 512106 T1560 oasc.ZkTestServer.run start zk server on port:49321
[junit4] 2> 512107 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 512110 T1589 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e27125 name:ZooKeeperConnection Watcher:127.0.0.1:49321/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 512110 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 512112 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 512113 T1591 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@25fd41 name:ZooKeeperConnection Watcher:127.0.0.1:49321 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 512114 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 512117 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 512119 T1593 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@125cfed name:ZooKeeperConnection Watcher:127.0.0.1:49321 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 512119 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 512120 T1560 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 512125 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 512128 T1560 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 512131 T1560 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 512134 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 512138 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 512140 T1595 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b2fed9 name:ZooKeeperConnection Watcher:127.0.0.1:49321/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 512140 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 512142 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 512146 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 512151 T1597 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 512151 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 512151 T1596 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 512153 T1599 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@60c158 name:ZooKeeperConnection Watcher:127.0.0.1:49321/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 512153 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 512154 T1560 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 512157 T1560 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 512162 T1560 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:49321_solr
[junit4] 2> 512162 T1560 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 512165 T1560 oasc.Overseer.start Overseer (id=92270739243532292-127.0.0.1:49321_solr-n_0000000000) starting
[junit4] 2> 512167 T1560 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 512172 T1560 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 512175 T1560 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 512179 T1560 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 512183 T1560 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 512187 T1560 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 512194 T1601 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 512195 T1600 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 512197 T1600 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 512198 T1600 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 512199 T1600 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 512199 T1600 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 512202 T1602 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 512203 T1596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 512203 T1597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 512696 T1560 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 512701 T1560 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 512704 T1602 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 512704 T1560 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 512705 T1600 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 512706 T1596 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 512706 T1597 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 512707 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 512708 T1604 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@139cd58 name:ZooKeeperConnection Watcher:127.0.0.1:49321/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 512708 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 512709 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 512710 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 512712 T1597 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 512712 T1605 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 512713 T1602 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 512713 T1560 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 512714 T1600 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 512714 T1600 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 512718 T1560 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 512720 T1602 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 512823 T1605 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 512823 T1560 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 512823 T1597 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 512825 T1597 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 512825 T1605 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 512826 T1560 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:49321 49321
[junit4] 2> 513553 T1583 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:49321 49321
[junit4] 2> 513556 T1560 oas.SolrTestCaseJ4.tearDown ###Ending testDoubleAssignment
[junit4] 2> 513557 T1600 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:49321_solr
[junit4] 2> 513559 T1606 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:316)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:288)
[junit4] 2>
[junit4] 2> 513565 T1560 oas.SolrTestCaseJ4.setUp ###Starting testStateChange
[junit4] 2> 513566 T1560 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 513567 T1607 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 513667 T1560 oasc.ZkTestServer.run start zk server on port:58067
[junit4] 2> 513668 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 513671 T1613 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1580c77 name:ZooKeeperConnection Watcher:127.0.0.1:58067/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 513671 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 513673 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 513674 T1615 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@198e297 name:ZooKeeperConnection Watcher:127.0.0.1:58067 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 513675 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 513677 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 513679 T1617 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f5c484 name:ZooKeeperConnection Watcher:127.0.0.1:58067 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 513679 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 513680 T1560 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 513684 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 513687 T1560 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 513690 T1560 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 513692 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 513696 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 513697 T1619 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1dfeec2 name:ZooKeeperConnection Watcher:127.0.0.1:58067/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 513698 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 513699 T1560 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 513702 T1560 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 513705 T1560 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:58067_solr
[junit4] 2> 513706 T1560 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 513708 T1560 oasc.Overseer.start Overseer (id=92270739345899523-127.0.0.1:58067_solr-n_0000000000) starting
[junit4] 2> 513711 T1560 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 513714 T1560 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 513717 T1560 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 513720 T1560 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 513723 T1560 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 513727 T1560 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 513732 T1621 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 513733 T1620 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 513734 T1560 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 513736 T1620 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 513737 T1620 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 513737 T1620 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 513740 T1622 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 513741 T1623 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 513786 T1560 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 513788 T1622 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 513791 T1620 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 513792 T1620 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"active"}
[junit4] 2> 513792 T1620 oasc.Overseer$ClusterStateUpdater.updateState node=core_node1 is already registered
[junit4] 2> 513793 T1620 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 513897 T1623 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 513942 T1560 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:58067 58067
[junit4] 2> 514514 T1607 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:58067 58067
[junit4] 2> 514516 T1560 oas.SolrTestCaseJ4.tearDown ###Ending testStateChange
[junit4] 2> 514517 T1620 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:58067_solr
[junit4] 2> 514519 T1624 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:316)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:288)
[junit4] 2>
[junit4] 2> 514525 T1560 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignmentBigger
[junit4] 2> 514526 T1560 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 514527 T1625 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 514627 T1560 oasc.ZkTestServer.run start zk server on port:40506
[junit4] 2> 514628 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514631 T1631 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e957b9 name:ZooKeeperConnection Watcher:127.0.0.1:40506 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514631 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514634 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514636 T1633 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15cce4e name:ZooKeeperConnection Watcher:127.0.0.1:40506 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514637 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514637 T1560 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 514642 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514644 T1635 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@310b8d name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514645 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514645 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 514648 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514650 T1637 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1edce8a name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514650 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514652 T1560 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 514655 T1560 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 514661 T1560 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:40506_solr
[junit4] 2> 514662 T1560 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 514665 T1560 oasc.Overseer.start Overseer (id=92270739408814083-127.0.0.1:40506_solr-n_0000000000) starting
[junit4] 2> 514668 T1560 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 514673 T1560 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 514677 T1560 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 514681 T1560 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 514685 T1560 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 514690 T1560 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 514697 T1639 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 514698 T1560 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 514699 T1638 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 514701 T1560 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 514703 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514706 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514708 T1641 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ca5b77 name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514708 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514710 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514713 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node0
[junit4] 2> 514716 T1643 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 514717 T1642 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 514717 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514718 T1645 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12728d5 name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514719 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514720 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514723 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 514732 T1642 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 514732 T1643 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 514732 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514732 T1646 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 514734 T1648 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1967ed7 name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514734 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514735 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514738 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 514741 T1643 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 514741 T1642 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 514741 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514741 T1646 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 514741 T1649 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 514743 T1651 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15d39b5 name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514743 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514744 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514747 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node3
[junit4] 2> 514750 T1646 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 514750 T1642 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 514751 T1649 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 514750 T1643 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 514751 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514751 T1652 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 514752 T1654 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1932095 name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514753 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514754 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514756 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node4
[junit4] 2> 514760 T1646 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 514761 T1649 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 514761 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514761 T1652 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 514761 T1655 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 514763 T1657 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@196c8c name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514760 T1642 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 514760 T1643 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 514763 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514766 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514769 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node5
[junit4] 2> 514772 T1642 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 514773 T1643 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 514773 T1655 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 514774 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514773 T1646 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 514773 T1649 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 514773 T1658 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 514773 T1652 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 514775 T1660 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a6a693 name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514776 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514778 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514782 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node6
[junit4] 2> 514786 T1646 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 514786 T1655 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 514787 T1649 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 514787 T1642 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 514788 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514787 T1652 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 514787 T1661 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 514787 T1658 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 514787 T1643 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 514789 T1663 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d1ea63 name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514790 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514792 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514795 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node7
[junit4] 2> 514798 T1649 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 514798 T1652 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 514799 T1643 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 514798 T1655 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 514798 T1646 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 514799 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514799 T1664 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 514799 T1658 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 514799 T1661 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 514799 T1642 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 514801 T1666 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2ea903 name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514802 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514804 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514807 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node8
[junit4] 2> 514811 T1655 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 514811 T1646 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 514811 T1649 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 514812 T1652 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 514812 T1642 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 514812 T1664 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 514811 T1658 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 514811 T1661 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 514811 T1643 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 514814 T1669 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@be00f3 name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514812 T1667 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 514812 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514816 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514818 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514821 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node9
[junit4] 2> 514825 T1646 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 514826 T1661 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 514826 T1667 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 514827 T1649 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 514826 T1670 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 514826 T1658 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 514825 T1655 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 514828 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514827 T1652 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 514829 T1672 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b79c28 name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514827 T1643 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 514827 T1664 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 514826 T1642 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 514830 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514833 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514836 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node10
[junit4] 2> 514840 T1642 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 514840 T1673 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 514841 T1670 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 514841 T1667 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 514841 T1661 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 514841 T1643 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 514841 T1658 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 514841 T1646 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 514840 T1649 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 514840 T1664 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 514840 T1652 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 514844 T1675 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b75239 name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514842 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514841 T1655 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 514845 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514847 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514850 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node11
[junit4] 2> 514854 T1664 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 514855 T1673 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 514855 T1643 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 514856 T1655 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 514855 T1661 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 514857 T1676 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 514855 T1646 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 514855 T1670 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 514855 T1658 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 514858 T1678 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10adb8f name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514855 T1667 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 514855 T1642 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 514854 T1649 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 514854 T1652 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 514857 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514860 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514861 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514864 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node12
[junit4] 2> 514867 T1667 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 514867 T1655 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 514869 T1643 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 514869 T1673 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 514869 T1652 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 514870 T1679 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 514869 T1670 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 514869 T1642 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 514869 T1661 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 514869 T1664 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 514869 T1658 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 514869 T1646 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 514869 T1676 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 514869 T1649 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 514871 T1681 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d73481 name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514870 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514874 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514875 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514878 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node13
[junit4] 2> 514881 T1661 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 514881 T1655 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 514882 T1667 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 514881 T1649 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 514883 T1652 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 514883 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514884 T1682 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 514883 T1643 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 514884 T1684 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e9c180 name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514882 T1646 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 514882 T1673 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 514882 T1642 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 514882 T1670 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 514885 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514883 T1658 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 514883 T1679 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 514883 T1676 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 514887 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514883 T1664 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 514889 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node14
[junit4] 2> 514892 T1667 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 514892 T1673 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 514893 T1682 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 514892 T1649 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 514894 T1652 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 514892 T1676 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 514894 T1685 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 514894 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514894 T1643 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 514894 T1670 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 514893 T1642 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 514893 T1664 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 514893 T1679 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 514893 T1646 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 514893 T1655 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 514893 T1658 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 514893 T1661 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 514895 T1687 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15432aa name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514898 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514899 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514901 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node15
[junit4] 2> 514903 T1682 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514904 T1676 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514904 T1658 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514905 T1664 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514904 T1670 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514904 T1655 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514904 T1667 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514904 T1679 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514903 T1661 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514903 T1673 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514907 T1690 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19b7e7d name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514905 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514905 T1688 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514905 T1642 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514905 T1646 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514904 T1685 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514904 T1652 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514904 T1649 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514904 T1643 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 514908 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514911 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514913 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node16
[junit4] 2> 514916 T1667 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514916 T1643 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514916 T1646 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514916 T1682 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514917 T1664 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514917 T1679 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514916 T1655 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514918 T1691 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514917 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514917 T1670 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514917 T1642 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514917 T1658 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514917 T1676 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514917 T1688 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514917 T1673 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514916 T1649 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514916 T1652 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514916 T1685 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514916 T1661 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 514919 T1693 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a81e73 name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514922 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514923 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514924 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node17
[junit4] 2> 514927 T1643 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514927 T1667 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514928 T1679 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514927 T1649 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514927 T1646 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514927 T1652 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514929 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514928 T1670 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514928 T1694 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514928 T1661 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514928 T1688 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514928 T1676 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514928 T1682 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514928 T1658 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514928 T1664 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514928 T1642 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514928 T1691 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514928 T1685 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514927 T1673 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514927 T1655 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 514930 T1696 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fbe91c name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514932 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514933 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514935 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node18
[junit4] 2> 514937 T1667 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514937 T1649 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514938 T1643 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514938 T1682 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514938 T1694 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514938 T1655 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514937 T1646 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514937 T1652 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514937 T1670 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514939 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514938 T1685 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514938 T1642 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514938 T1697 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514938 T1691 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514938 T1673 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514938 T1661 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514938 T1688 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514938 T1658 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514938 T1676 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514938 T1664 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514938 T1679 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 514940 T1699 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b097b7 name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514942 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514943 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514944 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node19
[junit4] 2> 514946 T1673 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514946 T1670 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514947 T1697 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514948 T1652 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514946 T1658 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514946 T1685 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514946 T1676 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514946 T1642 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514948 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514948 T1700 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514948 T1643 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514948 T1646 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514948 T1655 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514947 T1664 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514947 T1649 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514947 T1667 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514947 T1688 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514947 T1694 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514947 T1661 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514947 T1691 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514947 T1682 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514947 T1679 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 514949 T1702 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cdbe1c name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514952 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514952 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514954 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node20
[junit4] 2> 514956 T1642 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514957 T1679 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514956 T1670 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514958 T1646 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514958 T1658 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514958 T1661 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514958 T1697 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514957 T1643 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514957 T1676 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514957 T1652 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514957 T1688 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514957 T1694 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514957 T1691 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514956 T1685 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514960 T1703 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514959 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514958 T1649 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514958 T1700 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514958 T1667 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514958 T1682 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514958 T1655 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514957 T1673 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514957 T1664 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 514960 T1705 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19b0cc9 name:ZooKeeperConnection Watcher:127.0.0.1:40506/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 514962 T1560 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 514963 T1560 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 514964 T1560 oascc.SolrZkClient.makePath makePath: /live_nodes/node21
[junit4] 2> 514966 T1649 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 514969 T1667 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 514970 T1664 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 514970 T1658 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 514970 T1688 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 514970 T1676 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 514971 T1700 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 514970 T1682 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 514971 T1642 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 514970 T1670 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 514971 T1661 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 514971 T1652 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 514971 T1560 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 514971 T1691 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 514971 T1655 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 514971 T1703 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 514971
[...truncated too long message...]
2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 586790 T2394 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 586791 T2390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 586792 T2394 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 586792 T2394 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 586793 T2390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 586795 T2390 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 586795 T2394 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 588234 T2386 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 588236 T1560 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 588291 T2390 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 588293 T1560 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:56086 56086
[junit4] 2> 588790 T2395 oasc.OverseerCollectionProcessor.amILeader org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor.amILeader(OverseerCollectionProcessor.java:518)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:273)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 588882 T2395 oasc.OverseerCollectionProcessor.amILeader According to ZK I (id=92270742747545645-127.0.0.1:56086_solr-n_0000000014) am no longer a leader.
[junit4] 2> 588883 T2179 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:56086 56086
[junit4] 2> 588887 T1560 oas.SolrTestCaseJ4.tearDown ###Ending testShardLeaderChange
[junit4] 2> 588888 T2394 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:56086_solr
[junit4] 2> 588889 T2396 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:316)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:288)
[junit4] 2>
[junit4] 2> 591891 T1560 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solr.cloud.OverseerTest-18BE4D4FEE2A84CF-001
[junit4] 2> NOTE: test params are: codec=Lucene49: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=cs_CZ, timezone=Asia/Riyadh87
[junit4] 2> NOTE: Linux 3.13.0-32-generic i386/Oracle Corporation 1.7.0_65 (32-bit)/cpus=8,threads=1,free=153027408,total=256114688
[junit4] 2> NOTE: All tests run in this JVM: [HighlighterTest, TestSchemaSimilarityResource, TestDocSet, TermsComponentTest, TestAddFieldRealTimeGet, ConnectionManagerTest, TestBinaryResponseWriter, BasicFunctionalityTest, SpellCheckCollatorTest, PreAnalyzedFieldTest, BJQParserTest, SignatureUpdateProcessorFactoryTest, InfoHandlerTest, TestSolrDeletionPolicy2, TestLMJelinekMercerSimilarityFactory, IndexSchemaTest, TestQuerySenderListener, NotRequiredUniqueKeyTest, TestLeaderElectionZkExpiry, TestAnalyzedSuggestions, XmlUpdateRequestHandlerTest, TestHighFrequencyDictionaryFactory, TestSolrIndexConfig, TestReload, SpellPossibilityIteratorTest, TestDocBasedVersionConstraints, TestSuggestSpellingConverter, TestTrie, DirectUpdateHandlerTest, AddBlockUpdateTest, BadComponentTest, ResponseLogComponentTest, TestDistributedMissingSort, SimpleFacetsTest, TestJmxMonitoredMap, UUIDUpdateProcessorFallbackTest, TestUniqueKeyFieldResource, FileUtilsTest, SuggesterTSTTest, TestRequestStatusCollectionAPI, ParsingFieldUpdateProcessorsTest, TestRTGBase, TestBM25SimilarityFactory, BadIndexSchemaTest, TestFoldingMultitermQuery, TestDefaultSimilarityFactory, DocValuesMultiTest, TestManagedSchema, TestCopyFieldCollectionResource, TestFastOutputStream, JSONWriterTest, ZkCLITest, TestWordDelimiterFilterFactory, MoreLikeThisHandlerTest, TestSolr4Spatial2, TestSchemaNameResource, TestSchemaResource, ZkControllerTest, SyncSliceTest, ReturnFieldsTest, TestHashPartitioner, StandardRequestHandlerTest, ExternalFileFieldSortTest, TestRangeQuery, XsltUpdateRequestHandlerTest, TestCloudManagedSchemaConcurrent, OverseerTest]
[junit4] Completed on J0 in 83.72s, 9 tests, 1 error <<< FAILURES!
[...truncated 910 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:474: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:454: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1308: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:923: There were test failures: 411 suites, 1712 tests, 1 error, 39 ignored (18 assumptions)
Total time: 98 minutes 7 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.7.0_65 -client -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any