You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/02/08 07:40:11 UTC
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.6.0_45) - Build # 9296
- Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9296/
Java: 32bit/jdk1.6.0_45 -server -XX:+UseConcMarkSweepGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
at __randomizedtesting.SeedInfo.seed([C5556E5019163D35:C15DE1A30BB3D214]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:289)
at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:153)
at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:584)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:662)
Build Log:
[...truncated 9497 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> 191871 T1213 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-OverseerTest-1391839906497
[junit4] 2> 191871 T1213 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 191874 T1213 oas.SolrTestCaseJ4.setUp ###Starting testDoubleAssignment
[junit4] 2> 191874 T1213 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 191875 T1214 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 191975 T1213 oasc.ZkTestServer.run start zk server on port:53518
[junit4] 2> 191976 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 191979 T1220 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1697882 name:ZooKeeperConnection Watcher:127.0.0.1:53518/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 191979 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 191980 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 191981 T1222 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@306c48 name:ZooKeeperConnection Watcher:127.0.0.1:53518 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 191981 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 191984 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 191986 T1224 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c9ebbb name:ZooKeeperConnection Watcher:127.0.0.1:53518 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 191986 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 191986 T1213 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 191990 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 191993 T1213 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 191996 T1213 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 191999 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 192003 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 192005 T1226 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@aa00e4 name:ZooKeeperConnection Watcher:127.0.0.1:53518/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 192005 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 192007 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 192013 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 192020 T1220 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 192020 T1226 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 192021 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 192023 T1228 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a4d4e9 name:ZooKeeperConnection Watcher:127.0.0.1:53518/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 192023 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 192024 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 192028 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 192036 T1213 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:53518_solr
[junit4] 2> 192036 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 192040 T1213 oasc.Overseer.start Overseer (id=91215620112580612-127.0.0.1:53518_solr-n_0000000000) starting
[junit4] 2> 192043 T1213 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 192048 T1213 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 192054 T1213 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 192058 T1230 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 192060 T1230 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 192061 T1229 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 192065 T1229 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 192066 T1229 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 192066 T1229 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 192066 T1229 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 192070 T1228 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 192071 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 192071 T1220 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 192563 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 192582 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 192590 T1228 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 192591 T1213 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 192594 T1229 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 192599 T1220 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 192600 T1226 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 192601 T1228 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 192602 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 192607 T1232 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3ffcbe name:ZooKeeperConnection Watcher:127.0.0.1:53518/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 192607 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 192609 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 192613 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 192618 T1220 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 192618 T1232 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 192625 T1228 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 192625 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 192628 T1229 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 192628 T1229 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 192634 T1228 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 192639 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 192647 T1228 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 192657 T1228 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 192762 T1220 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 192763 T1232 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 192767 T1213 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 192775 T1220 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 192775 T1232 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 192776 T1215 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x144101f8ac70005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4] 2>
[junit4] 2> 192783 T1213 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:53518 53518
[junit4] 2> 192977 T1213 oas.SolrTestCaseJ4.tearDown ###Ending testDoubleAssignment
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 192985 T1213 oas.SolrTestCaseJ4.setUp ###Starting testPlaceholders
[junit4] 2> 192986 T1213 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 192986 T1233 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 193087 T1213 oasc.ZkTestServer.run start zk server on port:56645
[junit4] 2> 193088 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 193090 T1239 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a439c5 name:ZooKeeperConnection Watcher:127.0.0.1:56645/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 193090 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 193091 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 193093 T1241 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12f5fb6 name:ZooKeeperConnection Watcher:127.0.0.1:56645 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 193093 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 193125 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 193127 T1243 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@265039 name:ZooKeeperConnection Watcher:127.0.0.1:56645 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 193127 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 193127 T1213 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 193132 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 193135 T1213 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 193140 T1213 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 193144 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 193150 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 193151 T1245 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@30e58e name:ZooKeeperConnection Watcher:127.0.0.1:56645/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 193151 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 193153 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 193158 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 193162 T1239 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 193163 T1245 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 193164 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 193165 T1247 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d6d6d4 name:ZooKeeperConnection Watcher:127.0.0.1:56645/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 193165 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 193166 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 193170 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 193177 T1213 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56645_solr
[junit4] 2> 193178 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 193187 T1213 oasc.Overseer.start Overseer (id=91215620185653252-127.0.0.1:56645_solr-n_0000000000) starting
[junit4] 2> 193190 T1213 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 193195 T1213 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 193201 T1213 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 193207 T1249 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 193209 T1249 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 193210 T1248 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 193216 T1248 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 193217 T1248 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=12 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"12",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 193217 T1248 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3, shard4, shard5, shard6, shard7, shard8, shard9, shard10, shard11, shard12]
[junit4] 2> 193218 T1248 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 193221 T1247 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 193223 T1239 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 193223 T1245 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 193712 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 193728 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 193737 T1247 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 193738 T1213 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 193744 T1239 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 193745 T1245 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 193747 T1234 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x144101f8f220000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4] 2>
[junit4] 2> 193748 T1213 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:56645 56645
[junit4] 2> 193977 T1213 oas.SolrTestCaseJ4.tearDown ###Ending testPlaceholders
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 193984 T1213 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignment
[junit4] 2> 193984 T1213 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 193985 T1250 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 194085 T1213 oasc.ZkTestServer.run start zk server on port:36789
[junit4] 2> 194086 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 194088 T1256 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@386ac2 name:ZooKeeperConnection Watcher:127.0.0.1:36789 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 194089 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 194123 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 194125 T1258 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d2c22 name:ZooKeeperConnection Watcher:127.0.0.1:36789 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 194125 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 194125 T1213 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 194137 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 194138 T1260 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@af9150 name:ZooKeeperConnection Watcher:127.0.0.1:36789/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 194138 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 194139 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 194144 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 194145 T1262 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63c7dd name:ZooKeeperConnection Watcher:127.0.0.1:36789/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 194146 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 194147 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 194150 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 194157 T1213 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:36789_solr
[junit4] 2> 194158 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 194162 T1213 oasc.Overseer.start Overseer (id=91215620251123715-127.0.0.1:36789_solr-n_0000000000) starting
[junit4] 2> 194167 T1213 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 194173 T1213 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 194179 T1213 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 194182 T1264 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 194183 T1213 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 194185 T1264 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 194186 T1263 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 194190 T1213 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 194193 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 194198 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 194200 T1266 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b15f85 name:ZooKeeperConnection Watcher:127.0.0.1:36789/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 194200 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 194202 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 194206 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1
[junit4] 2> 194211 T1266 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 194212 T1260 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 194214 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 194217 T1263 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 194218 T1263 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 194218 T1263 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 194218 T1263 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 194222 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 194223 T1266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 194224 T1260 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 194715 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 194730 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 194739 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 194743 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 194744 T1263 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 194749 T1263 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 194750 T1263 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 194750 T1263 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 194753 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 194859 T1266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 194859 T1260 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 195243 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 195256 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 195267 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195267 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195267 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195270 T1263 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 195273 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195277 T1263 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 195277 T1263 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 195278 T1263 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 195282 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195385 T1260 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 195385 T1266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 195773 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 195786 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 195796 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195797 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195797 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195801 T1263 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 195802 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195813 T1263 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node4",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 195813 T1263 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 195813 T1263 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 195816 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 195920 T1266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 195920 T1260 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 196301 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 196313 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 196314 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 196314 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 196316 T1263 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 196318 T1263 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core5",
[junit4] 2> "core_node_name":"node5",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 196318 T1263 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 196318 T1263 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 196323 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 196428 T1266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 196428 T1260 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 196814 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 196832 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 196832 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 196832 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 196834 T1263 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 196835 T1263 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core6",
[junit4] 2> "core_node_name":"node6",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 196836 T1263 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 196836 T1263 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 196839 T1262 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 196942 T1266 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 196943 T1260 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 197332 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 197343 T1266 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 197346 T1213 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:36789 36789
[junit4] 2> 197451 T1213 oas.SolrTestCaseJ4.tearDown ###Ending testShardAssignment
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 197458 T1213 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignmentBigger
[junit4] 2> 197459 T1213 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 197460 T1267 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 197560 T1213 oasc.ZkTestServer.run start zk server on port:56311
[junit4] 2> 197561 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197683 T1273 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bc4c37 name:ZooKeeperConnection Watcher:127.0.0.1:56311 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197684 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197724 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197726 T1275 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@196d484 name:ZooKeeperConnection Watcher:127.0.0.1:56311 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197726 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197727 T1213 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 197730 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197732 T1277 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d003b name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197732 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197733 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 197735 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197737 T1279 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a21149 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197737 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197738 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 197741 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 197752 T1213 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56311_solr
[junit4] 2> 197752 T1213 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 197756 T1213 oasc.Overseer.start Overseer (id=91215620478926851-127.0.0.1:56311_solr-n_0000000000) starting
[junit4] 2> 197759 T1213 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 197765 T1213 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 197770 T1213 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 197773 T1281 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 197774 T1213 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 197775 T1281 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
[junit4] 2> 197776 T1280 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 197780 T1213 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 197784 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197788 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197790 T1283 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@874c04 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197790 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197792 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197795 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node0
[junit4] 2> 197800 T1283 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 197800 T1277 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 197801 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197802 T1285 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@103a2d8 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197803 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197804 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197808 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 197819 T1285 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 197820 T1277 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 197820 T1283 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 197820 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197822 T1287 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d12717 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197822 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197824 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197828 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 197833 T1285 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 197833 T1277 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 197834 T1283 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 197834 T1287 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 197835 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197836 T1289 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a799c1 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197837 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197838 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197842 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node3
[junit4] 2> 197849 T1287 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 197849 T1283 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 197850 T1289 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 197851 T1277 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 197851 T1285 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 197851 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197852 T1291 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@67ebee name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197853 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197854 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197858 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node4
[junit4] 2> 197862 T1285 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 197864 T1287 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 197864 T1289 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 197865 T1283 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 197865 T1291 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 197866 T1277 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 197867 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197868 T1293 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8b05ad name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197868 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197870 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197876 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node5
[junit4] 2> 197883 T1287 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 197883 T1283 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 197883 T1289 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 197884 T1285 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 197884 T1293 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 197884 T1291 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 197885 T1277 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4] 2> 197885 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197893 T1295 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e04525 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197893 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197895 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197899 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node6
[junit4] 2> 197905 T1287 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 197906 T1291 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 197907 T1283 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 197910 T1289 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 197910 T1293 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 197910 T1277 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 197910 T1285 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 197911 T1295 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4] 2> 197912 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197913 T1297 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c53ff name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197913 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197915 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197918 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node7
[junit4] 2> 197925 T1277 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197925 T1291 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197925 T1297 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197926 T1283 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197926 T1289 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197925 T1295 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197926 T1293 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197927 T1287 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197927 T1285 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4] 2> 197928 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197929 T1299 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@db33bc name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197929 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197931 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197934 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node8
[junit4] 2> 197942 T1285 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197943 T1287 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197943 T1299 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197944 T1293 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197944 T1295 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197944 T1283 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197943 T1291 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197944 T1289 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197945 T1277 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197946 T1297 oascc.ZkStateReader$3.process Updating live nodes... (9)
[junit4] 2> 197946 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197948 T1301 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@432225 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197948 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197950 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197953 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node9
[junit4] 2> 197961 T1287 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197961 T1295 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197961 T1283 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197962 T1289 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197962 T1277 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197962 T1293 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197962 T1297 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197962 T1299 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197963 T1301 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197963 T1291 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197964 T1285 oascc.ZkStateReader$3.process Updating live nodes... (10)
[junit4] 2> 197964 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197965 T1303 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@179ad95 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197966 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197967 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197971 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node10
[junit4] 2> 197980 T1287 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197980 T1293 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197981 T1285 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197981 T1299 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197981 T1295 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197981 T1283 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197981 T1289 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197982 T1277 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197981 T1301 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197982 T1297 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197983 T1291 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197983 T1303 oascc.ZkStateReader$3.process Updating live nodes... (11)
[junit4] 2> 197984 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 197985 T1305 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1dc68a2 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 197985 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 197987 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 197994 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node11
[junit4] 2> 198003 T1287 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198004 T1291 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198004 T1289 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198004 T1283 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198005 T1277 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198005 T1301 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198005 T1285 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198005 T1299 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198005 T1295 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198005 T1303 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198006 T1305 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198007 T1293 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198008 T1297 oascc.ZkStateReader$3.process Updating live nodes... (12)
[junit4] 2> 198008 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198009 T1307 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7caa0f name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198009 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198011 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198014 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node12
[junit4] 2> 198024 T1287 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198025 T1303 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198026 T1299 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198028 T1297 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198028 T1285 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198029 T1291 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198029 T1289 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198029 T1277 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198029 T1305 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198030 T1283 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198030 T1301 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198030 T1307 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198031 T1295 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198031 T1293 oascc.ZkStateReader$3.process Updating live nodes... (13)
[junit4] 2> 198032 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198033 T1309 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d89d8c name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198033 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198036 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198039 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node13
[junit4] 2> 198048 T1307 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198049 T1287 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198050 T1297 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198050 T1305 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198051 T1295 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198051 T1301 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198050 T1283 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198051 T1299 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198051 T1285 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198051 T1289 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198051 T1303 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198051 T1291 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198052 T1293 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198052 T1277 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198053 T1309 oascc.ZkStateReader$3.process Updating live nodes... (14)
[junit4] 2> 198054 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198055 T1311 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15a39a4 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198056 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198057 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198061 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node14
[junit4] 2> 198069 T1291 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198071 T1309 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198071 T1301 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198071 T1295 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198072 T1297 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198072 T1277 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198072 T1293 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198072 T1305 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198073 T1289 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198073 T1285 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198073 T1283 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198073 T1311 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198073 T1307 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198074 T1303 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198073 T1299 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198082 T1287 oascc.ZkStateReader$3.process Updating live nodes... (15)
[junit4] 2> 198083 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198084 T1313 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19de1ba name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198084 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198086 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198089 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node15
[junit4] 2> 198113 T1295 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198114 T1287 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198115 T1283 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198115 T1297 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198115 T1277 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198115 T1293 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198121 T1301 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198115 T1291 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198122 T1299 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198122 T1303 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198121 T1289 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198122 T1313 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198122 T1285 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198128 T1305 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198128 T1307 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198129 T1309 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198129 T1311 oascc.ZkStateReader$3.process Updating live nodes... (16)
[junit4] 2> 198129 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198131 T1315 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@47ab39 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198131 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198132 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198135 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node16
[junit4] 2> 198146 T1287 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198146 T1277 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198146 T1295 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198146 T1303 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198146 T1293 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198147 T1311 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198148 T1301 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198148 T1299 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198148 T1285 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198148 T1307 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198148 T1309 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198149 T1297 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198149 T1289 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198149 T1305 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198150 T1313 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198150 T1315 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198151 T1291 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198151 T1283 oascc.ZkStateReader$3.process Updating live nodes... (17)
[junit4] 2> 198151 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198152 T1317 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b860b7 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198153 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198154 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198158 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node17
[junit4] 2> 198165 T1291 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198172 T1287 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198173 T1285 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198173 T1277 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198173 T1305 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198173 T1301 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198174 T1293 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198173 T1283 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198174 T1317 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198174 T1299 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198175 T1303 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198175 T1307 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198175 T1313 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198175 T1309 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198175 T1295 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198176 T1315 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198176 T1289 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198177 T1311 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198177 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198177 T1297 oascc.ZkStateReader$3.process Updating live nodes... (18)
[junit4] 2> 198178 T1319 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d55c36 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198179 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198180 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198183 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node18
[junit4] 2> 198193 T1305 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198195 T1303 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198196 T1287 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198196 T1277 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198197 T1307 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198198 T1319 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198198 T1299 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198198 T1301 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198198 T1315 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198199 T1309 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198199 T1317 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198199 T1289 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198198 T1285 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198198 T1283 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198200 T1293 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198199 T1297 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198199 T1313 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198199 T1295 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198201 T1311 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198202 T1291 oascc.ZkStateReader$3.process Updating live nodes... (19)
[junit4] 2> 198202 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198204 T1321 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19a01f9 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198204 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198206 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198211 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node19
[junit4] 2> 198220 T1287 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198220 T1295 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198221 T1317 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198223 T1291 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198224 T1301 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198225 T1283 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198225 T1277 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198226 T1303 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198227 T1305 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198227 T1307 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198227 T1289 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198227 T1285 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198227 T1297 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198228 T1299 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198228 T1309 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198228 T1313 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198229 T1293 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198230 T1321 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198230 T1315 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198230 T1311 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198231 T1319 oascc.ZkStateReader$3.process Updating live nodes... (20)
[junit4] 2> 198232 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198233 T1323 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@dd36f3 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198233 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198235 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198238 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node20
[junit4] 2> 198249 T1319 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198250 T1301 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198251 T1313 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198252 T1287 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198252 T1291 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198253 T1285 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198253 T1277 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198253 T1289 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198254 T1305 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198254 T1317 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198254 T1283 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198254 T1315 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198254 T1297 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198255 T1307 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198255 T1309 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198254 T1303 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198254 T1299 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198255 T1323 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198255 T1321 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198257 T1295 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198257 T1293 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198256 T1311 oascc.ZkStateReader$3.process Updating live nodes... (21)
[junit4] 2> 198263 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198264 T1325 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5fbf68 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198265 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198266 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198269 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node21
[junit4] 2> 198282 T1287 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198283 T1277 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198284 T1289 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198284 T1309 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198284 T1285 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198284 T1291 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198285 T1319 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198285 T1293 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198285 T1317 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198285 T1299 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198285 T1305 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198286 T1283 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198286 T1301 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198286 T1323 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198286 T1313 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198287 T1295 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198287 T1321 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198287 T1307 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198286 T1303 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198286 T1315 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198287 T1325 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198289 T1311 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198289 T1297 oascc.ZkStateReader$3.process Updating live nodes... (22)
[junit4] 2> 198290 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198291 T1327 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8aa935 name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198291 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198293 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198298 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node22
[junit4] 2> 198311 T1319 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198311 T1315 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198319 T1287 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198319 T1277 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198320 T1323 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198320 T1285 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198320 T1301 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198320 T1317 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198320 T1291 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198321 T1293 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198322 T1299 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198322 T1309 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198322 T1303 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198322 T1283 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198322 T1305 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198323 T1307 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198322 T1289 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198323 T1295 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198323 T1321 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198324 T1313 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198323 T1325 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198325 T1327 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198326 T1311 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198326 T1297 oascc.ZkStateReader$3.process Updating live nodes... (23)
[junit4] 2> 198326 T1213 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198328 T1329 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13f333c name:ZooKeeperConnection Watcher:127.0.0.1:56311/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198328 T1213 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198329 T1213 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 198336 T1213 oascc.SolrZkClient.makePath makePath: /live_nodes/node23
[junit4] 2> 198351 T1287 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198352 T1307 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198352 T1303 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198353 T1311 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198353 T1317 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198353 T1309 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198354 T1277 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198354 T1289 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198354 T1295 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198354 T1321 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198354 T1297 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198355 T1315 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198355 T1325 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198355 T1291 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198355 T1293 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198356 T1319 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198356 T1327 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198356 T1323 oascc.ZkStateReader$3.process Updating live nodes... (24)
[junit4] 2> 198356 T1283 oascc.ZkStateReader$3.process Updating live nodes... (24)
[...truncated too long message...]
:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 230246 T1771 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 230246 T1213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 230250 T1772 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 230273 T1765 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 230275 T1765 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4] 2>
[junit4] 2> 230277 T1759 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 230281 T1213 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:48107 48107
[junit4] 2> 230437 T1213 oas.SolrTestCaseJ4.tearDown ###Ending testOverseerFailure
[junit4] 2> 230438 T1772 oasc.SolrException.log ERROR Cleaning up collection in zk was interrupted:collection1:java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:119)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:87)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:171)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:655)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.removeCore(Overseer.java:875)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:226)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4] 2>
[junit4] 2> 230438 T1772 oasc.Overseer$ClusterStateUpdater.run WARN Solr cannot talk to ZK, exiting Overseer main queue loop org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue-work/qn-
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:240)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:237)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:237)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:284)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:271)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:188)
[junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4] 2>
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerFailure -Dtests.seed=C5556E5019163D35 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en_NZ -Dtests.timezone=Asia/Damascus -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 3.21s J0 | OverseerTest.testOverseerFailure <<<
[junit4] > Throwable #1: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
[junit4] > at __randomizedtesting.SeedInfo.seed([C5556E5019163D35:C15DE1A30BB3D214]:0)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] > at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
[junit4] > at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
[junit4] > at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
[junit4] > at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:289)
[junit4] > at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:153)
[junit4] > at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:584)
[junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4] 2> 230457 T1213 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 230457 T1213 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 231776 T1765 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 233458 T1213 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: test params are: codec=Lucene46: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=en_NZ, timezone=Asia/Damascus
[junit4] 2> NOTE: Linux 3.8.0-35-generic i386/Sun Microsystems Inc. 1.6.0_45 (32-bit)/cpus=8,threads=1,free=33230816,total=99135488
[junit4] 2> NOTE: All tests run in this JVM: [FieldMutatingUpdateProcessorTest, DocumentBuilderTest, TriLevelCompositeIdRoutingTest, TestPostingsSolrHighlighter, TestSolrDeletionPolicy1, ZkCLITest, DirectUpdateHandlerOptimizeTest, ShardRoutingTest, LegacyHTMLStripCharFilterTest, SampleTest, RegexBoostProcessorTest, BlockCacheTest, QueryEqualityTest, TestCoreContainer, ScriptEngineTest, MBeansHandlerTest, OverseerRolesTest, OverseerTest]
[junit4] Completed on J0 in 41.61s, 8 tests, 1 error <<< FAILURES!
[...truncated 850 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:459: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:439: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1308: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:941: There were test failures: 358 suites, 1554 tests, 1 error, 36 ignored (8 assumptions)
Total time: 59 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.6.0_45 -server -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure