You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/05/07 14:10:15 UTC
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_55) - Build #
10253 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10253/
Java: 64bit/jdk1.7.0_55 -XX:+UseCompressedOops -XX:+UseParallelGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
Stack Trace:
org.apache.solr.common.SolrException: Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
at __randomizedtesting.SeedInfo.seed([BD8A990A69ECBB09:B98216F97B495428]:0)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:136)
at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:134)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:155)
at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:661)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:432)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:429)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:386)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:373)
at org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:128)
at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:31)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:123)
... 47 more
Build Log:
[...truncated 10946 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.OverseerTest-BD8A990A69ECBB09-001/init-core-data-001
[junit4] 2> 286738 T1037 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
[junit4] 2> 286738 T1037 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 286738 T1037 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 286743 T1037 oas.SolrTestCaseJ4.setUp ###Starting testBadQueueItem
[junit4] 2> 286743 T1037 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 286744 T1038 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 286844 T1037 oasc.ZkTestServer.run start zk server on port:52463
[junit4] 2> 286845 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 286948 T1044 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55b75f27 name:ZooKeeperConnection Watcher:127.0.0.1:52463 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 286948 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 286951 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 286952 T1046 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@435df5c7 name:ZooKeeperConnection Watcher:127.0.0.1:52463 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 286952 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 286953 T1037 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 286955 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 286956 T1048 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@287c715d name:ZooKeeperConnection Watcher:127.0.0.1:52463/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 286956 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 286957 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 286959 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 286960 T1050 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@662945b3 name:ZooKeeperConnection Watcher:127.0.0.1:52463/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 286960 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 286961 T1037 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 286963 T1037 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 286966 T1037 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:52463_solr
[junit4] 2> 286966 T1037 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 286967 T1037 oasc.Overseer.start Overseer (id=91715241211920387-127.0.0.1:52463_solr-n_0000000000) starting
[junit4] 2> 286969 T1037 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 286971 T1037 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 286973 T1037 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 286975 T1037 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 286980 T1037 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 286983 T1037 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 286988 T1052 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 286988 T1037 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 286988 T1052 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes at 127.0.0.1:52463_solr
[junit4] 2> 286988 T1051 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 286990 T1037 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 286991 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 286993 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 286994 T1054 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f0d4e33 name:ZooKeeperConnection Watcher:127.0.0.1:52463/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 286995 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 286996 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 286998 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1
[junit4] 2> 287001 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 287002 T1051 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 287002 T1051 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 287002 T1051 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 287003 T1051 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 287004 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 287005 T1054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 287005 T1048 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 287501 T1037 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 287507 T1037 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 287511 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 287512 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 287512 T1051 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 287515 T1051 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 287515 T1051 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 287515 T1051 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 287518 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 287620 T1054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 287620 T1048 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 288012 T1037 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 288019 T1037 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 288022 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 288022 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 288022 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 288023 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 288023 T1051 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 288026 T1051 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 288026 T1051 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 288026 T1051 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 288027 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 288130 T1048 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 288130 T1054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 288523 T1037 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 288528 T1037 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 288532 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 288532 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 288532 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 288533 T1051 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 288535 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 288636 T1048 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 288636 T1054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 288684 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 288684 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 288684 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 288686 T1051 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 288686 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 288686 T1051 oasc.Overseer$ClusterStateUpdater.checkKeyExistence ERROR Skipping invalid Overseer message because it has no collection specified: {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core0",
[junit4] 2> "core_node_name":"node0",
[junit4] 2> "collection":"",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 288692 T1051 oasc.Overseer$ClusterStateUpdater.checkKeyExistence ERROR Skipping invalid Overseer message because it has no collection specified: {
[junit4] 2> "operation":"deletecore",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core0",
[junit4] 2> "core_node_name":"node0",
[junit4] 2> "collection":""}
[junit4] 2> 288704 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 288705 T1051 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 288705 T1051 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection2 with shards [shard1, shard2, shard3]
[junit4] 2> 288705 T1051 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 288708 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 288811 T1048 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 288811 T1054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 289190 T1037 oascc.SolrZkClient.makePath makePath: /collections/collection2/leader_elect/shard3/election
[junit4] 2> 289202 T1037 oascc.SolrZkClient.makePath makePath: /collections/collection2/leaders/shard3
[junit4] 2> 289208 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 289208 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 289208 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 289210 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 289210 T1051 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 289216 T1051 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 289217 T1051 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 289217 T1051 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 289221 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 289324 T1048 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 289324 T1054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 289711 T1037 oascc.SolrZkClient.makePath makePath: /collections/collection2/leader_elect/shard2/election
[junit4] 2> 289722 T1037 oascc.SolrZkClient.makePath makePath: /collections/collection2/leaders/shard2
[junit4] 2> 289730 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 289730 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 289731 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 289734 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 289734 T1051 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 289741 T1051 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 289741 T1051 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 289742 T1051 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 289746 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 289849 T1054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 289849 T1048 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 290234 T1037 oascc.SolrZkClient.makePath makePath: /collections/collection2/leader_elect/shard1/election
[junit4] 2> 290247 T1037 oascc.SolrZkClient.makePath makePath: /collections/collection2/leaders/shard1
[junit4] 2> 290254 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 290254 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 290255 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 290257 T1051 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 290262 T1050 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 290364 T1048 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 290365 T1054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 290410 T1037 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:52463 52463
[junit4] 2> 290612 T1038 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:52463 52463
[junit4] 2> 290614 T1037 oas.SolrTestCaseJ4.tearDown ###Ending testBadQueueItem
[junit4] 2> 290615 T1051 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:52463_solr
[junit4] 2> 290617 T1055 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:306)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
[junit4] 2>
[junit4] 2> 290623 T1037 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignmentBigger
[junit4] 2> 290624 T1037 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 290625 T1056 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 290725 T1037 oasc.ZkTestServer.run start zk server on port:38988
[junit4] 2> 290726 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290730 T1062 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4560be7b name:ZooKeeperConnection Watcher:127.0.0.1:38988 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290731 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290735 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290737 T1064 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18a2615a name:ZooKeeperConnection Watcher:127.0.0.1:38988 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290737 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290738 T1037 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 290743 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290745 T1066 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2373f5d2 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290746 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290746 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 290749 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290751 T1068 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@33637477 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290752 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290753 T1037 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 290756 T1037 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 290763 T1037 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:38988_solr
[junit4] 2> 290763 T1037 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 290766 T1037 oasc.Overseer.start Overseer (id=91715241466331139-127.0.0.1:38988_solr-n_0000000000) starting
[junit4] 2> 290769 T1037 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 290774 T1037 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 290779 T1037 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 290784 T1037 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 290791 T1037 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 290797 T1037 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 290806 T1070 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 290807 T1037 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 290807 T1070 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes at 127.0.0.1:38988_solr
[junit4] 2> 290807 T1069 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 290810 T1037 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 290812 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290816 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290818 T1072 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@221e5071 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290818 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290820 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290824 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node0
[junit4] 2> 290829 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290831 T1074 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@52f06abb name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290831 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290834 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290838 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 290845 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290847 T1076 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@36b9a750 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290847 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290849 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290850 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 290852 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290853 T1078 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@79ab7c81 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290854 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290855 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290856 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node3
[junit4] 2> 290858 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290859 T1080 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32db1442 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290859 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290860 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290862 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node4
[junit4] 2> 290864 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290865 T1082 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6556711b name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290865 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290866 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290867 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node5
[junit4] 2> 290871 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290872 T1084 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ff6d563 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290872 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290873 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290875 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node6
[junit4] 2> 290877 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290879 T1086 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b4cad5b name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290879 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290881 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290884 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node7
[junit4] 2> 290889 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290891 T1088 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@41538058 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290891 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290893 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290897 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node8
[junit4] 2> 290903 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290904 T1090 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7fcdf187 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290905 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290907 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290909 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node9
[junit4] 2> 290912 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290913 T1092 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32cfb0e4 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290913 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290914 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290916 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node10
[junit4] 2> 290919 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290920 T1094 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6fe8e900 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290920 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290921 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290923 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node11
[junit4] 2> 290926 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290927 T1096 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55ca8919 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290927 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290928 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290930 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node12
[junit4] 2> 290933 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290934 T1098 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3d98b59e name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290934 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290935 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290936 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node13
[junit4] 2> 290940 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290941 T1100 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4e86010f name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290941 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290942 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290944 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node14
[junit4] 2> 290947 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290949 T1102 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@225ed91b name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290949 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290950 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290952 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node15
[junit4] 2> 290955 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290956 T1104 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d9f907c name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290956 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290957 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290958 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node16
[junit4] 2> 290964 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290965 T1106 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6606445f name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290965 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290966 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290968 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node17
[junit4] 2> 290971 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290972 T1108 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45a0978d name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290973 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290974 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290975 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node18
[junit4] 2> 290979 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290980 T1110 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@37e403b3 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290980 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290981 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290982 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node19
[junit4] 2> 290987 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290988 T1112 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@69fa70e5 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290988 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290989 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290990 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node20
[junit4] 2> 290994 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 290995 T1114 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@413a8278 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 290996 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 290996 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 290998 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node21
[junit4] 2> 291002 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291003 T1116 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@763954b0 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291003 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291004 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291006 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node22
[junit4] 2> 291011 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291013 T1118 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3aa7da04 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291013 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291014 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291015 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node23
[junit4] 2> 291019 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291020 T1120 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fe4168a name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291020 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291021 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291023 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node24
[junit4] 2> 291027 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291029 T1122 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7afa12ff name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291029 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291030 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291032 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node25
[junit4] 2> 291038 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291039 T1124 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9528997 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291039 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291040 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291042 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node26
[junit4] 2> 291047 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291048 T1126 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7069fdd6 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291048 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291049 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291051 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node27
[junit4] 2> 291056 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291057 T1128 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@74d81584 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291057 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291058 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291059 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node28
[junit4] 2> 291064 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291065 T1130 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@28e50a3e name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291065 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291066 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291068 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node29
[junit4] 2> 291073 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291074 T1132 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@54e33280 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291074 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291075 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291076 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node30
[junit4] 2> 291083 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291090 T1134 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d67d32 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291090 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291091 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291092 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node31
[junit4] 2> 291098 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291099 T1136 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fc50571 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291099 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291100 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291101 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node32
[junit4] 2> 291107 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291108 T1138 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@75cee0ef name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291108 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291109 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291110 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node33
[junit4] 2> 291115 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291116 T1140 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7abcc5c name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291117 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291117 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291118 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node34
[junit4] 2> 291124 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291125 T1142 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3575d8c7 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291125 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291125 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291127 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node35
[junit4] 2> 291134 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291135 T1144 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@586801b7 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291135 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291136 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291137 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node36
[junit4] 2> 291144 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291145 T1146 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d8b803b name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291145 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291146 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291147 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node37
[junit4] 2> 291153 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291154 T1148 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a5dc951 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291154 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291155 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291156 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node38
[junit4] 2> 291162 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291163 T1150 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3b8fd4dc name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291163 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291164 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291165 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node39
[junit4] 2> 291170 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291172 T1152 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@418895d6 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291172 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291172 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291174 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node40
[junit4] 2> 291179 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291180 T1154 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7195e205 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291180 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291181 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291183 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node41
[junit4] 2> 291189 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291190 T1156 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c2e98a9 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291190 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291191 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291192 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node42
[junit4] 2> 291199 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291200 T1158 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d74bed6 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291200 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291201 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291202 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node43
[junit4] 2> 291209 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291210 T1160 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@302d7039 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291210 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291210 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291212 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node44
[junit4] 2> 291218 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291221 T1162 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@35fc600c name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291221 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291222 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291223 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node45
[junit4] 2> 291231 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291232 T1164 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@89c4ba1 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291232 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291233 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291234 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node46
[junit4] 2> 291243 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291244 T1166 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3dd7d9ac name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291244 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291244 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291246 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node47
[junit4] 2> 291252 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291253 T1168 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19e6ea47 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291253 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291254 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291255 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node48
[junit4] 2> 291262 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291264 T1170 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f3e3a68 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291264 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291264 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291266 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node49
[junit4] 2> 291274 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291275 T1172 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d16a33 name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291276 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291276 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291278 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node50
[junit4] 2> 291284 T1037 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 291285 T1174 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18a05aac name:ZooKeeperConnection Watcher:127.0.0.1:38988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 291286 T1037 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 291286 T1037 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 291287 T1037 oascc.SolrZkClient.makePath makePath: /live_nodes/node51
[junit4] 2> 291298 T1068 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 291300 T1068 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 291302 T1069 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 291304 T1069 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 291304 T1069 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 291305 T1069 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 291317 T1108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291317 T1148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291317 T1106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291317 T1158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291317 T1082 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291318 T1152 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291318 T1130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291318 T1096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291317 T1112 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291318 T1084 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291317 T1116 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291317 T1154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291319 T1118 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291317 T1120 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291319 T1072 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291319 T1122 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291320 T1146 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291320 T1128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291318 T1102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291320 T1142 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291318 T1076 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291320 T1172 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291321 T1156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291321 T1086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291318 T1162 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291318 T1090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291318 T1114 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291318 T1164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291317 T1168 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291317 T1170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291317 T1088 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291317 T1092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291317 T1078 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291321 T1144 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291321 T1174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291321 T1100 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291321 T1150 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291321 T1074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291321 T1136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291321 T1132 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291321 T1094 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291321 T1098 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291320 T1080 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291320 T1124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291320 T1160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291320 T1126 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291320 T1104 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291320 T1166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291320 T1140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291320 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291320 T1110 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291320 T1134 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291319 T1066 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 52)
[junit4] 2> 291329 T1069 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 291329 T1069 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node4",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node4",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node4/solr/"}
[junit4] 2> 291329 T1069 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4] 2> 291329 T1069 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 291331 T1068 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 291331 T1069 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node3",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node3/solr/"}
[junit4] 2> 291332 T1069 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4] 2> 291332 T1069 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 291333 T1068 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 291334 T1069 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 291334 T1069 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4] 2> 291334 T1069 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 291336 T1068 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 291337 T1069 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node5",
[junit4] 2> "core":"core5",
[junit4] 2> "core_node_name":"node5",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node5/solr/"}
[junit4] 2> 291337 T1069 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4] 2> 291337 T1069 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 291338 T1068 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 291339 T1069 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node6",
[junit4] 2> "core":"core6",
[junit4] 2> "core_node_name":"node6",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node6/solr/"}
[junit4] 2> 291339 T1069 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4] 2> 291339 T1069 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 291341 T1068 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 291342 T1069 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node8",
[junit4] 2> "core":"core8",
[junit4] 2> "core_node_name":"node8",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node8/solr/"}
[junit4] 2> 291342 T1069 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4] 2> 291342 T1069 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 291344 T1068 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 291344 T1069 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node7",
[junit4] 2> "core":"core7",
[junit4] 2> "core_node_name":"node7",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node7/solr/"}
[junit4] 2> 291344 T1069 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4] 2> 291344 T1069 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 291346 T1068 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 291347 T1069 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node10",
[junit4] 2> "core":"core10",
[junit4] 2> "core_node_name":"node10",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node10/solr/"}
[junit4] 2> 291347 T1069 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4] 2> 291347 T1069 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 291348 T1068 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 291349 T1069 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node9",
[junit4] 2> "core":"core9",
[junit4] 2> "core_node_name":"node9",
[junit4] 2> "collection":"collection1
[...truncated too long message...]
Elector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 326763 T1454 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 327236 T1463 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 327237 T1463 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 327713 T1472 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 327714 T1472 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:303)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 329747 T1037 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.OverseerTest-BD8A990A69ECBB09-001
[junit4] 2> NOTE: test params are: codec=Lucene41, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=es_PE, timezone=JST
[junit4] 2> NOTE: Linux 3.8.0-39-generic amd64/Oracle Corporation 1.7.0_55 (64-bit)/cpus=8,threads=1,free=155152424,total=294125568
[junit4] 2> NOTE: All tests run in this JVM: [BasicDistributedZkTest, ChangedSchemaMergeTest, TestHighlightDedupGrouping, TestPseudoReturnFields, SolrIndexSplitterTest, TestSolrXmlPersistence, TestBadConfig, TestPerFieldSimilarity, HdfsCollectionsAPIDistributedZkTest, TestConfig, ParsingFieldUpdateProcessorsTest, ShardSplitTest, UnloadDistributedZkTest, SliceStateTest, JsonLoaderTest, CoreAdminHandlerTest, TestExtendedDismaxParser, TestMaxScoreQueryParser, TestDynamicFieldResource, TestShardHandlerFactory, TestLMJelinekMercerSimilarityFactory, TestSolrDeletionPolicy1, ZkControllerTest, MoreLikeThisHandlerTest, ConnectionManagerTest, TestFastLRUCache, TestSort, SpellPossibilityIteratorTest, TestCopyFieldCollectionResource, SuggesterFSTTest, RegexBoostProcessorTest, RecoveryZkTest, TestFunctionQuery, OverseerTest]
[junit4] Completed on J0 in 43.03s, 9 tests, 1 error <<< FAILURES!
[...truncated 991 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 398 suites, 1673 tests, 1 error, 37 ignored (16 assumptions)
Total time: 52 minutes 56 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_55 -XX:+UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (64bit/ibm-j9-jdk7) - Build #
10261 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10261/
Java: 64bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
All tests passed
Build Log:
[...truncated 20412 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:92: The following files contain @author tags, tabs or nocommits:
* solr/core/src/test/org/apache/solr/handler/TestReplicationHandler.java
Total time: 53 minutes 12 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_20-ea-b11) -
Build # 10260 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10260/
Java: 32bit/jdk1.8.0_20-ea-b11 -client -XX:+UseG1GC
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest
Error Message:
ERROR: SolrZkClient opens=21 closes=20
Stack Trace:
java.lang.AssertionError: ERROR: SolrZkClient opens=21 closes=20
at __randomizedtesting.SeedInfo.seed([6D0A8E25DED4B354]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:451)
at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:185)
at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:790)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 1) Thread[id=3041, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[6D0A8E25DED4B354]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 2) Thread[id=3040, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[6D0A8E25DED4B354]-SendThread(127.0.0.1:46497), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:
1) Thread[id=3041, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[6D0A8E25DED4B354]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
2) Thread[id=3040, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[6D0A8E25DED4B354]-SendThread(127.0.0.1:46497), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
at __randomizedtesting.SeedInfo.seed([6D0A8E25DED4B354]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=3040, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[6D0A8E25DED4B354]-SendThread(127.0.0.1:46497), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=3040, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[6D0A8E25DED4B354]-SendThread(127.0.0.1:46497), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
at __randomizedtesting.SeedInfo.seed([6D0A8E25DED4B354]:0)
REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch
Error Message:
some core start times did not change on reload
Stack Trace:
java.lang.AssertionError: some core start times did not change on reload
at __randomizedtesting.SeedInfo.seed([6D0A8E25DED4B354:ECEC003DA98BD368]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:836)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:203)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11314 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/init-core-data-001
[junit4] 2> 703456 T2890 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 703456 T2890 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 703462 T2890 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 703463 T2890 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 703463 T2891 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 703563 T2890 oasc.ZkTestServer.run start zk server on port:46497
[junit4] 2> 703565 T2890 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 713803 T2894 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 10235ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 713806 T2892 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x145db8e1b220000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 715564 T2897 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fa4265 name:ZooKeeperConnection Watcher:127.0.0.1:46497 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 715564 T2890 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 715565 T2890 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 715574 T2890 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 715576 T2899 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cfdd10 name:ZooKeeperConnection Watcher:127.0.0.1:46497/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 715576 T2890 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 715577 T2890 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 715583 T2890 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 715588 T2890 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 715592 T2890 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 715597 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 715598 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 715605 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 715606 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 715612 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 715613 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 715617 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 715619 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 715624 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 715625 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 715631 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 715632 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 715636 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 715637 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 715641 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 715642 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 715646 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 715647 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 715652 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 715652 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 715655 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 715656 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 715659 T2890 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 715660 T2901 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16649b6 name:ZooKeeperConnection Watcher:127.0.0.1:46497/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 715660 T2890 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> Using old style solr.xml
[junit4] 2> 715664 T2890 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 715665 T2903 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13716b1 name:ZooKeeperConnection Watcher:127.0.0.1:46497 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 715665 T2890 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 715665 T2890 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 715669 T2890 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 715671 T2905 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a48118 name:ZooKeeperConnection Watcher:127.0.0.1:46497/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 715671 T2890 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 715672 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf2/solrconfig.xml
[junit4] 2> 715672 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.xml
[junit4] 2> 715676 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
[junit4] 2> 715677 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf2/schema.xml
[junit4] 2> 715679 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 715680 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 715682 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
[junit4] 2> 715683 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf2/stopwords.txt
[junit4] 2> 715684 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
[junit4] 2> 715685 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf2/protwords.txt
[junit4] 2> 715688 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
[junit4] 2> 715689 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf2/currency.xml
[junit4] 2> 715691 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
[junit4] 2> 715691 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf2/open-exchange-rates.json
[junit4] 2> 715693 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
[junit4] 2> 715693 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
[junit4] 2> 715695 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
[junit4] 2> 715696 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf2/old_synonyms.txt
[junit4] 2> 715698 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
[junit4] 2> 715698 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf2/synonyms.txt
[junit4] 2> 715700 T2890 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
[junit4] 2> 715700 T2890 oascc.SolrZkClient.makePath makePath: /configs/conf2/elevate.xml
[junit4] 2> 716011 T2890 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 716021 T2890 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 716030 T2890 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:38483
[junit4] 2> 716036 T2890 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 716037 T2890 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 716038 T2890 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-002
[junit4] 2> 716039 T2890 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-002/'
[junit4] 2> 716072 T2890 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-002/solr.xml
[junit4] 2> 716086 T2890 oasc.CoreContainer.<init> New CoreContainer 7359148
[junit4] 2> 716087 T2890 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-002/]
[junit4] 2> 716087 T2890 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 716088 T2890 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 716088 T2890 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 716088 T2890 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 716089 T2890 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 716089 T2890 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 716089 T2890 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 716090 T2890 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 716090 T2890 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 716093 T2890 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 716094 T2890 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 716094 T2890 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 716095 T2890 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:46497/solr
[junit4] 2> 716095 T2890 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 716096 T2890 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 716098 T2916 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1933a3f name:ZooKeeperConnection Watcher:127.0.0.1:46497 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 716099 T2890 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 716105 T2890 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 716107 T2918 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@52451b name:ZooKeeperConnection Watcher:127.0.0.1:46497/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 716107 T2890 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 716110 T2890 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 716115 T2890 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 716119 T2890 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 716124 T2890 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 716129 T2890 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 716137 T2890 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 716140 T2890 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38483_
[junit4] 2> 716142 T2890 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38483_
[junit4] 2> 716147 T2890 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 716150 T2890 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 716155 T2890 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:38483_
[junit4] 2> 716156 T2890 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 716157 T2890 oasc.Overseer.start Overseer (id=91720770818015239-127.0.0.1:38483_-n_0000000000) starting
[junit4] 2> 716162 T2890 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 716181 T2920 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 716182 T2890 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 716183 T2920 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes at 127.0.0.1:38483_
[junit4] 2> 716187 T2890 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 716196 T2890 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 716201 T2919 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 716205 T2890 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 716206 T2890 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 716219 T2890 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 716222 T2923 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8b35f7 name:ZooKeeperConnection Watcher:127.0.0.1:46497/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 716224 T2890 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 716227 T2890 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 716233 T2890 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 716528 T2890 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 716529 T2890 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 716536 T2890 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 716540 T2890 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:41275
[junit4] 2> 716544 T2890 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 716546 T2890 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 716547 T2890 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-003
[junit4] 2> 716547 T2890 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-003/'
[junit4] 2> 716567 T2890 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-003/solr.xml
[junit4] 2> 716598 T2890 oasc.CoreContainer.<init> New CoreContainer 23065459
[junit4] 2> 716599 T2890 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-003/]
[junit4] 2> 716600 T2890 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 716600 T2890 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 716601 T2890 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 716601 T2890 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 716601 T2890 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 716602 T2890 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 716602 T2890 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 716603 T2890 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 716603 T2890 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 716607 T2890 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 716607 T2890 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 716607 T2890 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 716608 T2890 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:46497/solr
[junit4] 2> 716608 T2890 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 716609 T2890 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 716612 T2934 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@172b02b name:ZooKeeperConnection Watcher:127.0.0.1:46497 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 716613 T2890 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 716616 T2890 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 716618 T2936 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1acd247 name:ZooKeeperConnection Watcher:127.0.0.1:46497/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 716618 T2890 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 716624 T2890 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 717627 T2890 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41275_
[junit4] 2> 717628 T2890 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41275_
[junit4] 2> 717635 T2890 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 717636 T2890 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 717924 T2890 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 717925 T2890 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 717930 T2890 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 717932 T2890 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:54499
[junit4] 2> 717935 T2890 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 717935 T2890 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 717935 T2890 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-004
[junit4] 2> 717936 T2890 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-004/'
[junit4] 2> 717953 T2890 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-004/solr.xml
[junit4] 2> 717965 T2890 oasc.CoreContainer.<init> New CoreContainer 29398386
[junit4] 2> 717966 T2890 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-004/]
[junit4] 2> 717966 T2890 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 717967 T2890 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 717967 T2890 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 717967 T2890 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 717968 T2890 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 717968 T2890 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 717968 T2890 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 717969 T2890 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 717969 T2890 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 717971 T2890 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 717972 T2890 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 717972 T2890 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 717973 T2890 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:46497/solr
[junit4] 2> 717973 T2890 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 717974 T2890 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 717976 T2948 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@de81bc name:ZooKeeperConnection Watcher:127.0.0.1:46497 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 717976 T2890 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 717978 T2890 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 717980 T2950 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e02d42 name:ZooKeeperConnection Watcher:127.0.0.1:46497/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 717980 T2890 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 717987 T2890 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 718989 T2890 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54499_
[junit4] 2> 718990 T2890 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54499_
[junit4] 2> 719001 T2890 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 719003 T2890 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 719304 T2890 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 719304 T2890 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 719309 T2890 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 719311 T2890 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:48090
[junit4] 2> 719313 T2890 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 719314 T2890 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 719315 T2890 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-005
[junit4] 2> 719315 T2890 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-005/'
[junit4] 2> 719334 T2890 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-005/solr.xml
[junit4] 2> 719345 T2890 oasc.CoreContainer.<init> New CoreContainer 14930487
[junit4] 2> 719346 T2890 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-005/]
[junit4] 2> 719346 T2890 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 719347 T2890 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 719347 T2890 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 719347 T2890 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 719348 T2890 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 719348 T2890 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 719348 T2890 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 719349 T2890 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 719349 T2890 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 719352 T2890 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 719352 T2890 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 719352 T2890 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 719353 T2890 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:46497/solr
[junit4] 2> 719353 T2890 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 719354 T2890 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 719356 T2962 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a7a21 name:ZooKeeperConnection Watcher:127.0.0.1:46497 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 719356 T2890 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 719358 T2890 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 719360 T2964 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15ff79e name:ZooKeeperConnection Watcher:127.0.0.1:46497/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 719360 T2890 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 719366 T2890 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 720369 T2890 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48090_
[junit4] 2> 720370 T2890 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48090_
[junit4] 2> 720378 T2890 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 720379 T2890 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 720656 T2890 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
[junit4] 2> 720657 T2890 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 720660 T2890 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 720663 T2890 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:59603
[junit4] 2> 720665 T2890 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 720666 T2890 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 720666 T2890 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-006
[junit4] 2> 720666 T2890 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-006/'
[junit4] 2> 720698 T2890 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-006/solr.xml
[junit4] 2> 720710 T2890 oasc.CoreContainer.<init> New CoreContainer 16574351
[junit4] 2> 720710 T2890 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-006/]
[junit4] 2> 720711 T2890 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 720712 T2890 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 720712 T2890 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 720712 T2890 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 720713 T2890 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 720713 T2890 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 720713 T2890 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 720714 T2890 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 720714 T2890 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 720718 T2890 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 720718 T2890 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 720719 T2890 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 720719 T2890 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:46497/solr
[junit4] 2> 720719 T2890 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 720720 T2890 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 720722 T2976 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c257be name:ZooKeeperConnection Watcher:127.0.0.1:46497 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 720722 T2890 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 720725 T2890 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 720726 T2978 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@193cd2f name:ZooKeeperConnection Watcher:127.0.0.1:46497/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 720727 T2890 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 720733 T2890 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 721736 T2890 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59603_
[junit4] 2> 721737 T2890 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59603_
[junit4] 2> 721746 T2890 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 721747 T2890 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 721828 T2931 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&collection.configName=conf1&router.field=myOwnField&name=solrj_collection&router.name=compositeId&action=CREATE&numShards=2&wt=javabin&version=2
[junit4] 2> 721830 T2918 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4] 2> 721832 T2920 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"solrj_collection",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "router.field":"myOwnField",
[junit4] 2> "router.name":"compositeId"}
[junit4] 2> 721833 T2920 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
[junit4] 2> "operation":"createcollection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"solrj_collection",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "router.field":"myOwnField",
[junit4] 2> "router.name":"compositeId"}
[junit4] 2> 721834 T2920 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/solrj_collection
[junit4] 2> 721834 T2920 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection
[junit4] 2> 721838 T2918 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 721839 T2919 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 721840 T2919 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: solrj_collection
[junit4] 2> 721840 T2919 oasc.Overseer$ClusterStateUpdater.createCollection Create collection solrj_collection with shards [shard1, shard2]
[junit4] 2> 721849 T2918 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 721850 T2936 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 721850 T2978 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 721850 T2923 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 721850 T2918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 721850 T2964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 721850 T2950 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 721938 T2920 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection, shardNames [shard1, shard2] , replicationFactor : 2
[junit4] 2> 721939 T2920 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica1 as part of slice shard1 of collection solrj_collection on 127.0.0.1:48090_
[junit4] 2> 721941 T2920 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica2 as part of slice shard1 of collection solrj_collection on 127.0.0.1:54499_
[junit4] 2> 721945 T2920 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica1 as part of slice shard2 of collection solrj_collection on 127.0.0.1:41275_
[junit4] 2> 721947 T2920 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica2 as part of slice shard2 of collection solrj_collection on 127.0.0.1:38483_
[junit4] 2> 722023 T2944 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard1_replica2&action=CREATE&numShards=2&collection=solrj_collection&shard=shard1&wt=javabin&version=2
[junit4] 2> 722025 T2944 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica2 state=down collection=solrj_collection
[junit4] 2> 722027 T2918 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 722027 T2944 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 722029 T2919 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 722029 T2919 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:54499",
[junit4] 2> "core":"solrj_collection_shard1_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:54499_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 722032 T2918 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 722034 T2930 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard2_replica1&action=CREATE&numShards=2&collection=solrj_collection&shard=shard2&wt=javabin&version=2
[junit4] 2> 722036 T2930 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica1 state=down collection=solrj_collection
[junit4] 2> 722038 T2918 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 722040 T2919 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:41275",
[junit4] 2> "core":"solrj_collection_shard2_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:41275_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 722042 T2918 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 722042 T2930 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 722053 T2957 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard1_replica1&action=CREATE&numShards=2&collection=solrj_collection&shard=shard1&wt=javabin&version=2
[junit4] 2> 722055 T2911 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard2_replica2&action=CREATE&numShards=2&collection=solrj_collection&shard=shard2&wt=javabin&version=2
[junit4] 2> 722056 T2957 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica1 state=down collection=solrj_collection
[junit4] 2> 722057 T2918 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 722057 T2957 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 722058 T2911 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica2 state=down collection=solrj_collection
[junit4] 2> 722058 T2919 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:48090",
[junit4] 2> "core":"solrj_collection_shard1_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:48090_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 722059 T2918 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 722059 T2911 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 722062 T2919 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:38483",
[junit4] 2> "core":"solrj_collection_shard2_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:38483_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 722064 T2918 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 722167 T2936 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 722167 T2918 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 722167 T2964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 722167 T2923 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 722167 T2978 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 722167 T2950 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 723028 T2944 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica2
[junit4] 2> 723029 T2944 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-004/solr.xml
[junit4] 2> 723029 T2944 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 723030 T2944 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 723030 T2944 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 723032 T2944 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 723032 T2944 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-004/solrj_collection_shard1_replica2/'
[junit4] 2> 723044 T2930 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica1
[junit4] 2> 723045 T2930 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-003/solr.xml
[junit4] 2> 723046 T2930 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 723047 T2930 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 723048 T2930 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 723049 T2930 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 723049 T2930 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-003/solrj_collection_shard2_replica1/'
[junit4] 2> 723058 T2957 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica1
[junit4] 2> 723060 T2911 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica2
[junit4] 2> 723060 T2957 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-005/solr.xml
[junit4] 2> 723061 T2911 oasc.SolrXMLCoresLocator.doPersist Persisted core descriptions to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-002/solr.xml
[junit4] 2> 723061 T2957 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 723062 T2911 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 723063 T2957 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 723063 T2957 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 723063 T2911 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 723064 T2911 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 723064 T2957 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 723065 T2957 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-005/solrj_collection_shard1_replica1/'
[junit4] 2> 723067 T2911 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 723067 T2911 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-002/solrj_collection_shard2_replica2/'
[junit4] 2> 723081 T2944 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 723096 T2930 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 723098 T2944 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 723099 T2944 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 723117 T2930 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 723118 T2957 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 723119 T2930 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 723124 T2944 oass.IndexSchema.readSchema [solrj_collection_shard1_replica2] Schema name=test
[junit4] 2> 723125 T2911 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 723130 T2930 oass.IndexSchema.readSchema [solrj_collection_shard2_replica1] Schema name=test
[junit4] 2> 723139 T2957 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 723142 T2957 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 723147 T2911 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 723149 T2911 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 723168 T2957 oass.IndexSchema.readSchema [solrj_collection_shard1_replica1] Schema name=test
[junit4] 2> 723176 T2911 oass.IndexSchema.readSchema [solrj_collection_shard2_replica2] Schema name=test
[junit4] 2> 723319 T2930 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 723327 T2930 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 723329 T2930 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 723345 T2930 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 723348 T2930 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 723356 T2944 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 723358 T2930 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 723360 T2930 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 723361 T2930 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 723361 T2930 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 723363 T2930 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 723363 T2930 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 723364 T2944 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 723364 T2930 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica1' using configuration from collection solrj_collection
[junit4] 2> 723365 T2930 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 723365 T2930 oasc.SolrCore.<init> [solrj_collection_shard2_replica1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-003/solrj_collection_shard2_replica1/, dataDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-003/solrj_collection_shard2_replica1/data/
[junit4] 2> 723366 T2944 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 723366 T2930 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3b32c
[junit4] 2> 723370 T2930 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-003/solrj_collection_shard2_replica1/data
[junit4] 2> 723371 T2930 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-003/solrj_collection_shard2_replica1/data/index/
[junit4] 2> 723371 T2957 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 723372 T2930 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica1] Solr index directory '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-003/solrj_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 723374 T2930 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-003/solrj_collection_shard2_replica1/data/index
[junit4] 2> 723377 T2930 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=73.0107421875, floorSegmentMB=0.3388671875, forceMergeDeletesPctAllowed=13.023363852477885, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 723378 T2957 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 723381 T2957 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 723381 T2930 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@124feac lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9a8ebd),segFN=segments_1,generation=1}
[junit4] 2> 723383 T2930 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 723394 T2911 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 723397 T2944 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 723400 T2930 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 723401 T2930 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 723401 T2930 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 723402 T2930 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 723402 T2911 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 723403 T2930 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 723403 T2944 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 723404 T2957 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 723403 T2930 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 723405 T2911 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 723405 T2930 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 723406 T2930 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 723407 T2930 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 723408 T2944 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 723408 T2930 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 723408 T2957 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 723410 T2944 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 723410 T2930 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 723411 T2930 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 723411 T2944 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 723412 T2930 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 723413 T2944 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 723414 T2930 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 723415 T2930 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 723415 T2944 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 723415 T2944 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 723415 T2957 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 723416 T2944 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica2' using configuration from collection solrj_collection
[junit4] 2> 723416 T2930 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 723417 T2944 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 723417 T2957 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 723418 T2957 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 723418 T2930 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 723418 T2944 oasc.SolrCore.<init> [solrj_collection_shard1_replica2] Opening new SolrCore at ./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-004/solrj_collection_shard1_replica2/, dataDir=./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-004/solrj_collection_shard1_replica2/data/
[junit4] 2> 723419 T2957 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 723420 T2944 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3b32c
[junit4] 2> 723421 T2957 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 723421 T2957 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 723421 T2911 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 723422 T2944 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-004/solrj_collection_shard1_replica2/data
[junit4] 2> 723422 T2957 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica1' using configuration from collection solrj_collection
[junit4] 2> 723422 T2944 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-004/solrj_collection_shard1_replica2/data/index/
[junit4] 2> 723422 T2930 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 723423 T2944 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica2] Solr index directory './solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-004/solrj_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4] 2> 723423 T2957 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 723425 T2944 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-004/solrj_collection_shard1_replica2/data/index
[junit4] 2> 723425 T2911 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 723425 T2957 oasc.SolrCore.<init> [solrj_collection_shard1_replica1] Opening new SolrCore at ./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-005/solrj_collection_shard1_replica1/, dataDir=./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-005/solrj_collection_shard1_replica1/data/
[junit4] 2> 723425 T2944 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=73.0107421875, floorSegmentMB=0.3388671875, forceMergeDeletesPctAllowed=13.023363852477885, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 723426 T2930 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 723426 T2957 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3b32c
[junit4] 2> 723428 T2944 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@16381dd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@10b6cd0),segFN=segments_1,generation=1}
[junit4] 2> 723429 T2944 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 723428 T2930 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 723429 T2957 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-005/solrj_collection_shard1_replica1/data
[junit4] 2> 723431 T2957 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-005/solrj_collection_shard1_replica1/data/index/
[junit4] 2> 723431 T2911 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 723432 T2930 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=44, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=27.37890625, floorSegmentMB=1.900390625, forceMergeDeletesPctAllowed=13.614706086156122, segmentsPerTier=49.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 723432 T2957 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica1] Solr index directory './solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-005/solrj_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 723433 T2911 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 723434 T2911 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 723434 T2957 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-005/solrj_collection_shard1_replica1/data/index
[junit4] 2> 723433 T2930 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@124feac lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9a8ebd),segFN=segments_1,generation=1}
[junit4] 2> 723435 T2957 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=73.0107421875, floorSegmentMB=0.3388671875, forceMergeDeletesPctAllowed=13.023363852477885, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 723434 T2911 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 723436 T2957 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1d19142 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@180743e),segFN=segments_1,generation=1}
[junit4] 2> 723437 T2957 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 723435 T2930 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 723437 T2944 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 723438 T2911 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 723437 T2930 oass.SolrIndexSearcher.<init> Opening Searcher@10ddbce[solrj_collection_shard2_replica1] main
[junit4] 2> 723438 T2911 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 723438 T2944 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 723439 T2911 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica2' using configuration from collection solrj_collection
[junit4] 2> 723439 T2930 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 723440 T2911 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 723440 T2944 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 723441 T2911 oasc.SolrCore.<init> [solrj_collection_shard2_replica2] Opening new SolrCore at ./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-002/solrj_collection_shard2_replica2/, dataDir=./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-002/solrj_collection_shard2_replica2/data/
[junit4] 2> 723441 T2944 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 723442 T2930 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 723442 T2930 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 723442 T2911 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@a3b32c
[junit4] 2> 723442 T2944 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 723444 T2930 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 723444 T2957 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 723444 T2911 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-002/solrj_collection_shard2_replica2/data
[junit4] 2> 723444 T2944 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 723445 T2911 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-002/solrj_collection_shard2_replica2/data/index/
[junit4] 2> 723445 T2957 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 723444 T2930 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 723446 T2957 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 723446 T2911 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica2] Solr index directory './solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-002/solrj_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4] 2> 723446 T2944 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 723448 T2911 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.CollectionsAPIDistributedZkTest-6D0A8E25DED4B354-001/tempDir-002/solrj_collection_shard2_replica2/data/index
[junit4] 2> 723447 T2957 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 723447 T2930 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 723449 T2957 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 723449 T2911 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=73.0107421875, floorSegmentMB=0.3388671875, forceMergeDeletesPctAllowed=13.023363852477885, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 723448 T2944 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 723450 T2930 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 723450 T2957 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 723451 T2930 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 723452 T2930 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 723452 T2930 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 723450 T2911 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@17803c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@bd0c9b),segFN=segments_1,generation=1}
[junit4] 2> 723450 T2944 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 723453 T2911 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 723451 T2957 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 723454 T2944 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 723454 T2957 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 723455 T2957 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 723455 T2944 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 723456 T2944 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 723456 T2957 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 723457 T2944 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 723458 T2984 oasc.SolrCore.registerSearcher [solrj_collection_shard2_replica1] Registered new searcher Searcher@10ddbce[solrj_collection_shard2_replica1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 723458 T2957 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 723458 T2957 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHan
[...truncated too long message...]
T3040 oazc.StaticHostProvider.next WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 821105 T3040 oaz.ClientCnxn$SendThread.run WARN Session 0x145db8e1b220012 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[junit4] 2>
[junit4] 2> 821603 T3040 oaz.ClientCnxn$SendThread.run WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
[junit4] 2>
[junit4] 2> 822104 T3040 oazc.StaticHostProvider.next WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 822105 T3040 oaz.ClientCnxn$SendThread.run WARN Session 0x145db8e1b220012 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[junit4] 2>
[junit4] 2> 119148 T2889 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=3040, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[6D0A8E25DED4B354]-SendThread(127.0.0.1:46497), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
[junit4] 2> NOTE: test params are: codec=Lucene45, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=sr_ME_#Latn, timezone=Europe/Uzhgorod
[junit4] 2> NOTE: Linux 3.8.0-39-generic i386/Oracle Corporation 1.8.0_20-ea (32-bit)/cpus=8,threads=2,free=67515848,total=246415360
[junit4] 2> NOTE: All tests run in this JVM: [ResourceLoaderTest, NoCacheHeaderTest, SortByFunctionTest, TestSolrQueryParserResource, PluginInfoTest, CursorPagingTest, DirectSolrSpellCheckerTest, FileUtilsTest, TestComplexPhraseQParserPlugin, TestZkChroot, ParsingFieldUpdateProcessorsTest, TestAnalyzedSuggestions, TestFiltering, TestCustomSort, TestDefaultSearchFieldResource, TestDynamicFieldResource, QueryResultKeyTest, TestExpandComponent, TestArbitraryIndexDir, TestUpdate, UniqFieldsUpdateProcessorFactoryTest, SpellCheckCollatorTest, TestLMDirichletSimilarityFactory, TermVectorComponentDistributedTest, DirectUpdateHandlerTest, TestIndexSearcher, TestIBSimilarityFactory, StandardRequestHandlerTest, TestCollationField, DistributedDebugComponentTest, RangeFacetTest, TestStressRecovery, AsyncMigrateRouteKeyTest, StressHdfsTest, CollectionsAPIAsyncDistributedZkTest, AtomicUpdatesTest, TestFuzzyAnalyzedSuggestions, SuggesterTSTTest, TestRequestStatusCollectionAPI, TestFieldCollectionResource, TestDocumentBuilder, TestWordDelimiterFilterFactory, TestSweetSpotSimilarityFactory, TestHashPartitioner, TestManagedSchemaFieldResource, TestComponentsName, TestManagedResource, TestRandomMergePolicy, TestSimpleQParserPlugin, HardAutoCommitTest, DOMUtilTest, SystemInfoHandlerTest, ChaosMonkeySafeLeaderTest, TestFoldingMultitermQuery, TestSolrJ, CoreAdminRequestStatusTest, TestOmitPositions, TestReload, BasicZkTest, JSONWriterTest, HdfsRecoveryZkTest, TestRecovery, TestCSVLoader, FullSolrCloudDistribCmdsTest, TestSolrXMLSerializer, TestCloudManagedSchema, AssignTest, DistanceFunctionTest, TestSchemaNameResource, TestSolr4Spatial, TestSolrXmlPersistor, TestDistributedSearch, TestCodecSupport, TestRTGBase, MinimalSchemaTest, PrimUtilsTest, TestPartialUpdateDeduplication, CoreContainerCoreInitFailuresTest, XmlUpdateRequestHandlerTest, BadIndexSchemaTest, PingRequestHandlerTest, TestSchemaVersionResource, TestQuerySenderNoQuery, RequestHandlersTest, HdfsLockFactoryTest, SolrPluginUtilsTest, TestOverriddenPrefixQueryForCustomFieldType, TestStressReorder, TestSolrCoreProperties, EnumFieldTest, HdfsBasicDistributedZkTest, ZkSolrClientTest, TestDFRSimilarityFactory, TestIndexingPerformance, DocExpirationUpdateProcessorFactoryTest, HdfsSyncSliceTest, SignatureUpdateProcessorFactoryTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.seed=6D0A8E25DED4B354 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=sr_ME_#Latn -Dtests.timezone=Europe/Uzhgorod -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | CollectionsAPIDistributedZkTest (suite) <<<
[junit4] > Throwable #1: java.lang.AssertionError: ERROR: SolrZkClient opens=21 closes=20
[junit4] > at __randomizedtesting.SeedInfo.seed([6D0A8E25DED4B354]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:451)
[junit4] > at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:185)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:
[junit4] > 1) Thread[id=3041, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[6D0A8E25DED4B354]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
[junit4] > 2) Thread[id=3040, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[6D0A8E25DED4B354]-SendThread(127.0.0.1:46497), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > at __randomizedtesting.SeedInfo.seed([6D0A8E25DED4B354]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=3040, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[6D0A8E25DED4B354]-SendThread(127.0.0.1:46497), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
[junit4] > at __randomizedtesting.SeedInfo.seed([6D0A8E25DED4B354]:0)
[junit4] Completed on J1 in 119.22s, 1 test, 2 failures, 2 errors <<< FAILURES!
[...truncated 594 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 398 suites, 1673 tests, 3 suite-level errors, 1 failure, 704 ignored (16 assumptions)
Total time: 58 minutes 44 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0_20-ea-b11 -client -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_60-ea-b15) -
Build # 10259 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10259/
Java: 64bit/jdk1.7.0_60-ea-b15 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
All tests passed
Build Log:
[...truncated 20421 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:92: The following files contain @author tags, tabs or nocommits:
* solr/core/src/test/org/apache/solr/handler/TestReplicationHandler.java
Total time: 64 minutes 42 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_60-ea-b15 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_55) - Build #
10258 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10258/
Java: 64bit/jdk1.7.0_55 -XX:-UseCompressedOops -XX:+UseG1GC
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZk2Test
Error Message:
ERROR: SolrZkClient opens=13 closes=12
Stack Trace:
java.lang.AssertionError: ERROR: SolrZkClient opens=13 closes=12
at __randomizedtesting.SeedInfo.seed([B0CE312826844528]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:451)
at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:185)
at sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:790)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZk2Test
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZk2Test: 1) Thread[id=3185, name=TEST-BasicDistributedZk2Test.testDistribSearch-seed#[B0CE312826844528]-SendThread(127.0.0.1:44425), state=TIMED_WAITING, group=TGRP-BasicDistributedZk2Test] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) 2) Thread[id=3186, name=TEST-BasicDistributedZk2Test.testDistribSearch-seed#[B0CE312826844528]-EventThread, state=WAITING, group=TGRP-BasicDistributedZk2Test] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZk2Test:
1) Thread[id=3185, name=TEST-BasicDistributedZk2Test.testDistribSearch-seed#[B0CE312826844528]-SendThread(127.0.0.1:44425), state=TIMED_WAITING, group=TGRP-BasicDistributedZk2Test]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
2) Thread[id=3186, name=TEST-BasicDistributedZk2Test.testDistribSearch-seed#[B0CE312826844528]-EventThread, state=WAITING, group=TGRP-BasicDistributedZk2Test]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
at __randomizedtesting.SeedInfo.seed([B0CE312826844528]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZk2Test
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=3185, name=TEST-BasicDistributedZk2Test.testDistribSearch-seed#[B0CE312826844528]-SendThread(127.0.0.1:44425), state=TIMED_WAITING, group=TGRP-BasicDistributedZk2Test] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=3185, name=TEST-BasicDistributedZk2Test.testDistribSearch-seed#[B0CE312826844528]-SendThread(127.0.0.1:44425), state=TIMED_WAITING, group=TGRP-BasicDistributedZk2Test]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
at __randomizedtesting.SeedInfo.seed([B0CE312826844528]:0)
REGRESSION: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 503 Server is shutting down or failed to initialize</title> </head> <body> <h2>HTTP ERROR: 503</h2> <p>Problem accessing /qrb/mq/admin/cores. Reason: <pre> Server is shutting down or failed to initialize</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html>
Stack Trace:
java.lang.AssertionError: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 503 Server is shutting down or failed to initialize</title>
</head>
<body>
<h2>HTTP ERROR: 503</h2>
<p>Problem accessing /qrb/mq/admin/cores. Reason:
<pre> Server is shutting down or failed to initialize</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([B0CE312826844528:3128BF3051DB2514]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:176)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:82)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11191 lines...]
[junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/init-core-data-001
[junit4] 2> 685415 T3141 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /qrb/mq
[junit4] 2> 685419 T3141 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 685419 T3141 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 685420 T3142 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 685520 T3141 oasc.ZkTestServer.run start zk server on port:44425
[junit4] 2> 685520 T3141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 685567 T3148 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@22ad90f6 name:ZooKeeperConnection Watcher:127.0.0.1:44425 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 685568 T3141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 685568 T3141 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 685572 T3141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 685573 T3150 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@66c5f0e9 name:ZooKeeperConnection Watcher:127.0.0.1:44425/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 685574 T3141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 685574 T3141 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 685577 T3141 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 685580 T3141 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 685581 T3141 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 685583 T3141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 685583 T3141 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 685586 T3141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 685587 T3141 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 685589 T3141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 685589 T3141 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 685591 T3141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 685591 T3141 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 685593 T3141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 685593 T3141 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 685594 T3141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 685595 T3141 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 685596 T3141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 685596 T3141 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 685598 T3141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 685598 T3141 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 685600 T3141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 685600 T3141 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 685602 T3141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 685602 T3141 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 685605 T3141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 685605 T3141 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 685883 T3141 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 685887 T3141 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34666
[junit4] 2> 685887 T3141 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 685888 T3141 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 685888 T3141 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-002
[junit4] 2> 685888 T3141 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-002/'
[junit4] 2> 685909 T3141 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-002/solr.xml
[junit4] 2> 685962 T3141 oasc.CoreContainer.<init> New CoreContainer 2026325894
[junit4] 2> 685962 T3141 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-002/]
[junit4] 2> 685964 T3141 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 685964 T3141 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 685964 T3141 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 685965 T3141 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 685965 T3141 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 685966 T3141 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 685966 T3141 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 685966 T3141 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 685967 T3141 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 685967 T3141 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 685968 T3141 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 685968 T3141 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 685969 T3141 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44425/solr
[junit4] 2> 685969 T3141 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 685970 T3141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 685971 T3161 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6328ec5c name:ZooKeeperConnection Watcher:127.0.0.1:44425 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 685972 T3141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 685974 T3141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 685975 T3163 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7dcaa32f name:ZooKeeperConnection Watcher:127.0.0.1:44425/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 685975 T3141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 685976 T3141 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 685978 T3141 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 685980 T3141 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 685982 T3141 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 685983 T3141 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 685985 T3141 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 685986 T3141 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34666_qrb%2Fmq
[junit4] 2> 685987 T3141 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34666_qrb%2Fmq
[junit4] 2> 685989 T3141 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 685990 T3141 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 687046 T3145 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 1055ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 687050 T3141 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:34666_qrb%2Fmq
[junit4] 2> 687050 T3141 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 687052 T3141 oasc.Overseer.start Overseer (id=91719318907584515-127.0.0.1:34666_qrb%2Fmq-n_0000000000) starting
[junit4] 2> 687054 T3141 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 687059 T3165 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 687060 T3141 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 687060 T3165 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes at 127.0.0.1:34666_qrb%2Fmq
[junit4] 2> 687062 T3141 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 687063 T3141 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 687064 T3164 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 687068 T3166 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 687068 T3166 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 687069 T3163 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 687069 T3166 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 687070 T3164 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 687070 T3164 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:34666/qrb/mq",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:34666_qrb%2Fmq",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 687070 T3164 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 687070 T3164 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 687072 T3163 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 687072 T3163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 688069 T3166 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 688069 T3166 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 688070 T3166 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 688070 T3166 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 688071 T3166 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 688071 T3166 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-002/collection1/'
[junit4] 2> 688071 T3166 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 688072 T3166 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 688072 T3166 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 688135 T3166 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 688183 T3166 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 688184 T3166 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 688193 T3166 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 688519 T3166 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 688525 T3166 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 688528 T3166 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 688536 T3166 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 688539 T3166 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 688542 T3166 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 688542 T3166 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 688543 T3166 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 688543 T3166 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 688544 T3166 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 688544 T3166 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 688544 T3166 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 688544 T3166 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 688545 T3166 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-002/collection1/, dataDir=./solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-001/control/data/
[junit4] 2> 688545 T3166 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@34e04c26
[junit4] 2> 688545 T3166 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-001/control/data
[junit4] 2> 688546 T3166 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-001/control/data/index/
[junit4] 2> 688546 T3166 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 688546 T3166 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-001/control/data/index
[junit4] 2> 688546 T3166 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=93.748046875, floorSegmentMB=0.89453125, forceMergeDeletesPctAllowed=24.729679776307954, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 688547 T3166 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1245c4a3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1b94be11),segFN=segments_1,generation=1}
[junit4] 2> 688547 T3166 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 688550 T3166 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 688550 T3166 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 688551 T3166 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 688551 T3166 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 688551 T3166 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 688551 T3166 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 688552 T3166 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 688552 T3166 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 688552 T3166 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 688552 T3166 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 688553 T3166 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 688553 T3166 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 688553 T3166 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 688554 T3166 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 688554 T3166 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 688554 T3166 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 688555 T3166 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 688557 T3166 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 688559 T3166 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 688560 T3166 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 688560 T3166 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=81.392578125, floorSegmentMB=0.609375, forceMergeDeletesPctAllowed=7.647986325812458, segmentsPerTier=15.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 688560 T3166 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1245c4a3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1b94be11),segFN=segments_1,generation=1}
[junit4] 2> 688561 T3166 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 688561 T3166 oass.SolrIndexSearcher.<init> Opening Searcher@13ee0014[collection1] main
[junit4] 2> 688561 T3166 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 688562 T3166 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 688562 T3166 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 688563 T3166 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 688563 T3166 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 688563 T3166 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 688563 T3166 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 688564 T3166 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 688564 T3166 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 688564 T3166 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 688567 T3167 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@13ee0014[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 688568 T3166 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 688569 T3170 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34666/qrb/mq collection:control_collection shard:shard1
[junit4] 2> 688569 T3141 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 688569 T3141 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 688570 T3170 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 688570 T3141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 688589 T3172 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fd08d0 name:ZooKeeperConnection Watcher:127.0.0.1:44425/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 688589 T3141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 688609 T3141 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 688668 T3141 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 688668 T3170 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 688708 T3163 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 688709 T3170 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 688709 T3170 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1069 name=collection1 org.apache.solr.core.SolrCore@64e8fbf6 url=http://127.0.0.1:34666/qrb/mq/collection1 node=127.0.0.1:34666_qrb%2Fmq C1069_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:34666/qrb/mq, core=collection1, node_name=127.0.0.1:34666_qrb%2Fmq}
[junit4] 2> 688709 T3170 C1069 P34666 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34666/qrb/mq/collection1/
[junit4] 2> 688709 T3170 C1069 P34666 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 688710 T3170 C1069 P34666 oasc.SyncStrategy.syncToMe http://127.0.0.1:34666/qrb/mq/collection1/ has no replicas
[junit4] 2> 688710 T3170 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34666/qrb/mq/collection1/ shard1
[junit4] 2> 688710 T3170 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 688711 T3164 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 688901 T3141 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 688902 T3141 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 688904 T3141 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34555
[junit4] 2> 688905 T3141 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 688905 T3141 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 688906 T3141 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-003
[junit4] 2> 688906 T3141 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-003/'
[junit4] 2> 688920 T3141 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-003/solr.xml
[junit4] 2> 688961 T3141 oasc.CoreContainer.<init> New CoreContainer 278058062
[junit4] 2> 688961 T3141 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-003/]
[junit4] 2> 688962 T3141 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 688962 T3141 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 688963 T3141 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 688963 T3141 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 688963 T3141 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 688963 T3141 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 688963 T3141 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 688964 T3141 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 688964 T3141 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 688964 T3141 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 688965 T3141 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 688965 T3141 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 688965 T3141 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44425/solr
[junit4] 2> 688965 T3141 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 688966 T3141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 695335 T3143 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x145da3c0e870004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 695435 T3172 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fd08d0 name:ZooKeeperConnection Watcher:127.0.0.1:44425/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 695435 T3172 oascc.ConnectionManager.process zkClient has disconnected
[junit4] 2> 697067 T3172 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fd08d0 name:ZooKeeperConnection Watcher:127.0.0.1:44425/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 702067 T3143 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x145da3c0e870003, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 702167 T3163 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 702168 T3163 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 702168 T3163 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7dcaa32f name:ZooKeeperConnection Watcher:127.0.0.1:44425/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 702168 T3163 oascc.ConnectionManager.process zkClient has disconnected
[junit4] 2> 703736 T3143 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x145da3c0e870004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 703824 T3163 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7dcaa32f name:ZooKeeperConnection Watcher:127.0.0.1:44425/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 703825 T3163 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 703825 T3163 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 703836 T3172 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fd08d0 name:ZooKeeperConnection Watcher:127.0.0.1:44425/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 703836 T3172 oascc.ConnectionManager.process zkClient has disconnected
[junit4] 2> 705333 T3172 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fd08d0 name:ZooKeeperConnection Watcher:127.0.0.1:44425/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 712002 T3143 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x145da3c0e870004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 712102 T3172 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fd08d0 name:ZooKeeperConnection Watcher:127.0.0.1:44425/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 712102 T3172 oascc.ConnectionManager.process zkClient has disconnected
[junit4] 2> 713351 T3172 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fd08d0 name:ZooKeeperConnection Watcher:127.0.0.1:44425/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 715433 T3145 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 26703ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 715434 T3145 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:404)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 715435 T3145 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:404)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 715435 T3145 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:170)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 715436 T3183 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17b05562 name:ZooKeeperConnection Watcher:127.0.0.1:44425 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 715436 T3145 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:170)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 715436 T3141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 715436 T3145 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:170)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 715437 T3145 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:404)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 715438 T3145 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:170)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 715438 T3145 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:404)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 715439 T3170 oasc.SolrException.log ERROR There was a problem trying to register as the leader:org.apache.solr.common.SolrException: Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:136)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:291)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:142)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:110)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:267)
[junit4] 2> at org.apache.solr.cloud.ZkController.joinElection(ZkController.java:956)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:773)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:736)
[junit4] 2> at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:200)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/control_collection/leaders/shard1
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:432)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:429)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:386)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:373)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:128)
[junit4] 2> at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:31)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:123)
[junit4] 2> ... 11 more
[junit4] 2>
[junit4] 2> 715439 T3145 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:170)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 715439 T3170 oasc.ShardLeaderElectionContext.rejoinLeaderElection There may be a better leader candidate than us - going back into recovery
[junit4] 2> 715440 T3170 oasc.ElectionContext.cancelElection canceling election /collections/control_collection/leader_elect/shard1/election/91719318907584515-core_node1-n_0000000000
[junit4] 2> 715441 T3170 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> 715441 T3184 C1069 P34666 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 715441 T3163 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 715442 T3145 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.process(NIOServerCnxn.java:1118)
[junit4] 2> at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:120)
[junit4] 2> at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:92)
[junit4] 2> at org.apache.zookeeper.server.DataTree.setData(DataTree.java:620)
[junit4] 2> at org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:807)
[junit4] 2> at org.apache.zookeeper.server.ZKDatabase.processTxn(ZKDatabase.java:329)
[junit4] 2> at org.apache.zookeeper.server.ZooKeeperServer.processTxn(ZooKeeperServer.java:994)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:116)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 715442 T3172 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 715441 T3184 C1069 P34666 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 715442 T3145 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.process(NIOServerCnxn.java:1118)
[junit4] 2> at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:120)
[junit4] 2> at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:92)
[junit4] 2> at org.apache.zookeeper.server.DataTree.setData(DataTree.java:620)
[junit4] 2> at org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:807)
[junit4] 2> at org.apache.zookeeper.server.ZKDatabase.processTxn(ZKDatabase.java:329)
[junit4] 2> at org.apache.zookeeper.server.ZooKeeperServer.processTxn(ZooKeeperServer.java:994)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:116)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 715443 T3163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 715444 T3170 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 715457 T3170 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4] 2> 715457 T3170 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4] 2> 715457 T3170 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 715457 T3170 oasc.RecoveryStrategy.close WARN Stopping recovery for zkNodeName=core_node1core=collection1
[junit4] 2> 715459 T3164 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 715460 T3163 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 715540 T3183 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17b05562 name:ZooKeeperConnection Watcher:127.0.0.1:44425 got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 715541 T3183 oascc.ConnectionManager.process zkClient has disconnected
[junit4] 2> 715563 T3172 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 715563 T3163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 717348 T3183 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17b05562 name:ZooKeeperConnection Watcher:127.0.0.1:44425 got event WatchedEvent state:Expired type:None path:null path:null type:None
[junit4] 2> 717349 T3183 oascc.ConnectionManager.process Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
[junit4] 2> 717350 T3183 oascc.DefaultConnectionStrategy.reconnect Connection expired - starting a new one...
[junit4] 2> 717350 T3183 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 717353 T3186 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17b05562 name:ZooKeeperConnection Watcher:127.0.0.1:44425 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 717353 T3183 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 717354 T3183 oascc.ConnectionManager$1.update Connection with ZooKeeper reestablished.
[junit4] 2> 717354 T3183 oascc.DefaultConnectionStrategy.reconnect Reconnected to ZooKeeper
[junit4] 2> 717354 T3183 oascc.ConnectionManager.process Connected:true
[junit4] 2> 717449 T3141 oasc.ZkContainer.initZooKeeper ERROR org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /solr
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:226)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:223)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:223)
[junit4] 2> at org.apache.solr.cloud.ZkController.checkChrootPath(ZkController.java:662)
[junit4] 2> at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:127)
[junit4] 2> at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:61)
[junit4] 2> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:216)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:190)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:137)
[junit4] 2> at org.eclipse.jetty.servlet.FilterHolder.doStart(FilterHolder.java:119)
[junit4] 2> at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:719)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1309)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1345)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1085)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:931)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:888)
[junit4] 2> at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:340)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:325)
[junit4] 2> at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:174)
[junit4] 2> at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:65)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:430)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:403)
[junit4] 2> at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJetty(AbstractFullDistribZkTestBase.java:481)
[junit4] 2> at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:361)
[junit4] 2> at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:305)
[junit4] 2> at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:606)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4] 2> at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 717451 T3141 oass.SolrDispatchFilter.init ERROR Could not start Solr. Check solr/home property and the logs
[junit4] 2> 717452 T3141 oasc.SolrException.log ERROR null:org.apache.solr.common.cloud.ZooKeeperException:
[junit4] 2> at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:186)
[junit4] 2> at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:61)
[junit4] 2> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:216)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:190)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:137)
[junit4] 2> at org.eclipse.jetty.servlet.FilterHolder.doStart(FilterHolder.java:119)
[junit4] 2> at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:719)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1309)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1345)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1085)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:931)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:888)
[junit4] 2> at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:340)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:325)
[junit4] 2> at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:174)
[junit4] 2> at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:65)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:430)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:403)
[junit4] 2> at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJetty(AbstractFullDistribZkTestBase.java:481)
[junit4] 2> at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:361)
[junit4] 2> at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:305)
[junit4] 2> at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:863)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:606)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4] 2> at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /solr
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:226)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$4.execute(SolrZkClient.java:223)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:223)
[junit4] 2> at org.apache.solr.cloud.ZkController.checkChrootPath(ZkController.java:662)
[junit4] 2> at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:127)
[junit4] 2> ... 63 more
[junit4] 2>
[junit4] 2> 717453 T3141 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 717696 T3141 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 717697 T3141 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 717699 T3141 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:46979
[junit4] 2> 717700 T3141 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 717700 T3141 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 717700 T3141 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-004
[junit4] 2> 717700 T3141 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-004/'
[junit4] 2> 717715 T3141 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-004/solr.xml
[junit4] 2> 717755 T3141 oasc.CoreContainer.<init> New CoreContainer 689216134
[junit4] 2> 717755 T3141 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solr.cloud.BasicDistributedZk2Test-B0CE312826844528-001/tempDir-004/]
[junit4] 2> 717756 T3141 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 717757 T3141 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 717757 T3141 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 717757 T3141 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 717757 T3141 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 717758 T3141 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 717758 T3141 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 717758 T3141 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 717758 T3141 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 717759 T3141 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 717759 T3141 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 717759 T3141 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 717760 T3141 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44425/solr
[junit4] 2> 717760 T3141 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 717760 T3141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 717762 T3197 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ace87c name:ZooKeeperConnection Watcher:127.0.0.1:44425 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 717763 T3141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 717765 T3141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 717767 T3199 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d40733b name:ZooKeeperConnection Watcher:127.0.0.1:44425/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 717767 T3141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 717773 T3141 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 718775 T3141 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46979_qrb%2Fmq
[junit4] 2> 718776 T3141 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:46979_qrb%2Fmq
[junit4] 2> 718778 T3145 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.process(NIOServerCnxn.java:1118)
[junit4] 2> at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:120)
[junit4] 2> at org.apache.zookeeper.server.WatchManager.triggerWatch(WatchManager.java:92)
[junit4] 2> at org.apache.zookeeper.server.DataTree.createNode(DataTree.java:523)
[junit4] 2> at org.apache.zookeeper.server.DataTree.processTxn(DataTree.java:791)
[junit4] 2> at org.apache.zookeeper.server.ZKDatabase.processTxn(ZKDatabase.java:329)
[junit4] 2> at org.apache.zookeeper.server.ZooKeeperServer.processTxn(ZooKeeperServer.java:994)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:116)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 718780 T3145 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionK
[...truncated too long message...]
ientCnxn.java:994)
[junit4] 2>
[junit4] 2> 751198 T3185 oazc.StaticHostProvider.next WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 751199 T3185 oaz.ClientCnxn$SendThread.run WARN Session 0x145da3c0e870006 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[junit4] 2>
[junit4] 2> 751698 T3185 oaz.ClientCnxn$SendThread.run WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
[junit4] 2>
[junit4] 2> 752198 T3185 oazc.StaticHostProvider.next WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2>
[junit4] 2> 752199 T3185 oaz.ClientCnxn$SendThread.run WARN Session 0x145da3c0e870006 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[junit4] 2>
[junit4] 2> 752698 T3185 oaz.ClientCnxn$SendThread.run WARN Unexpected exception java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
[junit4] 2>
[junit4] 2> 67785 T3140 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=3185, name=TEST-BasicDistributedZk2Test.testDistribSearch-seed#[B0CE312826844528]-SendThread(127.0.0.1:44425), state=TIMED_WAITING, group=TGRP-BasicDistributedZk2Test]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2> NOTE: test params are: codec=Lucene42, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ro_RO, timezone=Asia/Ulan_Bator
[junit4] 2> NOTE: Linux 3.8.0-39-generic amd64/Oracle Corporation 1.7.0_55 (64-bit)/cpus=8,threads=2,free=226738032,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [HdfsUnloadDistributedZkTest, CustomCollectionTest, CollectionsAPIDistributedZkTest, TestQuerySenderNoQuery, TestFiltering, TestRecovery, TestDistribDocBasedVersion, TestManagedStopFilterFactory, DateMathParserTest, TestStressLucene, TestSchemaResource, CachingDirectoryFactoryTest, TestExceedMaxTermLength, RequestHandlersTest, SolrIndexConfigTest, StressHdfsTest, BJQParserTest, PluginInfoTest, QueryResultKeyTest, DistributedQueryComponentOptimizationTest, TestZkChroot, SortByFunctionTest, StatsComponentTest, TestIndexSearcher, DirectSolrSpellCheckerTest, FileUtilsTest, TestReload, SpellCheckComponentTest, BadIndexSchemaTest, TestBinaryResponseWriter, XsltUpdateRequestHandlerTest, TestCSVLoader, TestSweetSpotSimilarityFactory, DisMaxRequestHandlerTest, TestSolrQueryParserResource, DistanceFunctionTest, TestManagedSchemaFieldResource, SSLMigrationTest, ChaosMonkeySafeLeaderTest, TestLRUCache, HighlighterTest, SystemInfoHandlerTest, CurrencyFieldXmlFileTest, CoreAdminRequestStatusTest, TestRangeQuery, MultiTermTest, TestSolr4Spatial, LeaderElectionIntegrationTest, NoCacheHeaderTest, TestStressRecovery, ShardRoutingCustomTest, TestDocumentBuilder, HighlighterConfigTest, OverseerRolesTest, XmlUpdateRequestHandlerTest, TestXIncludeConfig, DeleteShardTest, AlternateDirectoryTest, TestLMDirichletSimilarityFactory, PrimUtilsTest, TestCollationFieldDocValues, TestUpdate, TestCSVResponseWriter, NumericFieldsTest, DocumentAnalysisRequestHandlerTest, DistributedDebugComponentTest, TestMultiCoreConfBootstrap, OpenExchangeRatesOrgProviderTest, TestConfigSets, TestBinaryField, BasicDistributedZk2Test]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZk2Test -Dtests.seed=B0CE312826844528 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ro_RO -Dtests.timezone=Asia/Ulan_Bator -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J1 | BasicDistributedZk2Test (suite) <<<
[junit4] > Throwable #1: java.lang.AssertionError: ERROR: SolrZkClient opens=13 closes=12
[junit4] > at __randomizedtesting.SeedInfo.seed([B0CE312826844528]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:451)
[junit4] > at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:185)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZk2Test:
[junit4] > 1) Thread[id=3185, name=TEST-BasicDistributedZk2Test.testDistribSearch-seed#[B0CE312826844528]-SendThread(127.0.0.1:44425), state=TIMED_WAITING, group=TGRP-BasicDistributedZk2Test]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > 2) Thread[id=3186, name=TEST-BasicDistributedZk2Test.testDistribSearch-seed#[B0CE312826844528]-EventThread, state=WAITING, group=TGRP-BasicDistributedZk2Test]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
[junit4] > at __randomizedtesting.SeedInfo.seed([B0CE312826844528]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=3185, name=TEST-BasicDistributedZk2Test.testDistribSearch-seed#[B0CE312826844528]-SendThread(127.0.0.1:44425), state=TIMED_WAITING, group=TGRP-BasicDistributedZk2Test]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > at __randomizedtesting.SeedInfo.seed([B0CE312826844528]:0)
[junit4] Completed on J1 in 67.82s, 1 test, 2 failures, 2 errors <<< FAILURES!
[...truncated 699 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 398 suites, 1673 tests, 3 suite-level errors, 1 failure, 944 ignored (12 assumptions)
Total time: 48 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_55 -XX:-UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_20-ea-b11) -
Build # 10257 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10257/
Java: 64bit/jdk1.8.0_20-ea-b11 -XX:-UseCompressedOops -XX:+UseParallelGC
All tests passed
Build Log:
[...truncated 20655 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:92: The following files contain @author tags, tabs or nocommits:
* solr/core/src/test/org/apache/solr/handler/TestReplicationHandler.java
Total time: 59 minutes 29 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0_20-ea-b11 -XX:-UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_55) - Build #
10256 - Still Failing!
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
FYI - This has been fixed by Noble.
On Thu, May 8, 2014 at 4:33 AM, Policeman Jenkins Server <
jenkins@thetaphi.de> wrote:
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10256/
> Java: 32bit/jdk1.7.0_55 -server -XX:+UseSerialGC
>
> All tests passed
>
> Build Log:
> [...truncated 20568 lines...]
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:92: The
> following files contain @author tags, tabs or nocommits:
> * solr/core/src/test/org/apache/solr/handler/TestReplicationHandler.java
>
> Total time: 68 minutes 25 seconds
> Build step 'Invoke Ant' marked build as failure
> Description set: Java: 32bit/jdk1.7.0_55 -server -XX:+UseSerialGC
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
--
Regards,
Shalin Shekhar Mangar.
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_55) - Build #
10256 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10256/
Java: 32bit/jdk1.7.0_55 -server -XX:+UseSerialGC
All tests passed
Build Log:
[...truncated 20568 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:92: The following files contain @author tags, tabs or nocommits:
* solr/core/src/test/org/apache/solr/handler/TestReplicationHandler.java
Total time: 68 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_55 -server -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (32bit/ibm-j9-jdk7) - Build #
10255 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10255/
Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
All tests passed
Build Log:
[...truncated 20469 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:92: The following files contain @author tags, tabs or nocommits:
* solr/core/src/test/org/apache/solr/handler/TestReplicationHandler.java
Total time: 70 minutes 29 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_55) - Build #
10254 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10254/
Java: 64bit/jdk1.7.0_55 -XX:+UseCompressedOops -XX:+UseParallelGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.LeaderElectionIntegrationTest.testSimpleSliceLeaderElection
Error Message:
No registered leader was found after waiting for 30000ms , collection: collection1 slice: shard1
Stack Trace:
org.apache.solr.common.SolrException: No registered leader was found after waiting for 30000ms , collection: collection1 slice: shard1
at __randomizedtesting.SeedInfo.seed([C14376E1B583D6BB:9F77BBAFF6FC4A5C]:0)
at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:567)
at org.apache.solr.cloud.LeaderElectionIntegrationTest.getLeader(LeaderElectionIntegrationTest.java:255)
at org.apache.solr.cloud.LeaderElectionIntegrationTest.testSimpleSliceLeaderElection(LeaderElectionIntegrationTest.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:793)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:453)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:360)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11196 lines...]
[junit4] Suite: org.apache.solr.cloud.LeaderElectionIntegrationTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/init-core-data-001
[junit4] 2> 903247 T3830 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
[junit4] 2> 903251 T3830 oas.SolrTestCaseJ4.setUp ###Starting testSimpleSliceLeaderElection
[junit4] 2> 903252 T3830 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 903252 T3831 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 903352 T3830 oasc.ZkTestServer.run start zk server on port:58550
[junit4] 2> 903353 T3830 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 906590 T3834 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 3236ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 906592 T3837 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3aea16e name:ZooKeeperConnection Watcher:127.0.0.1:58550 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 906592 T3830 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 906592 T3830 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 906596 T3830 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 906597 T3839 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3602ffab name:ZooKeeperConnection Watcher:127.0.0.1:58550/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 906597 T3830 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 906597 T3830 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 906601 T3830 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 906604 T3830 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 906605 T3830 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 906606 T3830 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 906607 T3830 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 906609 T3830 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 906610 T3830 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 906612 T3830 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 906612 T3830 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 906614 T3830 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 906614 T3830 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 906615 T3830 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 906615 T3830 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 906617 T3830 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 906617 T3830 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 906619 T3830 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 906619 T3830 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 906621 T3830 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 906621 T3830 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 906622 T3830 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 906623 T3830 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 906624 T3830 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 906625 T3830 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 906626 T3830 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 906626 T3830 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 906628 T3830 oasc.LeaderElectionIntegrationTest.setUp ####SETUP_START testSimpleSliceLeaderElection
[junit4] 2> 906629 T3830 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 906629 T3830 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr
[junit4] 2> 906629 T3830 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/'
[junit4] 2> 906639 T3830 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/solr.xml
[junit4] 2> 906672 T3830 oasc.CoreContainer.<init> New CoreContainer 1743319407
[junit4] 2> 906672 T3830 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/]
[junit4] 2> 906673 T3830 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 906673 T3830 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: https
[junit4] 2> 906674 T3830 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 906674 T3830 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 906674 T3830 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 906674 T3830 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 906674 T3830 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 906674 T3830 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 906675 T3830 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 906677 T3830 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 906677 T3830 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 906677 T3830 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 906677 T3830 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58550/solr
[junit4] 2> 906677 T3830 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 906678 T3830 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 906680 T3841 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@327c0fc6 name:ZooKeeperConnection Watcher:127.0.0.1:58550 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 906680 T3830 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 906682 T3830 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 906683 T3843 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@36094cf2 name:ZooKeeperConnection Watcher:127.0.0.1:58550/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 906683 T3830 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 906684 T3830 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 906686 T3830 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 906687 T3830 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 906689 T3830 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 906691 T3830 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 906692 T3830 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 906693 T3830 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7000_solr
[junit4] 2> 906694 T3830 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7000_solr
[junit4] 2> 906695 T3830 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 906696 T3830 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 906698 T3830 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:7000_solr
[junit4] 2> 906698 T3830 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 906699 T3830 oasc.Overseer.start Overseer (id=91716089152077827-127.0.0.1:7000_solr-n_0000000000) starting
[junit4] 2> 906704 T3830 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 906710 T3845 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 906710 T3830 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 906710 T3845 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes at 127.0.0.1:7000_solr
[junit4] 2> 906711 T3830 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 906712 T3830 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 906713 T3844 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 906717 T3846 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 906717 T3846 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 906718 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 906718 T3846 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 906719 T3844 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 906719 T3844 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:7000/solr",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:7000_solr",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":null,
[junit4] 2> "core_node_name":null}
[junit4] 2> 906721 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 906721 T3843 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 907718 T3846 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 907719 T3846 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 907719 T3846 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 907720 T3846 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 907721 T3846 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 907721 T3846 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/'
[junit4] 2> 907722 T3846 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
[junit4] 2> 907722 T3846 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
[junit4] 2> 907722 T3846 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/README' to classloader
[junit4] 2> 907764 T3846 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 907816 T3846 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 907817 T3846 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 907822 T3846 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 908078 T3846 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 908082 T3846 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 908084 T3846 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 908092 T3846 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 908095 T3846 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 908097 T3846 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 908098 T3846 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 908098 T3846 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 908098 T3846 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 908099 T3846 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 908099 T3846 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 908099 T3846 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 908099 T3846 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 908100 T3846 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-001/
[junit4] 2> 908100 T3846 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bf3f9d
[junit4] 2> 908100 T3846 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-001
[junit4] 2> 908101 T3846 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-001/index/
[junit4] 2> 908101 T3846 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-001/index' doesn't exist. Creating new index...
[junit4] 2> 908101 T3846 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-001/index
[junit4] 2> 908101 T3846 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=24, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=50.537109375, floorSegmentMB=0.7001953125, forceMergeDeletesPctAllowed=6.83503378867266, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 908102 T3846 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1b9f8f3e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@58af6268)),segFN=segments_1,generation=1}
[junit4] 2> 908102 T3846 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 908102 T3846 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4] 2> 908105 T3846 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 908105 T3846 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 908105 T3846 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 908105 T3846 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 908105 T3846 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 908106 T3846 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 908106 T3846 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
[junit4] 2> 908106 T3846 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4] 2> 908106 T3846 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 908106 T3846 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 908107 T3846 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 908107 T3846 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 908107 T3846 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 908107 T3846 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 908107 T3846 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 908108 T3846 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 908108 T3846 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4] 2> 908108 T3846 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4] 2> 908109 T3846 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 908109 T3846 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 908109 T3846 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 908109 T3846 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4] 2> 908110 T3846 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4] 2> 908110 T3846 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4] 2> 908110 T3846 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 908110 T3846 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 908111 T3846 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 908111 T3846 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 908111 T3846 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 908111 T3846 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 908112 T3846 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 908112 T3846 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 908112 T3846 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 908113 T3846 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4] 2> 908113 T3846 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4] 2> 908115 T3846 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 908116 T3846 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4] 2> 908117 T3846 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 908118 T3846 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 908118 T3846 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=37, maxMergedSegmentMB=27.5966796875, floorSegmentMB=1.4912109375, forceMergeDeletesPctAllowed=11.041327110877518, segmentsPerTier=28.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 908118 T3846 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1b9f8f3e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@58af6268)),segFN=segments_1,generation=1}
[junit4] 2> 908119 T3846 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 908119 T3846 oass.SolrIndexSearcher.<init> Opening Searcher@6b011d9f[collection1] main
[junit4] 2> 908119 T3846 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 908120 T3846 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 908120 T3846 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
[junit4] 2> 908120 T3846 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
[junit4] 2> 908120 T3846 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
[junit4] 2> 908121 T3846 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 908121 T3846 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 908121 T3846 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4] 2> 908125 T3846 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 908147 T3847 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6b011d9f[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 908150 T3846 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 908151 T3850 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:7000/solr collection:collection1 shard:shard1
[junit4] 2> 908151 T3850 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 908152 T3830 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 908152 T3830 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr
[junit4] 2> 908152 T3830 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/'
[junit4] 2> 908155 T3850 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 908157 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 908157 T3850 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 908157 T3850 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 908157 T3850 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:7000/solr/collection1/ shard1
[junit4] 2> 908158 T3844 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 908158 T3850 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 908160 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 908161 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 908163 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 908163 T3830 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/solr.xml
[junit4] 2> 908197 T3830 oasc.CoreContainer.<init> New CoreContainer 418903141
[junit4] 2> 908197 T3830 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/]
[junit4] 2> 908198 T3830 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 908198 T3830 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: https
[junit4] 2> 908198 T3830 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 908198 T3830 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 908198 T3830 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 908199 T3830 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 908199 T3830 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 908199 T3830 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 908199 T3830 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 908201 T3830 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 908201 T3830 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 908201 T3830 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 908202 T3830 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58550/solr
[junit4] 2> 908202 T3830 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 908202 T3830 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 908205 T3852 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1640bad7 name:ZooKeeperConnection Watcher:127.0.0.1:58550 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 908205 T3830 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 908206 T3830 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 908207 T3854 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46393f65 name:ZooKeeperConnection Watcher:127.0.0.1:58550/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 908208 T3830 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 908213 T3830 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 908265 T3854 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 908265 T3843 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 908312 T3850 oasc.ZkController.register We are http://127.0.0.1:7000/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4] 2> 908313 T3850 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:7000/solr
[junit4] 2> 908313 T3850 oasc.ZkController.checkRecovery WARN Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4] 2> 908313 T3850 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 908313 T3850 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 908314 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 908315 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 908315 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 908315 T3850 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 908316 T3844 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 908317 T3844 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:7000/solr",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:7000_solr",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":null,
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 908318 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 908420 T3843 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 908420 T3854 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 909215 T3830 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7001_solr
[junit4] 2> 909217 T3830 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7001_solr
[junit4] 2> 909228 T3855 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 909228 T3855 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 909231 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 909231 T3855 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 909232 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 909232 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 909233 T3844 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 909234 T3844 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:7001/solr",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:7001_solr",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":null,
[junit4] 2> "core_node_name":null}
[junit4] 2> 909238 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 909340 T3843 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 909340 T3854 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 910232 T3855 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 910232 T3855 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 910233 T3855 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 910233 T3855 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 910234 T3855 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 910234 T3855 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/'
[junit4] 2> 910235 T3855 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
[junit4] 2> 910235 T3855 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
[junit4] 2> 910235 T3855 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/README' to classloader
[junit4] 2> 910272 T3855 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 910317 T3855 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 910318 T3855 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 910324 T3855 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 910593 T3855 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 910598 T3855 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 910600 T3855 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 910608 T3855 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 910612 T3855 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 910616 T3855 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 910617 T3855 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 910617 T3855 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 910617 T3855 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 910618 T3855 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 910618 T3855 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 910618 T3855 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 910619 T3855 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 910619 T3855 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-002/
[junit4] 2> 910619 T3855 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bf3f9d
[junit4] 2> 910620 T3855 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-002
[junit4] 2> 910620 T3855 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-002/index/
[junit4] 2> 910620 T3855 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-002/index' doesn't exist. Creating new index...
[junit4] 2> 910621 T3855 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-002/index
[junit4] 2> 910621 T3855 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=24, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=50.537109375, floorSegmentMB=0.7001953125, forceMergeDeletesPctAllowed=6.83503378867266, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 910622 T3855 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@13de788c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@ba99bfa)),segFN=segments_1,generation=1}
[junit4] 2> 910622 T3855 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 910623 T3855 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4] 2> 910627 T3855 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 910628 T3855 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 910628 T3855 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 910628 T3855 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 910628 T3855 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 910629 T3855 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 910629 T3855 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
[junit4] 2> 910629 T3855 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4] 2> 910629 T3855 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 910630 T3855 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 910630 T3855 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 910630 T3855 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 910631 T3855 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 910631 T3855 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 910631 T3855 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 910632 T3855 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 910633 T3855 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4] 2> 910633 T3855 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4] 2> 910633 T3855 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 910634 T3855 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 910635 T3855 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 910635 T3855 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4] 2> 910635 T3855 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4] 2> 910636 T3855 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4] 2> 910636 T3855 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 910637 T3855 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 910637 T3855 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 910638 T3855 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 910638 T3855 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 910639 T3855 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 910639 T3855 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 910640 T3855 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 910640 T3855 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 910641 T3855 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4] 2> 910641 T3855 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4] 2> 910643 T3855 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 910644 T3855 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4] 2> 910646 T3855 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 910646 T3855 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 910646 T3855 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=37, maxMergedSegmentMB=27.5966796875, floorSegmentMB=1.4912109375, forceMergeDeletesPctAllowed=11.041327110877518, segmentsPerTier=28.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 910647 T3855 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@13de788c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@ba99bfa)),segFN=segments_1,generation=1}
[junit4] 2> 910647 T3855 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 910647 T3855 oass.SolrIndexSearcher.<init> Opening Searcher@6d3fe2b0[collection1] main
[junit4] 2> 910648 T3855 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 910648 T3855 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 910649 T3855 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
[junit4] 2> 910649 T3855 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
[junit4] 2> 910649 T3855 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
[junit4] 2> 910649 T3855 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 910649 T3855 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 910649 T3855 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4] 2> 910653 T3855 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 910673 T3856 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6d3fe2b0[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 910678 T3855 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 910679 T3859 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:7001/solr collection:collection1 shard:shard1
[junit4] 2> 910680 T3830 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 910680 T3830 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr
[junit4] 2> 910680 T3830 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/'
[junit4] 2> 910681 T3859 oasc.ZkController.register We are http://127.0.0.1:7001/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4] 2> 910682 T3859 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:7001/solr
[junit4] 2> 910682 T3859 oasc.ZkController.checkRecovery WARN Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4] 2> 910682 T3859 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 910682 T3859 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 910683 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 910683 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 910683 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 910683 T3859 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 910684 T3844 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 910685 T3844 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:7001/solr",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:7001_solr",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":null,
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 910686 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 910690 T3830 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/solr.xml
[junit4] 2> 910722 T3830 oasc.CoreContainer.<init> New CoreContainer 1050308606
[junit4] 2> 910723 T3830 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/]
[junit4] 2> 910724 T3830 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 910724 T3830 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: https
[junit4] 2> 910724 T3830 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 910724 T3830 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 910724 T3830 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 910724 T3830 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 910725 T3830 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 910725 T3830 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 910725 T3830 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 910727 T3830 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 910727 T3830 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 910727 T3830 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 910728 T3830 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58550/solr
[junit4] 2> 910728 T3830 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 910728 T3830 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 910730 T3861 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3973cd5a name:ZooKeeperConnection Watcher:127.0.0.1:58550 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 910730 T3830 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 910731 T3830 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 910732 T3863 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@27a084ee name:ZooKeeperConnection Watcher:127.0.0.1:58550/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 910732 T3830 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 910736 T3830 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 910788 T3843 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 910788 T3854 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 910788 T3863 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 911738 T3830 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7002_solr
[junit4] 2> 911739 T3830 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7002_solr
[junit4] 2> 917074 T3832 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x145d74c12190003, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 917075 T3832 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x145d74c12190005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 917076 T3832 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x145d74c12190007, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 917174 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 917175 T3843 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@36094cf2 name:ZooKeeperConnection Watcher:127.0.0.1:58550/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 917175 T3843 oascc.ConnectionManager.process zkClient has disconnected
[junit4] 2> 917175 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 917175 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 917175 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: Disconnected type None
[junit4] 2> 917175 T3854 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46393f65 name:ZooKeeperConnection Watcher:127.0.0.1:58550/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 917175 T3854 oascc.ConnectionManager.process zkClient has disconnected
[junit4] 2> 917177 T3863 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@27a084ee name:ZooKeeperConnection Watcher:127.0.0.1:58550/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4] 2> 917177 T3863 oascc.ConnectionManager.process zkClient has disconnected
[junit4] 2> 918269 T3863 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@27a084ee name:ZooKeeperConnection Watcher:127.0.0.1:58550/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 918325 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 918325 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 918325 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 918325 T3843 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@36094cf2 name:ZooKeeperConnection Watcher:127.0.0.1:58550/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 918325 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: null state: SyncConnected type None
[junit4] 2> 918593 T3854 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46393f65 name:ZooKeeperConnection Watcher:127.0.0.1:58550/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 919980 T3834 oazsp.FileTxnLog.commit WARN fsync-ing the write ahead log in SyncThread:0 took 8239ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[junit4] 2> 919982 T3834 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:404)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:200)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131)
[junit4] 2>
[junit4] 2> 919982 T3834 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:170)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:169)
[junit4] 2>
[junit4] 2> 919983 T3834 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:170)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:169)
[junit4] 2>
[junit4] 2> 919983 T3834 oazs.NIOServerCnxn.sendBuffer ERROR Unexpected Exception: java.nio.channels.CancelledKeyException
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
[junit4] 2> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1081)
[junit4] 2> at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:170)
[junit4] 2> at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:169)
[junit4] 2>
[junit4] 2> 919989 T3864 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 919989 T3864 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 919990 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 919990 T3864 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 919990 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 919991 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 919991 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 919991 T3844 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 919992 T3844 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:7002/solr",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:7002_solr",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":null,
[junit4] 2> "core_node_name":null}
[junit4] 2> 919993 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 919993 T3843 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 919994 T3863 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 919993 T3854 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 920991 T3864 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 920991 T3864 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 920992 T3864 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 920992 T3864 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 920994 T3864 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 920994 T3864 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/'
[junit4] 2> 920995 T3864 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
[junit4] 2> 920995 T3864 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
[junit4] 2> 920996 T3864 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/README' to classloader
[junit4] 2> 921051 T3864 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 921096 T3864 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 921098 T3864 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 921103 T3864 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 921359 T3864 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 921364 T3864 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 921366 T3864 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 921380 T3864 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 921383 T3864 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 921386 T3864 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 921387 T3864 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 921387 T3864 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 921388 T3864 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 921388 T3864 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 921388 T3864 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 921389 T3864 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 921389 T3864 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 921389 T3864 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-003/
[junit4] 2> 921389 T3864 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bf3f9d
[junit4] 2> 921390 T3864 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-003
[junit4] 2> 921390 T3864 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-003/index/
[junit4] 2> 921390 T3864 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-003/index' doesn't exist. Creating new index...
[junit4] 2> 921391 T3864 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-003/index
[junit4] 2> 921391 T3864 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=24, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=50.537109375, floorSegmentMB=0.7001953125, forceMergeDeletesPctAllowed=6.83503378867266, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 921392 T3864 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@58cddfa3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5962eb26)),segFN=segments_1,generation=1}
[junit4] 2> 921392 T3864 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 921392 T3864 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4] 2> 921396 T3864 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 921396 T3864 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 921396 T3864 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 921396 T3864 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 921397 T3864 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 921397 T3864 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 921397 T3864 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
[junit4] 2> 921397 T3864 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4] 2> 921398 T3864 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 921398 T3864 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 921398 T3864 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 921398 T3864 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 921399 T3864 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 921399 T3864 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 921399 T3864 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 921400 T3864 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 921400 T3864 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4] 2> 921401 T3864 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4] 2> 921401 T3864 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 921402 T3864 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 921402 T3864 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 921402 T3864 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4] 2> 921403 T3864 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4] 2> 921403 T3864 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4] 2> 921403 T3864 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 921404 T3864 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 921404 T3864 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 921404 T3864 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 921405 T3864 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 921405 T3864 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 921405 T3864 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 921406 T3864 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 921406 T3864 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 921407 T3864 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4] 2> 921407 T3864 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4] 2> 921409 T3864 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 921410 T3864 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4] 2> 921412 T3864 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 921412 T3864 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 921413 T3864 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=37, maxMergedSegmentMB=27.5966796875, floorSegmentMB=1.4912109375, forceMergeDeletesPctAllowed=11.041327110877518, segmentsPerTier=28.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 921413 T3864 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@58cddfa3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5962eb26)),segFN=segments_1,generation=1}
[junit4] 2> 921413 T3864 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 921413 T3864 oass.SolrIndexSearcher.<init> Opening Searcher@772bfefa[collection1] main
[junit4] 2> 921414 T3864 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 921414 T3864 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 921415 T3864 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
[junit4] 2> 921415 T3864 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
[junit4] 2> 921415 T3864 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
[junit4] 2> 921415 T3864 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
[junit4] 2> 921415 T3864 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 921415 T3864 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4] 2> 921422 T3864 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 921442 T3865 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@772bfefa[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 921447 T3864 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 921448 T3868 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:7002/solr collection:collection1 shard:shard1
[junit4] 2> 921449 T3830 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 921449 T3830 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr
[junit4] 2> 921450 T3830 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/'
[junit4] 2> 921450 T3868 oasc.ZkController.register We are http://127.0.0.1:7002/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4] 2> 921450 T3868 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:7002/solr
[junit4] 2> 921450 T3868 oasc.ZkController.checkRecovery WARN Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4] 2> 921450 T3868 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 921450 T3868 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 921451 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 921451 T3868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 921452 T3844 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 921452 T3844 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:7002/solr",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:7002_solr",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":null,
[junit4] 2> "core_node_name":"core_node3"}
[junit4] 2> 921454 T3843 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 921460 T3830 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/solr.xml
[junit4] 2> 921493 T3830 oasc.CoreContainer.<init> New CoreContainer 584441044
[junit4] 2> 921494 T3830 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/]
[junit4] 2> 921494 T3830 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 921495 T3830 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: https
[junit4] 2> 921495 T3830 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 921495 T3830 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 921495 T3830 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 921495 T3830 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 921495 T3830 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 921496 T3830 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 921496 T3830 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 921498 T3830 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 921498 T3830 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 921498 T3830 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 921498 T3830 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58550/solr
[junit4] 2> 921499 T3830 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 921499 T3830 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 921501 T3870 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3898d4c8 name:ZooKeeperConnection Watcher:127.0.0.1:58550 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 921501 T3830 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 921502 T3830 oascc.ConnectionManager.waitForConnected Waiting for client to connect to
[...truncated too long message...]
2> "collection":"collection1",
[junit4] 2> "numShards":null,
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 988084 T3830 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
[junit4] 2> 988084 T3830 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 988084 T3830 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 988084 T3830 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 988085 T3830 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 988085 T3830 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 988085 T3830 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-007/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-007/index;done=false>>]
[junit4] 2> 988086 T3972 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 988086 T3830 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-007/index
[junit4] 2> 988086 T3830 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-007 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-007;done=false>>]
[junit4] 2> 988086 T3830 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001/tempDir-007
[junit4] 2> 988087 T3974 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:7000_solr
[junit4] 2> 988088 T3830 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:52692 52692
[junit4] 2> 989587 T3972 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 989587 T3973 oasc.SolrException.log ERROR Error registering SolrCore:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /live_nodes
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:437)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.updateClusterState(ZkStateReader.java:242)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:827)
[junit4] 2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:253)
[junit4] 2> at org.apache.solr.common.cloud.ConnectionManager$1$1.run(ConnectionManager.java:166)
[junit4] 2>
[junit4] 2> 989587 T3976 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:306)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
[junit4] 2>
[junit4] 2> 998285 T3902 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:52692 52692
[junit4] 2> 998289 T3830 oas.SolrTestCaseJ4.tearDown ###Ending testLeaderElectionAfterClientTimeout
[junit4] 2> 1000292 T3830 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solr.cloud.LeaderElectionIntegrationTest-C14376E1B583D6BB-001
[junit4] 2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=es_VE, timezone=Australia/Melbourne
[junit4] 2> NOTE: Linux 3.8.0-39-generic amd64/Oracle Corporation 1.7.0_55 (64-bit)/cpus=8,threads=1,free=164801192,total=371195904
[junit4] 2> NOTE: All tests run in this JVM: [TestSimpleQParserPlugin, TestDefaultSimilarityFactory, TestPseudoReturnFields, TestCharFilters, SolrInfoMBeanTest, TestDocBasedVersionConstraints, TestCopyFieldCollectionResource, TestInitQParser, PrimitiveFieldTypeTest, TestSort, DistributedQueryElevationComponentTest, WordBreakSolrSpellCheckerTest, AutoCommitTest, SOLR749Test, AbstractAnalyticsFacetTest, QueryEqualityTest, AnalysisErrorHandlingTest, FieldMutatingUpdateProcessorTest, DistribCursorPagingTest, UUIDFieldTest, QueryElevationComponentTest, RecoveryZkTest, TestSolrXmlPersistence, SpellPossibilityIteratorTest, TestDocSet, TestReplicationHandler, TestWriterPerf, TestSolrQueryParser, TestPerFieldSimilarity, TestTrie, TestShardHandlerFactory, OverseerTest, FieldFacetExtrasTest, TestAnalyzeInfixSuggestions, CursorMarkTest, PreAnalyzedFieldTest, ChaosMonkeyNothingIsSafeTest, SuggestComponentTest, TestLuceneMatchVersion, SliceStateTest, DistributedSpellCheckComponentTest, TestFaceting, TestSearchPerf, TestManagedSynonymFilterFactory, CSVRequestHandlerTest, TestFieldResource, TestHighlightDedupGrouping, TestSurroundQueryParser, TestSchemaVersionResource, TestStressVersions, PreAnalyzedUpdateProcessorTest, SpellingQueryConverterTest, MBeansHandlerTest, SimpleFacetsTest, TestConfig, TestFileDictionaryLookup, ZkControllerTest, TestJmxMonitoredMap, UnloadDistributedZkTest, TestBadConfig, PeerSyncTest, TestUtils, QueryFacetTest, ZkCLITest, BinaryUpdateRequestHandlerTest, TestFieldTypeResource, SearchHandlerTest, TestFieldSortValues, TestRecoveryHdfs, TestCoreContainer, DistributedQueryComponentCustomSortTest, ExternalFileFieldSortTest, TestRestManager, ChangedSchemaMergeTest, CopyFieldTest, SuggesterTest, TestSearcherReuse, TestSolrXmlPersistor, TestClassNameShortening, RAMDirectoryFactoryTest, CacheHeaderTest, EchoParamsTest, SoftAutoCommitTest, TestRequestStatusCollectionAPI, SchemaVersionSpecificBehaviorTest, TestDistributedMissingSort, ShardSplitTest, TestRemoteStreaming, TestStandardQParsers, TestValueSourceCache, ModifyConfFileTest, QueryParsingTest, ScriptEngineTest, UpdateParamsTest, BasicDistributedZkTest, LeaderElectionIntegrationTest]
[junit4] Completed on J0 in 97.06s, 2 tests, 1 error <<< FAILURES!
[...truncated 608 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1296: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:920: There were test failures: 398 suites, 1673 tests, 1 error, 37 ignored (16 assumptions)
Total time: 57 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_55 -XX:+UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure