You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/06/22 11:19:11 UTC
[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.6.0_45) - Build # 6181
- Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6181/
Java: 64bit/jdk1.6.0_45 -XX:+UseCompressedOops -XX:+UseParallelGC
2 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
shard3 is not consistent. Got 54 from http://127.0.0.1:53424/zbo/zd/collection1lastClient and got 52 from http://127.0.0.1:39945/zbo/zd/collection1
Stack Trace:
java.lang.AssertionError: shard3 is not consistent. Got 54 from http://127.0.0.1:53424/zbo/zd/collection1lastClient and got 52 from http://127.0.0.1:39945/zbo/zd/collection1
at __randomizedtesting.SeedInfo.seed([947A66B1BF3424BD:159CE8A9C86B4481]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:137)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:662)
REGRESSION: org.apache.solr.cloud.RecoveryZkTest.testDistribSearch
Error Message:
expected:<194> but was:<52>
Stack Trace:
java.lang.AssertionError: expected:<194> but was:<52>
at __randomizedtesting.SeedInfo.seed([947A66B1BF3424BD:159CE8A9C86B4481]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.RecoveryZkTest.doTest(RecoveryZkTest.java:108)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:662)
Build Log:
[...truncated 9004 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4:junit4] 2> 354772 T743 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /zbo/zd
[junit4:junit4] 2> 354779 T743 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeySafeLeaderTest-1371891900957
[junit4:junit4] 2> 354780 T743 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 354781 T744 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 354881 T743 oasc.ZkTestServer.run start zk server on port:44356
[junit4:junit4] 2> 354882 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 354892 T750 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@20311fb2 name:ZooKeeperConnection Watcher:127.0.0.1:44356 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 354892 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 354893 T743 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 354897 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 354898 T752 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ccf26f3 name:ZooKeeperConnection Watcher:127.0.0.1:44356/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 354899 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 354899 T743 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 354903 T743 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 354906 T743 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 354909 T743 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 354912 T743 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 354912 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 354917 T743 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 354917 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 354921 T743 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 354922 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 354925 T743 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 354925 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 354928 T743 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 354929 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 354932 T743 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 354933 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 354936 T743 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 354937 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 354940 T743 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 354941 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 354943 T743 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 354944 T743 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 355016 T743 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 355019 T743 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49744
[junit4:junit4] 2> 355019 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 355020 T743 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 355020 T743 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371891901125
[junit4:junit4] 2> 355020 T743 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371891901125/solr.xml
[junit4:junit4] 2> 355021 T743 oasc.CoreContainer.<init> New CoreContainer 157625754
[junit4:junit4] 2> 355021 T743 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371891901125/'
[junit4:junit4] 2> 355021 T743 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371891901125/'
[junit4:junit4] 2> 355061 T743 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 355061 T743 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 355061 T743 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 355062 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 355062 T743 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 355062 T743 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 355063 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 355063 T743 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 355063 T743 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 355064 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 355067 T743 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 355067 T743 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44356/solr
[junit4:junit4] 2> 355068 T743 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 355069 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 355071 T763 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6eedd5e4 name:ZooKeeperConnection Watcher:127.0.0.1:44356 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 355071 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 355073 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 355077 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 355079 T765 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d79045 name:ZooKeeperConnection Watcher:127.0.0.1:44356/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 355079 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 355091 T743 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 355095 T743 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 355099 T743 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 355101 T743 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49744_zbo%2Fzd
[junit4:junit4] 2> 355102 T743 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49744_zbo%2Fzd
[junit4:junit4] 2> 355106 T743 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 355113 T743 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 355116 T743 oasc.Overseer.start Overseer (id=89908307621314563-127.0.0.1:49744_zbo%2Fzd-n_0000000000) starting
[junit4:junit4] 2> 355120 T743 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 355124 T767 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 355124 T743 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 355128 T743 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 355131 T743 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 355134 T766 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 355136 T768 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 355136 T768 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 356639 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 356640 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49744_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49744/zbo/zd"}
[junit4:junit4] 2> 356640 T766 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 356640 T766 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 356644 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 357138 T768 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371891901125/collection1
[junit4:junit4] 2> 357138 T768 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 357139 T768 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 357139 T768 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 357141 T768 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371891901125/collection1/'
[junit4:junit4] 2> 357141 T768 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371891901125/collection1/lib/README' to classloader
[junit4:junit4] 2> 357141 T768 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371891901125/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 357171 T768 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 357200 T768 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 357202 T768 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 357205 T768 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 357518 T768 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 357518 T768 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 357519 T768 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 357525 T768 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 357527 T768 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 357543 T768 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 357546 T768 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 357549 T768 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 357550 T768 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 357550 T768 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 357550 T768 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 357552 T768 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 357552 T768 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 357552 T768 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 357558 T768 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371891901125/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/control/data/
[junit4:junit4] 2> 357558 T768 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@402fb002
[junit4:junit4] 2> 357559 T768 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 357559 T768 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/control/data
[junit4:junit4] 2> 357559 T768 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/control/data/index/
[junit4:junit4] 2> 357560 T768 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 357560 T768 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/control/data/index
[junit4:junit4] 2> 357563 T768 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 357563 T768 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 357565 T768 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 357565 T768 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 357566 T768 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 357566 T768 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 357567 T768 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 357567 T768 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 357567 T768 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 357567 T768 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 357568 T768 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 357569 T768 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 357572 T768 oass.SolrIndexSearcher.<init> Opening Searcher@5918a58d main
[junit4:junit4] 2> 357573 T768 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 357573 T768 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 357577 T769 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5918a58d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 357579 T768 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 357579 T768 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49744/zbo/zd collection:control_collection shard:shard1
[junit4:junit4] 2> 357580 T768 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 357588 T768 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 357592 T768 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 357592 T768 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 357592 T768 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49744/zbo/zd/collection1/
[junit4:junit4] 2> 357593 T768 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 357593 T768 oasc.SyncStrategy.syncToMe http://127.0.0.1:49744/zbo/zd/collection1/ has no replicas
[junit4:junit4] 2> 357593 T768 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49744/zbo/zd/collection1/
[junit4:junit4] 2> 357593 T768 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 358149 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 358157 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 358203 T768 oasc.ZkController.register We are http://127.0.0.1:49744/zbo/zd/collection1/ and leader is http://127.0.0.1:49744/zbo/zd/collection1/
[junit4:junit4] 2> 358204 T768 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49744/zbo/zd
[junit4:junit4] 2> 358204 T768 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 358204 T768 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 358205 T768 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 358207 T768 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 358208 T743 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 358208 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 358209 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 358212 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 358213 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 358214 T772 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17350bd2 name:ZooKeeperConnection Watcher:127.0.0.1:44356/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 358214 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 358216 T743 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 358220 T743 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 358282 T743 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 358284 T743 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53383
[junit4:junit4] 2> 358285 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 358285 T743 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 358286 T743 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371891904397
[junit4:junit4] 2> 358286 T743 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371891904397/solr.xml
[junit4:junit4] 2> 358286 T743 oasc.CoreContainer.<init> New CoreContainer 1761762876
[junit4:junit4] 2> 358287 T743 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371891904397/'
[junit4:junit4] 2> 358287 T743 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371891904397/'
[junit4:junit4] 2> 358326 T743 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 358326 T743 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 358327 T743 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 358327 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 358327 T743 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 358327 T743 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 358328 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 358328 T743 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 358328 T743 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 358329 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 358332 T743 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 358332 T743 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44356/solr
[junit4:junit4] 2> 358333 T743 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 358333 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 358335 T783 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@788c5a93 name:ZooKeeperConnection Watcher:127.0.0.1:44356 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 358335 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 358338 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 358341 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 358342 T785 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3929f1c name:ZooKeeperConnection Watcher:127.0.0.1:44356/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 358343 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 358347 T743 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 359351 T743 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53383_zbo%2Fzd
[junit4:junit4] 2> 359353 T743 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53383_zbo%2Fzd
[junit4:junit4] 2> 359358 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 359359 T772 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 359360 T785 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 359362 T765 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 359367 T786 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 359367 T786 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 359664 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 359665 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49744_zbo%2Fzd_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49744_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49744/zbo/zd"}
[junit4:junit4] 2> 359669 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:53383_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:53383/zbo/zd"}
[junit4:junit4] 2> 359669 T766 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 3
[junit4:junit4] 2> 359669 T766 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 359673 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 359674 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 359674 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 360368 T786 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371891904397/collection1
[junit4:junit4] 2> 360369 T786 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 360370 T786 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 360370 T786 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 360372 T786 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371891904397/collection1/'
[junit4:junit4] 2> 360372 T786 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371891904397/collection1/lib/README' to classloader
[junit4:junit4] 2> 360372 T786 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371891904397/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 360396 T786 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 360431 T786 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 360433 T786 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 360436 T786 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 360989 T786 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 360990 T786 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 360990 T786 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 360996 T786 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 360998 T786 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 361014 T786 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 361017 T786 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 361020 T786 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 361021 T786 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 361021 T786 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 361021 T786 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 361022 T786 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 361023 T786 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 361023 T786 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 361023 T786 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371891904397/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty1/
[junit4:junit4] 2> 361023 T786 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@402fb002
[junit4:junit4] 2> 361024 T786 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 361024 T786 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty1
[junit4:junit4] 2> 361024 T786 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty1/index/
[junit4:junit4] 2> 361025 T786 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 361025 T786 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty1/index
[junit4:junit4] 2> 361057 T786 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 361058 T786 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 361059 T786 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 361060 T786 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 361060 T786 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 361061 T786 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 361061 T786 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 361061 T786 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 361061 T786 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 361062 T786 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 361062 T786 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 361064 T786 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 361067 T786 oass.SolrIndexSearcher.<init> Opening Searcher@3ee8ee3b main
[junit4:junit4] 2> 361067 T786 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 361067 T786 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 361074 T787 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3ee8ee3b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 361078 T786 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 361078 T786 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:53383/zbo/zd collection:collection1 shard:shard1
[junit4:junit4] 2> 361080 T786 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 361089 T786 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 361095 T786 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 361095 T786 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 361096 T786 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:53383/zbo/zd/collection1/
[junit4:junit4] 2> 361096 T786 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 361096 T786 oasc.SyncStrategy.syncToMe http://127.0.0.1:53383/zbo/zd/collection1/ has no replicas
[junit4:junit4] 2> 361097 T786 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:53383/zbo/zd/collection1/
[junit4:junit4] 2> 361097 T786 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 361181 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 361190 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 361190 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 361190 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 361208 T786 oasc.ZkController.register We are http://127.0.0.1:53383/zbo/zd/collection1/ and leader is http://127.0.0.1:53383/zbo/zd/collection1/
[junit4:junit4] 2> 361208 T786 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53383/zbo/zd
[junit4:junit4] 2> 361208 T786 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 361208 T786 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 361208 T786 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 361210 T786 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 361211 T743 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 361211 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 361212 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 361279 T743 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 361285 T743 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:35304
[junit4:junit4] 2> 361286 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 361286 T743 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 361287 T743 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371891907392
[junit4:junit4] 2> 361287 T743 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371891907392/solr.xml
[junit4:junit4] 2> 361287 T743 oasc.CoreContainer.<init> New CoreContainer 742631224
[junit4:junit4] 2> 361288 T743 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371891907392/'
[junit4:junit4] 2> 361288 T743 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371891907392/'
[junit4:junit4] 2> 361333 T743 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 361333 T743 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 361334 T743 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 361334 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 361334 T743 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 361335 T743 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 361335 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 361335 T743 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 361336 T743 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 361336 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 361339 T743 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 361340 T743 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44356/solr
[junit4:junit4] 2> 361340 T743 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 361341 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 361343 T799 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4e35a61c name:ZooKeeperConnection Watcher:127.0.0.1:44356 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 361343 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 361345 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 361349 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 361350 T801 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3c2e1b2f name:ZooKeeperConnection Watcher:127.0.0.1:44356/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 361350 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 361355 T743 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 362359 T743 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35304_zbo%2Fzd
[junit4:junit4] 2> 362360 T743 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35304_zbo%2Fzd
[junit4:junit4] 2> 362363 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 362363 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 362363 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 362363 T801 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 362365 T772 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 362365 T765 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 362365 T785 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 362371 T802 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 362371 T802 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 362695 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 362696 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:53383_zbo%2Fzd_collection1",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:53383_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:53383/zbo/zd"}
[junit4:junit4] 2> 362700 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35304_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35304/zbo/zd"}
[junit4:junit4] 2> 362700 T766 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 362700 T766 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 362704 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 362704 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 362704 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 362704 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 363374 T802 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371891907392/collection1
[junit4:junit4] 2> 363374 T802 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 363375 T802 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 363375 T802 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 363377 T802 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371891907392/collection1/'
[junit4:junit4] 2> 363377 T802 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371891907392/collection1/lib/README' to classloader
[junit4:junit4] 2> 363378 T802 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371891907392/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 363401 T802 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 363430 T802 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 363432 T802 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 363435 T802 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 363751 T802 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 363751 T802 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 363752 T802 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 363759 T802 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 363762 T802 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 363776 T802 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 363780 T802 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 363783 T802 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 363784 T802 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 363784 T802 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 363784 T802 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 363785 T802 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 363785 T802 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 363786 T802 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 363786 T802 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371891907392/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty2/
[junit4:junit4] 2> 363786 T802 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@402fb002
[junit4:junit4] 2> 363787 T802 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 363787 T802 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty2
[junit4:junit4] 2> 363787 T802 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty2/index/
[junit4:junit4] 2> 363787 T802 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 363788 T802 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty2/index
[junit4:junit4] 2> 363790 T802 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 363791 T802 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 363793 T802 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 363793 T802 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 363793 T802 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 363794 T802 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 363794 T802 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 363795 T802 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 363795 T802 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 363795 T802 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 363796 T802 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 363797 T802 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 363800 T802 oass.SolrIndexSearcher.<init> Opening Searcher@14745f2d main
[junit4:junit4] 2> 363801 T802 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 363801 T802 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 363806 T803 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@14745f2d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 363808 T802 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 363808 T802 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35304/zbo/zd collection:collection1 shard:shard2
[junit4:junit4] 2> 363809 T802 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 363816 T802 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 363818 T802 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 363819 T802 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 363819 T802 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:35304/zbo/zd/collection1/
[junit4:junit4] 2> 363819 T802 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 363819 T802 oasc.SyncStrategy.syncToMe http://127.0.0.1:35304/zbo/zd/collection1/ has no replicas
[junit4:junit4] 2> 363819 T802 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:35304/zbo/zd/collection1/
[junit4:junit4] 2> 363820 T802 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 364212 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 364222 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 364222 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 364222 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 364222 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 364231 T802 oasc.ZkController.register We are http://127.0.0.1:35304/zbo/zd/collection1/ and leader is http://127.0.0.1:35304/zbo/zd/collection1/
[junit4:junit4] 2> 364231 T802 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35304/zbo/zd
[junit4:junit4] 2> 364232 T802 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 364232 T802 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 364232 T802 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 364234 T802 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 364235 T743 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 364235 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 364236 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 364302 T743 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 364304 T743 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:57819
[junit4:junit4] 2> 364305 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 364306 T743 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 364306 T743 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371891910416
[junit4:junit4] 2> 364306 T743 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371891910416/solr.xml
[junit4:junit4] 2> 364307 T743 oasc.CoreContainer.<init> New CoreContainer 1370239662
[junit4:junit4] 2> 364307 T743 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371891910416/'
[junit4:junit4] 2> 364307 T743 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371891910416/'
[junit4:junit4] 2> 364346 T743 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 364346 T743 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 364347 T743 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 364347 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 364347 T743 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 364348 T743 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 364348 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 364348 T743 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 364348 T743 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 364349 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 364352 T743 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 364352 T743 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44356/solr
[junit4:junit4] 2> 364353 T743 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 364354 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 364355 T815 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2ec959a4 name:ZooKeeperConnection Watcher:127.0.0.1:44356 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 364356 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 364358 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 364361 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 364363 T817 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5ddf50fe name:ZooKeeperConnection Watcher:127.0.0.1:44356/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 364363 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 364368 T743 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 365372 T743 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57819_zbo%2Fzd
[junit4:junit4] 2> 365374 T743 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57819_zbo%2Fzd
[junit4:junit4] 2> 365377 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 365377 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 365377 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 365377 T801 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 365377 T817 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 365378 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 365379 T785 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 365379 T772 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 365380 T765 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 365384 T818 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 365384 T818 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 365731 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 365732 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:35304_zbo%2Fzd_collection1",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35304_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35304/zbo/zd"}
[junit4:junit4] 2> 365736 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57819_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57819/zbo/zd"}
[junit4:junit4] 2> 365736 T766 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 365737 T766 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard3
[junit4:junit4] 2> 365741 T817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 365741 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 365741 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 365741 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 365741 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 366385 T818 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371891910416/collection1
[junit4:junit4] 2> 366386 T818 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 366387 T818 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 366387 T818 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 366388 T818 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371891910416/collection1/'
[junit4:junit4] 2> 366389 T818 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371891910416/collection1/lib/README' to classloader
[junit4:junit4] 2> 366389 T818 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371891910416/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 366413 T818 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 366441 T818 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 366443 T818 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 366446 T818 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 366758 T818 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 366758 T818 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 366759 T818 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 366765 T818 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 366767 T818 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 366782 T818 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 366785 T818 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 366788 T818 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 366789 T818 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 366789 T818 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 366789 T818 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 366791 T818 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 366791 T818 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 366791 T818 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 366791 T818 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371891910416/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty3/
[junit4:junit4] 2> 366791 T818 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@402fb002
[junit4:junit4] 2> 366792 T818 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 366792 T818 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty3
[junit4:junit4] 2> 366792 T818 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty3/index/
[junit4:junit4] 2> 366793 T818 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 366793 T818 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty3/index
[junit4:junit4] 2> 366795 T818 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 366796 T818 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 366797 T818 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 366798 T818 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 366798 T818 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 366799 T818 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 366799 T818 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 366799 T818 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 366799 T818 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 366800 T818 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 366800 T818 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 366802 T818 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 366805 T818 oass.SolrIndexSearcher.<init> Opening Searcher@60a23db3 main
[junit4:junit4] 2> 366805 T818 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 366805 T818 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 366809 T819 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@60a23db3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 366811 T818 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 366811 T818 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57819/zbo/zd collection:collection1 shard:shard3
[junit4:junit4] 2> 366812 T818 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4:junit4] 2> 366820 T818 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 366823 T818 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 366823 T818 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 366823 T818 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57819/zbo/zd/collection1/
[junit4:junit4] 2> 366823 T818 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 366823 T818 oasc.SyncStrategy.syncToMe http://127.0.0.1:57819/zbo/zd/collection1/ has no replicas
[junit4:junit4] 2> 366823 T818 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57819/zbo/zd/collection1/
[junit4:junit4] 2> 366824 T818 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4:junit4] 2> 367249 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 367406 T817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 367406 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 367406 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 367406 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 367407 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 367430 T818 oasc.ZkController.register We are http://127.0.0.1:57819/zbo/zd/collection1/ and leader is http://127.0.0.1:57819/zbo/zd/collection1/
[junit4:junit4] 2> 367430 T818 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57819/zbo/zd
[junit4:junit4] 2> 367431 T818 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 367431 T818 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 367431 T818 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 367434 T818 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 367436 T743 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 367436 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 367437 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 367507 T743 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 367510 T743 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34078
[junit4:junit4] 2> 367510 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 367511 T743 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 367511 T743 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371891913619
[junit4:junit4] 2> 367511 T743 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371891913619/solr.xml
[junit4:junit4] 2> 367512 T743 oasc.CoreContainer.<init> New CoreContainer 1205066171
[junit4:junit4] 2> 367512 T743 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371891913619/'
[junit4:junit4] 2> 367512 T743 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371891913619/'
[junit4:junit4] 2> 367550 T743 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 367551 T743 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 367551 T743 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 367551 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 367552 T743 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 367552 T743 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 367552 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 367552 T743 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 367553 T743 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 367553 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 367556 T743 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 367557 T743 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44356/solr
[junit4:junit4] 2> 367557 T743 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 367558 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 367559 T831 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@59890985 name:ZooKeeperConnection Watcher:127.0.0.1:44356 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 367560 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 367562 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 367565 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 367567 T833 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61539c24 name:ZooKeeperConnection Watcher:127.0.0.1:44356/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 367567 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 367571 T743 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 368575 T743 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34078_zbo%2Fzd
[junit4:junit4] 2> 368576 T743 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34078_zbo%2Fzd
[junit4:junit4] 2> 368579 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 368579 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 368579 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 368580 T801 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 368580 T817 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 368580 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 368580 T817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 368581 T833 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 368582 T765 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 368582 T772 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 368582 T785 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 368587 T834 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 368587 T834 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 368915 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 368916 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:57819_zbo%2Fzd_collection1",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard3",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57819_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57819/zbo/zd"}
[junit4:junit4] 2> 368920 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34078_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34078/zbo/zd"}
[junit4:junit4] 2> 368920 T766 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 368920 T766 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 368924 T817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 368925 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 368925 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 368925 T833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 368924 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 368924 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 369589 T834 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371891913619/collection1
[junit4:junit4] 2> 369589 T834 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 369590 T834 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 369590 T834 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 369591 T834 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371891913619/collection1/'
[junit4:junit4] 2> 369592 T834 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371891913619/collection1/lib/README' to classloader
[junit4:junit4] 2> 369592 T834 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371891913619/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 369616 T834 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 369645 T834 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 369646 T834 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 369652 T834 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 369987 T834 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 369987 T834 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 369988 T834 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 369993 T834 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 369996 T834 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 370010 T834 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 370013 T834 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 370016 T834 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 370017 T834 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 370017 T834 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 370017 T834 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 370018 T834 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 370018 T834 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 370019 T834 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 370019 T834 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371891913619/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty4/
[junit4:junit4] 2> 370019 T834 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@402fb002
[junit4:junit4] 2> 370019 T834 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 370020 T834 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty4
[junit4:junit4] 2> 370020 T834 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty4/index/
[junit4:junit4] 2> 370020 T834 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 370021 T834 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty4/index
[junit4:junit4] 2> 370023 T834 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 370023 T834 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 370025 T834 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 370025 T834 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 370026 T834 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 370026 T834 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 370027 T834 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 370027 T834 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 370027 T834 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 370028 T834 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 370028 T834 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 370030 T834 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 370032 T834 oass.SolrIndexSearcher.<init> Opening Searcher@572ae69a main
[junit4:junit4] 2> 370033 T834 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 370033 T834 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 370040 T835 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@572ae69a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 370043 T834 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 370043 T834 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34078/zbo/zd collection:collection1 shard:shard1
[junit4:junit4] 2> 370047 T834 oasc.ZkController.register We are http://127.0.0.1:34078/zbo/zd/collection1/ and leader is http://127.0.0.1:53383/zbo/zd/collection1/
[junit4:junit4] 2> 370047 T834 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34078/zbo/zd
[junit4:junit4] 2> 370047 T834 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 370047 T834 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C354 name=collection1 org.apache.solr.core.SolrCore@2d727992 url=http://127.0.0.1:34078/zbo/zd/collection1 node=127.0.0.1:34078_zbo%2Fzd C354_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:34078_zbo%2Fzd, base_url=http://127.0.0.1:34078/zbo/zd}
[junit4:junit4] 2> 370052 T836 C354 P34078 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 370052 T834 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 370053 T836 C354 P34078 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 370054 T836 C354 P34078 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 370054 T836 C354 P34078 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 370054 T743 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 370055 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 370055 T836 C354 P34078 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 370056 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 370068 T778 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:34078_zbo%2Fzd_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 370161 T743 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 370163 T743 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:58034
[junit4:junit4] 2> 370164 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 370165 T743 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 370165 T743 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371891916238
[junit4:junit4] 2> 370165 T743 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371891916238/solr.xml
[junit4:junit4] 2> 370166 T743 oasc.CoreContainer.<init> New CoreContainer 1994746135
[junit4:junit4] 2> 370166 T743 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371891916238/'
[junit4:junit4] 2> 370166 T743 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371891916238/'
[junit4:junit4] 2> 370215 T743 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 370216 T743 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 370216 T743 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 370217 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 370217 T743 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 370217 T743 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 370218 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 370218 T743 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 370218 T743 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 370219 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 370222 T743 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 370222 T743 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44356/solr
[junit4:junit4] 2> 370223 T743 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 370224 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 370226 T848 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@575ddcc9 name:ZooKeeperConnection Watcher:127.0.0.1:44356 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 370226 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 370228 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 370232 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 370233 T850 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72a69cb0 name:ZooKeeperConnection Watcher:127.0.0.1:44356/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 370233 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 370237 T743 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 370436 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 370437 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:34078_zbo%2Fzd_collection1",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34078_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34078/zbo/zd"}
[junit4:junit4] 2> 370442 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 370442 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 370442 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 370442 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 370442 T850 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 370442 T833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 370442 T817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 371069 T778 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:34078_zbo%2Fzd_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 371069 T778 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:34078_zbo%252Fzd_collection1&state=recovering&nodeName=127.0.0.1:34078_zbo%252Fzd&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1003
[junit4:junit4] 2> 371241 T743 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58034_zbo%2Fzd
[junit4:junit4] 2> 371242 T743 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58034_zbo%2Fzd
[junit4:junit4] 2> 371245 T850 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 371245 T833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 371246 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 371246 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 371245 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 371246 T801 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 371247 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 371247 T817 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 371247 T817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 371249 T833 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 371250 T850 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 371250 T765 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 371251 T772 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 371251 T785 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 371262 T851 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 371263 T851 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 371948 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 371949 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:58034_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58034/zbo/zd"}
[junit4:junit4] 2> 371949 T766 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 371949 T766 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 372046 T817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 372047 T833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 372047 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 372046 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 372046 T850 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 372046 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 372046 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 372264 T851 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371891916238/collection1
[junit4:junit4] 2> 372265 T851 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 372266 T851 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 372266 T851 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 372268 T851 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371891916238/collection1/'
[junit4:junit4] 2> 372268 T851 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371891916238/collection1/lib/README' to classloader
[junit4:junit4] 2> 372268 T851 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371891916238/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 372293 T851 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 372323 T851 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 372325 T851 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 372328 T851 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 372702 T851 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 372702 T851 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 372703 T851 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 372709 T851 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 372711 T851 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 372727 T851 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 372731 T851 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 372736 T851 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 372738 T851 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 372738 T851 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 372738 T851 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 372739 T851 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 372739 T851 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 372740 T851 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 372740 T851 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371891916238/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty5/
[junit4:junit4] 2> 372740 T851 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@402fb002
[junit4:junit4] 2> 372741 T851 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 372741 T851 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty5
[junit4:junit4] 2> 372741 T851 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty5/index/
[junit4:junit4] 2> 372742 T851 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty5/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 372742 T851 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty5/index
[junit4:junit4] 2> 372757 T851 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty5/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 372757 T851 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 372759 T851 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 372759 T851 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 372760 T851 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 372760 T851 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 372761 T851 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 372761 T851 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 372761 T851 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 372762 T851 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 372762 T851 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 372764 T851 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 372766 T851 oass.SolrIndexSearcher.<init> Opening Searcher@c9f0ead main
[junit4:junit4] 2> 372767 T851 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 372767 T851 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 372772 T852 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c9f0ead main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 372775 T851 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 372775 T851 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58034/zbo/zd collection:collection1 shard:shard2
[junit4:junit4] 2> 372778 T851 oasc.ZkController.register We are http://127.0.0.1:58034/zbo/zd/collection1/ and leader is http://127.0.0.1:35304/zbo/zd/collection1/
[junit4:junit4] 2> 372779 T851 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58034/zbo/zd
[junit4:junit4] 2> 372779 T851 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 372779 T851 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C355 name=collection1 org.apache.solr.core.SolrCore@14916364 url=http://127.0.0.1:58034/zbo/zd/collection1 node=127.0.0.1:58034_zbo%2Fzd C355_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:58034_zbo%2Fzd, base_url=http://127.0.0.1:58034/zbo/zd}
[junit4:junit4] 2> 372779 T853 C355 P58034 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 372780 T853 C355 P58034 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 372780 T853 C355 P58034 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 372780 T851 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 372781 T853 C355 P58034 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 372782 T743 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 372782 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 372782 T853 C355 P58034 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 372782 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 372787 T797 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:58034_zbo%2Fzd_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 372878 T743 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 372880 T743 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53996
[junit4:junit4] 2> 372881 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 372881 T743 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 372882 T743 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371891918963
[junit4:junit4] 2> 372882 T743 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371891918963/solr.xml
[junit4:junit4] 2> 372882 T743 oasc.CoreContainer.<init> New CoreContainer 1795016410
[junit4:junit4] 2> 372883 T743 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371891918963/'
[junit4:junit4] 2> 372883 T743 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371891918963/'
[junit4:junit4] 2> 372925 T743 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 372926 T743 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 372926 T743 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 372927 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 372927 T743 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 372927 T743 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 372928 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 372928 T743 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 372928 T743 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 372929 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 372932 T743 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 372933 T743 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44356/solr
[junit4:junit4] 2> 372933 T743 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 372934 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 372935 T865 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@204461a2 name:ZooKeeperConnection Watcher:127.0.0.1:44356 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 372936 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 372937 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 372941 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 372942 T867 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1aecdfce name:ZooKeeperConnection Watcher:127.0.0.1:44356/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 372942 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 372946 T743 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C356 name=collection1 org.apache.solr.core.SolrCore@2d727992 url=http://127.0.0.1:34078/zbo/zd/collection1 node=127.0.0.1:34078_zbo%2Fzd C356_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:34078_zbo%2Fzd, base_url=http://127.0.0.1:34078/zbo/zd}
[junit4:junit4] 2> 373070 T836 C356 P34078 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:53383/zbo/zd/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 373073 T836 C356 P34078 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 373077 T836 C356 P34078 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:34078/zbo/zd START replicas=[http://127.0.0.1:53383/zbo/zd/collection1/] nUpdates=100
[junit4:junit4] 2> 373080 T836 C356 P34078 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 373080 T836 C356 P34078 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 373080 T836 C356 P34078 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 373080 T836 C356 P34078 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 373081 T836 C356 P34078 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 373081 T836 C356 P34078 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:53383/zbo/zd/collection1/. core=collection1
[junit4:junit4] 2> 373081 T836 C356 P34078 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C357 name=collection1 org.apache.solr.core.SolrCore@1826ed67 url=http://127.0.0.1:53383/zbo/zd/collection1 node=127.0.0.1:53383_zbo%2Fzd C357_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:53383_zbo%2Fzd, base_url=http://127.0.0.1:53383/zbo/zd, leader=true}
[junit4:junit4] 2> 373087 T778 C357 P53383 oasc.SolrCore.execute [collection1] webapp=/zbo/zd path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 373089 T780 C357 P53383 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 373091 T780 C357 P53383 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 373091 T780 C357 P53383 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 373096 T780 C357 P53383 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 373096 T780 C357 P53383 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 373097 T780 C357 P53383 oass.SolrIndexSearcher.<init> Opening Searcher@72be18b7 realtime
[junit4:junit4] 2> 373097 T780 C357 P53383 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 373097 T780 C357 P53383 oasup.LogUpdateProcessor.finish [collection1] webapp=/zbo/zd path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 373098 T836 C356 P34078 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 373099 T836 C356 P34078 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4] 2> 373103 T836 C356 P34078 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 373105 T779 C357 P53383 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 373106 T779 C357 P53383 oasc.SolrCore.execute [collection1] webapp=/zbo/zd path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 373107 T836 C356 P34078 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 373107 T836 C356 P34078 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 373107 T836 C356 P34078 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 373109 T778 C357 P53383 oasc.SolrCore.execute [collection1] webapp=/zbo/zd path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 373110 T836 C356 P34078 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 373111 T836 C356 P34078 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty4/index.20130622140519287
[junit4:junit4] 2> 373111 T836 C356 P34078 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty4/index.20130622140519287 lockFactory=org.apache.lucene.store.NativeFSLockFactory@39363716 fullCopy=false
[junit4:junit4] 2> 373116 T780 C357 P53383 oasc.SolrCore.execute [collection1] webapp=/zbo/zd path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1
[junit4:junit4] 2> 373120 T836 C356 P34078 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 373127 T836 C356 P34078 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 373127 T836 C356 P34078 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 373131 T836 C356 P34078 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty4/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 373131 T836 C356 P34078 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 373132 T836 C356 P34078 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 373132 T836 C356 P34078 oass.SolrIndexSearcher.<init> Opening Searcher@1678a557 main
[junit4:junit4] 2> 373133 T835 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1678a557 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 373133 T836 C356 P34078 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty4/index.20130622140519287 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty4/index.20130622140519287;done=true>>]
[junit4:junit4] 2> 373134 T836 C356 P34078 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty4/index.20130622140519287
[junit4:junit4] 2> 373134 T836 C356 P34078 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty4/index.20130622140519287
[junit4:junit4] 2> 373134 T836 C356 P34078 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 373134 T836 C356 P34078 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 373135 T836 C356 P34078 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 373135 T836 C356 P34078 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 373137 T836 C356 P34078 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 373553 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 373554 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:58034_zbo%2Fzd_collection1",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:58034_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58034/zbo/zd"}
[junit4:junit4] 2> 373557 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:34078_zbo%2Fzd_collection1",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34078_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34078/zbo/zd"}
[junit4:junit4] 2> 373576 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 373576 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 373577 T850 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 373577 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 373577 T867 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 373577 T833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 373576 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 373576 T817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 373788 T797 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:58034_zbo%2Fzd_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 373789 T797 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:58034_zbo%252Fzd_collection1&state=recovering&nodeName=127.0.0.1:58034_zbo%252Fzd&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 373950 T743 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53996_zbo%2Fzd
[junit4:junit4] 2> 373951 T743 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53996_zbo%2Fzd
[junit4:junit4] 2> 373953 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 373954 T850 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 373954 T833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 373954 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 373954 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 373955 T817 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 373955 T817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 373955 T867 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 373956 T867 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 373956 T801 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 373956 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 373958 T785 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 373958 T833 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 373958 T772 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 373959 T850 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 373959 T765 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 373970 T870 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 373970 T870 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 375086 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 375087 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:53996_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:53996/zbo/zd"}
[junit4:junit4] 2> 375087 T766 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 375087 T766 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard3
[junit4:junit4] 2> 375091 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 375091 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 375091 T817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 375092 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 375092 T833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 375092 T867 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 375092 T850 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 375092 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> C355_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:58034_zbo%2Fzd, base_url=http://127.0.0.1:58034/zbo/zd}
[junit4:junit4] 2> 375790 T853 C355 P58034 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:35304/zbo/zd/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 375790 T853 C355 P58034 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:58034/zbo/zd START replicas=[http://127.0.0.1:35304/zbo/zd/collection1/] nUpdates=100
[junit4:junit4] 2> 375791 T853 C355 P58034 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 375791 T853 C355 P58034 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 375791 T853 C355 P58034 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 375791 T853 C355 P58034 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 375791 T853 C355 P58034 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 375791 T853 C355 P58034 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:35304/zbo/zd/collection1/. core=collection1
[junit4:junit4] 2> 375792 T853 C355 P58034 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C358 name=collection1 org.apache.solr.core.SolrCore@2c5141d8 url=http://127.0.0.1:35304/zbo/zd/collection1 node=127.0.0.1:35304_zbo%2Fzd C358_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:35304_zbo%2Fzd, base_url=http://127.0.0.1:35304/zbo/zd, leader=true}
[junit4:junit4] 2> 375801 T797 C358 P35304 oasc.SolrCore.execute [collection1] webapp=/zbo/zd path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 375802 T795 C358 P35304 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 375804 T795 C358 P35304 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 375804 T795 C358 P35304 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 375816 T795 C358 P35304 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 375816 T795 C358 P35304 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 375817 T795 C358 P35304 oass.SolrIndexSearcher.<init> Opening Searcher@219da9b realtime
[junit4:junit4] 2> 375817 T795 C358 P35304 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 375818 T795 C358 P35304 oasup.LogUpdateProcessor.finish [collection1] webapp=/zbo/zd path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 17
[junit4:junit4] 2> ASYNC NEW_CORE C359 name=collection1 org.apache.solr.core.SolrCore@14916364 url=http://127.0.0.1:58034/zbo/zd/collection1 node=127.0.0.1:58034_zbo%2Fzd C359_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:58034_zbo%2Fzd, base_url=http://127.0.0.1:58034/zbo/zd}
[junit4:junit4] 2> 375819 T853 C359 P58034 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 375819 T853 C359 P58034 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 375821 T794 C358 P35304 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 375822 T794 C358 P35304 oasc.SolrCore.execute [collection1] webapp=/zbo/zd path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 375823 T853 C359 P58034 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 375823 T853 C359 P58034 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 375823 T853 C359 P58034 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 375825 T797 C358 P35304 oasc.SolrCore.execute [collection1] webapp=/zbo/zd path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 375826 T853 C359 P58034 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 375826 T853 C359 P58034 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty5/index.20130622140522003
[junit4:junit4] 2> 375826 T853 C359 P58034 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty5/index.20130622140522003 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6e021e4 fullCopy=false
[junit4:junit4] 2> 375829 T795 C358 P35304 oasc.SolrCore.execute [collection1] webapp=/zbo/zd path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 375830 T853 C359 P58034 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 375837 T853 C359 P58034 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 375837 T853 C359 P58034 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 375838 T853 C359 P58034 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty5/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty5/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 375839 T853 C359 P58034 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 375839 T853 C359 P58034 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 375839 T853 C359 P58034 oass.SolrIndexSearcher.<init> Opening Searcher@4a6ef07c main
[junit4:junit4] 2> 375840 T852 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4a6ef07c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 375841 T853 C359 P58034 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty5/index.20130622140522003 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty5/index.20130622140522003;done=true>>]
[junit4:junit4] 2> 375841 T853 C359 P58034 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty5/index.20130622140522003
[junit4:junit4] 2> 375841 T853 C359 P58034 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty5/index.20130622140522003
[junit4:junit4] 2> 375842 T853 C359 P58034 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 375842 T853 C359 P58034 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 375842 T853 C359 P58034 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 375842 T853 C359 P58034 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 375844 T853 C359 P58034 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 375972 T870 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371891918963/collection1
[junit4:junit4] 2> 375972 T870 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 375973 T870 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 375973 T870 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 375975 T870 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371891918963/collection1/'
[junit4:junit4] 2> 375976 T870 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371891918963/collection1/lib/README' to classloader
[junit4:junit4] 2> 375976 T870 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371891918963/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 376012 T870 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 376042 T870 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 376043 T870 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 376047 T870 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 376376 T870 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 376377 T870 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 376377 T870 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 376383 T870 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 376386 T870 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 376400 T870 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 376404 T870 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 376406 T870 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 376408 T870 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 376408 T870 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 376408 T870 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 376410 T870 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 376410 T870 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 376410 T870 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 376410 T870 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371891918963/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty6/
[junit4:junit4] 2> 376410 T870 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@402fb002
[junit4:junit4] 2> 376411 T870 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 376411 T870 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty6
[junit4:junit4] 2> 376411 T870 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty6/index/
[junit4:junit4] 2> 376412 T870 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty6/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 376412 T870 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty6/index
[junit4:junit4] 2> 376427 T870 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371891900957/jetty6/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 376428 T870 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 376430 T870 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 376430 T870 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 376430 T870 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 376431 T870 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 376431 T870 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 376431 T870 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 376434 T870 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 376434 T870 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 376435 T870 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 376436 T870 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 376439 T870 oass.SolrIndexSearcher.<init> Opening Searcher@226e273a main
[junit4:junit4] 2> 376440 T870 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 376440 T870 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 376444 T873 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@226e273a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 376446 T870 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 376446 T870 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:53996/zbo/zd collection:collection1 shard:shard3
[junit4:junit4] 2> 376450 T870 oasc.ZkController.register We are http://127.0.0.1:53996/zbo/zd/collection1/ and leader is http://127.0.0.1:57819/zbo/zd/collection1/
[junit4:junit4] 2> 376450 T870 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53996/zbo/zd
[junit4:junit4] 2> 376450 T870 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 376450 T870 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C360 name=collection1 org.apache.solr.core.SolrCore@74ec1f4b url=http://127.0.0.1:53996/zbo/zd/collection1 node=127.0.0.1:53996_zbo%2Fzd C360_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:53996_zbo%2Fzd, base_url=http://127.0.0.1:53996/zbo/zd}
[junit4:junit4] 2> 376451 T874 C360 P53996 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 376451 T874 C360 P53996 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 376451 T870 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 376452 T874 C360 P53996 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 376452 T874 C360 P53996 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 376453 T743 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 376453 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 376453 T874 C360 P53996 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 376453 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 376459 T810 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:53996_zbo%2Fzd_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 376521 T743 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 376523 T743 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:42393
[junit4:junit4] 2> 376524 T743 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 376525 T743 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 376525 T743 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1371891922633
[junit4:junit4] 2> 376525 T743 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1371891922633/solr.xml
[junit4:junit4] 2> 376526 T743 oasc.CoreContainer.<init> New CoreContainer 1906350350
[junit4:junit4] 2> 376526 T743 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1371891922633/'
[junit4:junit4] 2> 376526 T743 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1371891922633/'
[junit4:junit4] 2> 376565 T743 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 376565 T743 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 376566 T743 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 376566 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 376566 T743 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 376567 T743 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 376567 T743 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 376567 T743 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 376567 T743 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 376568 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 376571 T743 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 376571 T743 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44356/solr
[junit4:junit4] 2> 376572 T743 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 376573 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 376574 T886 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4289418b name:ZooKeeperConnection Watcher:127.0.0.1:44356 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 376575 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 376577 T743 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 376580 T743 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 376581 T888 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e74a9a9 name:ZooKeeperConnection Watcher:127.0.0.1:44356/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 376581 T743 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 376586 T743 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 376599 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 376600 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:58034_zbo%2Fzd_collection1",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:58034_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58034/zbo/zd"}
[junit4:junit4] 2> 376605 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:53996_zbo%2Fzd_collection1",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard3",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:53996_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:53996/zbo/zd"}
[junit4:junit4] 2> 376609 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 376610 T850 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 376610 T817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 376610 T867 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 376610 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 376610 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 376610 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 376610 T833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 376611 T888 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 377459 T810 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:53996_zbo%2Fzd_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 377460 T810 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:53996_zbo%252Fzd_collection1&state=recovering&nodeName=127.0.0.1:53996_zbo%252Fzd&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 377590 T743 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42393_zbo%2Fzd
[junit4:junit4] 2> 377591 T743 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42393_zbo%2Fzd
[junit4:junit4] 2> 377593 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 377593 T850 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 377593 T833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 377594 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 377594 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 377594 T801 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 377595 T817 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 377595 T817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 377595 T867 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 377595 T867 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 377595 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 377595 T888 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 377596 T888 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 377600 T785 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 377601 T772 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 377601 T833 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 377602 T765 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 377604 T850 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 377607 T889 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 377608 T889 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 378118 T766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 378119 T766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:42393_zbo%2Fzd",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:42393/zbo/zd"}
[junit4:junit4] 2> 378119 T766 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 378119 T766 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 378123 T785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 378123 T850 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 378124 T867 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 378124 T817 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 378123 T801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 378124 T765 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 378124 T833 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 378124 T772 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 378125 T888 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 378609 T889 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1371891922633/collection1
[junit4:junit4] 2> 378610 T889 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 378610 T889 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 378611 T889 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 378612 T889 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1371891922633/collection1/'
[junit4:junit4] 2> 378612 T889 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1371891922633/collection1/lib/README' to classloader
[junit4:junit4] 2> 378613 T889 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1371891922633/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 378648 T889 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 378680 T889 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 378681 T889 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 378685 T889 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 379013 T889 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 379014 T889 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 379014 T889 oass.Ope
[...truncated too long message...]
7 T2133 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51858/collection1/
[junit4:junit4] 2> 815487 T2133 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 815487 T2133 oasc.SyncStrategy.syncToMe http://127.0.0.1:51858/collection1/ has no replicas
[junit4:junit4] 2> 815487 T2133 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:51858/collection1/
[junit4:junit4] 2> 815487 T2133 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 815492 T2145 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:43384__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:43384_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43384"}
[junit4:junit4] 2> 815499 T2133 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 815502 T2051 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 815555 T2051 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 51858
[junit4:junit4] 2> 815555 T2051 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1295766079
[junit4:junit4] 2> 816556 T2051 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 816557 T2051 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 816558 T2051 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@7b135360
[junit4:junit4] 2> 816564 T2051 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=43,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=68,cumulative_deletesById=36,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 816565 T2051 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 816565 T2051 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 816566 T2051 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 816567 T2051 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 816567 T2051 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 816568 T2051 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1371892253230/jetty2 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1371892253230/jetty2;done=false>>]
[junit4:junit4] 2> 816568 T2051 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1371892253230/jetty2
[junit4:junit4] 2> 816568 T2051 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.RecoveryZkTest-1371892253230/jetty2/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.RecoveryZkTest-1371892253230/jetty2/index;done=false>>]
[junit4:junit4] 2> 816568 T2051 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1371892253230/jetty2/index
[junit4:junit4] 2> 816571 T2145 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89908330707812362-127.0.0.1:51858_-n_0000000003) am no longer a leader.
[junit4:junit4] 2> 816593 T2051 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 816648 T2051 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 816650 T2051 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:46814 46814
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 1> safe stop:
[junit4:junit4] 1> safe stop:
[junit4:junit4] 1> safe stop:
[junit4:junit4] 1> safe stop:
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=RecoveryZkTest -Dtests.method=testDistribSearch -Dtests.seed=947A66B1BF3424BD -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=uk -Dtests.timezone=Europe/Vatican -Dtests.file.encoding=US-ASCII
[junit4:junit4] FAILURE 110s J1 | RecoveryZkTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: expected:<194> but was:<52>
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([947A66B1BF3424BD:159CE8A9C86B4481]:0)
[junit4:junit4] > at org.apache.solr.cloud.RecoveryZkTest.doTest(RecoveryZkTest.java:108)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2> 816655 T2051 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 109609 T2050 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
[junit4:junit4] 2> 817872 T2133 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 817872 T2133 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 817872 T2133 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_l=PostingsFormat(name=Memory doPackFST= false), text=Pulsing41(freqCutoff=19 minBlockSize=52 maxBlockSize=171), _version_=PostingsFormat(name=Memory doPackFST= false), rnd_b=PostingsFormat(name=Direct), intDefault=PostingsFormat(name=Memory doPackFST= false), id=PostingsFormat(name=Direct), timestamp=PostingsFormat(name=Memory doPackFST= false), a_t=PostingsFormat(name=Memory doPackFST= false), range_facet_sl=Pulsing41(freqCutoff=19 minBlockSize=52 maxBlockSize=171), range_facet_si=PostingsFormat(name=Asserting), other_tl1=PostingsFormat(name=Memory doPackFST= false), multiDefault=PostingsFormat(name=Asserting), a_si=PostingsFormat(name=Asserting)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=uk, timezone=Europe/Vatican
[junit4:junit4] 2> NOTE: Linux 3.2.0-48-generic amd64/Sun Microsystems Inc. 1.6.0_45 (64-bit)/cpus=8,threads=1,free=149687392,total=319356928
[junit4:junit4] 2> NOTE: All tests run in this JVM: [HardAutoCommitTest, TestShardHandlerFactory, TestLuceneMatchVersion, IndexReaderFactoryTest, MultiTermTest, UniqFieldsUpdateProcessorFactoryTest, TestRandomFaceting, TestWordDelimiterFilterFactory, TestFieldResource, CopyFieldTest, DirectSolrSpellCheckerTest, NumericFieldsTest, TestRTGBase, TestPartialUpdateDeduplication, TestUpdate, PreAnalyzedUpdateProcessorTest, ShardRoutingCustomTest, HighlighterConfigTest, ConvertedLegacyTest, DocValuesMultiTest, OpenCloseCoreStressTest, TestRecovery, TestFastOutputStream, TestMergePolicyConfig, TestAddFieldRealTimeGet, XmlUpdateRequestHandlerTest, SampleTest, TestSolrQueryParserDefaultOperatorResource, StatsComponentTest, TestJmxMonitoredMap, OutputWriterTest, StandardRequestHandlerTest, TestIndexSearcher, SimplePostToolTest, TestStressLucene, IndexSchemaRuntimeFieldTest, PathHierarchyTokenizerFactoryTest, TestCodecSupport, TestSolrJ, TestUniqueKeyFieldResource, TestBinaryField, TestHashPartitioner, TestDFRSimilarityFactory, BasicZkTest, TestAnalyzedSuggestions, SolrCoreCheckLockOnStartupTest, TestSolrXMLSerializer, NotRequiredUniqueKeyTest, SignatureUpdateProcessorFactoryTest, TestCollationField, TestSolr4Spatial, UpdateParamsTest, TestRangeQuery, ChaosMonkeySafeLeaderTest, TestNumberUtils, TestFieldTypeResource, TestLMJelinekMercerSimilarityFactory, DistanceFunctionTest, TestSolrQueryParserResource, TestBinaryResponseWriter, SpellPossibilityIteratorTest, SolrTestCaseJ4Test, SolrCmdDistributorTest, SortByFunctionTest, TestSystemIdResolver, CollectionsAPIDistributedZkTest, TestDistributedSearch, ZkSolrClientTest, BasicDistributedZk2Test, RecoveryZkTest]
[junit4:junit4] Completed on J1 in 110.89s, 1 test, 1 failure <<< FAILURES!
[...truncated 439 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:392: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:372: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1246: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:890: There were test failures: 298 suites, 1256 tests, 2 failures, 13 ignored (7 assumptions)
Total time: 37 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.6.0_45 -XX:+UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b93) - Build #
6184 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6184/
Java: 32bit/jdk1.8.0-ea-b93 -server -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:43073/ebg/lc/collection1lastClient and got 246 from http://127.0.0.1:52252/ebg/lc/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:43073/ebg/lc/collection1lastClient and got 246 from http://127.0.0.1:52252/ebg/lc/collection1
at __randomizedtesting.SeedInfo.seed([FD6B0656D09C0FB9:7C8D884EA7C36F85]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:491)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
Build Log:
[...truncated 9500 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 365409 T1514 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /ebg/lc
[junit4:junit4] 2> 365413 T1514 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1371908984250
[junit4:junit4] 2> 365414 T1514 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 365414 T1515 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 365514 T1514 oasc.ZkTestServer.run start zk server on port:33456
[junit4:junit4] 2> 365515 T1514 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 365606 T1521 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a24510 name:ZooKeeperConnection Watcher:127.0.0.1:33456 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 365606 T1514 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 365606 T1514 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 365611 T1514 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 365613 T1523 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@165b654 name:ZooKeeperConnection Watcher:127.0.0.1:33456/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 365613 T1514 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 365613 T1514 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 365616 T1514 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 365619 T1514 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 365621 T1514 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 365623 T1514 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 365624 T1514 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 365627 T1514 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 365628 T1514 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 365631 T1514 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 365631 T1514 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 365634 T1514 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 365634 T1514 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 365636 T1514 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 365637 T1514 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 365639 T1514 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 365640 T1514 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 365642 T1514 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 365642 T1514 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 365645 T1514 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 365646 T1514 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 365648 T1514 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 365649 T1514 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 365751 T1514 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 365756 T1514 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50855
[junit4:junit4] 2> 365757 T1514 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 365757 T1514 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 365758 T1514 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371908984488
[junit4:junit4] 2> 365758 T1514 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371908984488/solr.xml
[junit4:junit4] 2> 365758 T1514 oasc.CoreContainer.<init> New CoreContainer 28260414
[junit4:junit4] 2> 365759 T1514 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371908984488/'
[junit4:junit4] 2> 365759 T1514 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371908984488/'
[junit4:junit4] 2> 365829 T1514 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 365829 T1514 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 365830 T1514 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 365830 T1514 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 365831 T1514 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 365831 T1514 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 365831 T1514 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 365832 T1514 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 365832 T1514 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 365833 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 365836 T1514 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 365836 T1514 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33456/solr
[junit4:junit4] 2> 365837 T1514 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 365838 T1514 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 365840 T1534 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78d3e6 name:ZooKeeperConnection Watcher:127.0.0.1:33456 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 365840 T1514 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 365842 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 365846 T1514 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 365847 T1536 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@85a932 name:ZooKeeperConnection Watcher:127.0.0.1:33456/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 365847 T1514 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 365849 T1514 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 365852 T1514 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 365855 T1514 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 365857 T1514 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50855_ebg%2Flc
[junit4:junit4] 2> 365858 T1514 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50855_ebg%2Flc
[junit4:junit4] 2> 365861 T1514 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 365867 T1514 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 365869 T1514 oasc.Overseer.start Overseer (id=89909427191939075-127.0.0.1:50855_ebg%2Flc-n_0000000000) starting
[junit4:junit4] 2> 365872 T1514 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 365876 T1538 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 365876 T1514 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 365879 T1514 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 365880 T1514 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 365883 T1537 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 365886 T1539 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 365886 T1539 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 367387 T1537 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 367387 T1537 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "node_name":"127.0.0.1:50855_ebg%2Flc",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50855/ebg/lc",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":null}
[junit4:junit4] 2> 367387 T1537 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 367388 T1537 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 367391 T1536 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 367887 T1539 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371908984488/collection1
[junit4:junit4] 2> 367887 T1539 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 367888 T1539 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 367888 T1539 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 367889 T1539 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371908984488/collection1/'
[junit4:junit4] 2> 367890 T1539 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371908984488/collection1/lib/README' to classloader
[junit4:junit4] 2> 367890 T1539 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371908984488/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 367923 T1539 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 367965 T1539 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 367967 T1539 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 367971 T1539 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 368348 T1539 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 368348 T1539 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 368349 T1539 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 368354 T1539 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 368356 T1539 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 368368 T1539 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 368370 T1539 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 368373 T1539 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 368374 T1539 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 368374 T1539 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 368374 T1539 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 368375 T1539 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 368375 T1539 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 368376 T1539 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 368376 T1539 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371908984488/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371908984249/control/data/
[junit4:junit4] 2> 368376 T1539 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@147c2e4
[junit4:junit4] 2> 368376 T1539 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 368377 T1539 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/control/data
[junit4:junit4] 2> 368377 T1539 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371908984249/control/data/index/
[junit4:junit4] 2> 368377 T1539 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371908984249/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 368377 T1539 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/control/data/index
[junit4:junit4] 2> 368381 T1539 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c33b7f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 368381 T1539 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 368383 T1539 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 368383 T1539 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 368384 T1539 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 368384 T1539 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 368384 T1539 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 368384 T1539 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 368385 T1539 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 368385 T1539 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 368385 T1539 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 368387 T1539 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 368389 T1539 oass.SolrIndexSearcher.<init> Opening Searcher@1140399 main
[junit4:junit4] 2> 368390 T1539 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 368390 T1539 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 368393 T1540 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1140399 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 368395 T1539 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 368395 T1539 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50855/ebg/lc collection:control_collection shard:shard1
[junit4:junit4] 2> 368396 T1539 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 368402 T1539 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 368404 T1539 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 368405 T1539 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 368405 T1539 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50855/ebg/lc/collection1/
[junit4:junit4] 2> 368405 T1539 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 368405 T1539 oasc.SyncStrategy.syncToMe http://127.0.0.1:50855/ebg/lc/collection1/ has no replicas
[junit4:junit4] 2> 368406 T1539 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50855/ebg/lc/collection1/
[junit4:junit4] 2> 368406 T1539 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 368894 T1537 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 368901 T1536 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 368913 T1539 oasc.ZkController.register We are http://127.0.0.1:50855/ebg/lc/collection1/ and leader is http://127.0.0.1:50855/ebg/lc/collection1/
[junit4:junit4] 2> 368913 T1539 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50855/ebg/lc
[junit4:junit4] 2> 368914 T1539 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 368914 T1539 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 368914 T1539 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 368916 T1539 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 368916 T1514 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 368917 T1514 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 368917 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 368920 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 368921 T1514 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 368922 T1543 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cac12 name:ZooKeeperConnection Watcher:127.0.0.1:33456/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 368923 T1514 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 368924 T1514 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 368926 T1514 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 368994 T1514 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 368996 T1514 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:43073
[junit4:junit4] 2> 368996 T1514 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 368997 T1514 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 368997 T1514 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371908987762
[junit4:junit4] 2> 368997 T1514 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371908987762/solr.xml
[junit4:junit4] 2> 368998 T1514 oasc.CoreContainer.<init> New CoreContainer 33065911
[junit4:junit4] 2> 368998 T1514 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371908987762/'
[junit4:junit4] 2> 368998 T1514 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371908987762/'
[junit4:junit4] 2> 369043 T1514 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 369044 T1514 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 369044 T1514 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 369044 T1514 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 369045 T1514 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 369045 T1514 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 369045 T1514 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 369045 T1514 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 369046 T1514 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 369046 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 369049 T1514 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 369049 T1514 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33456/solr
[junit4:junit4] 2> 369050 T1514 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 369050 T1514 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 369052 T1554 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7690 name:ZooKeeperConnection Watcher:127.0.0.1:33456 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 369052 T1514 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 369054 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 369057 T1514 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 369058 T1556 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7193f1 name:ZooKeeperConnection Watcher:127.0.0.1:33456/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 369058 T1514 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 369062 T1514 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 370064 T1514 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43073_ebg%2Flc
[junit4:junit4] 2> 370066 T1514 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43073_ebg%2Flc
[junit4:junit4] 2> 370068 T1536 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 370068 T1543 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 370068 T1556 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 370070 T1536 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 370074 T1557 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 370075 T1557 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 370406 T1537 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 370407 T1537 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "node_name":"127.0.0.1:50855_ebg%2Flc",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50855/ebg/lc",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50855_ebg%2Flc_collection1"}
[junit4:junit4] 2> 370411 T1537 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "node_name":"127.0.0.1:43073_ebg%2Flc",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43073/ebg/lc",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":null}
[junit4:junit4] 2> 370412 T1537 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 370412 T1537 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 370417 T1536 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 370417 T1543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 370417 T1556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 371076 T1557 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371908987762/collection1
[junit4:junit4] 2> 371076 T1557 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 371077 T1557 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 371077 T1557 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 371078 T1557 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371908987762/collection1/'
[junit4:junit4] 2> 371079 T1557 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371908987762/collection1/lib/README' to classloader
[junit4:junit4] 2> 371079 T1557 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371908987762/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 371106 T1557 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 371138 T1557 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 371139 T1557 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 371143 T1557 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 371505 T1557 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 371505 T1557 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 371505 T1557 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 371511 T1557 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 371513 T1557 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 371525 T1557 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 371528 T1557 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 371531 T1557 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 371532 T1557 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 371532 T1557 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 371532 T1557 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 371534 T1557 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 371534 T1557 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 371534 T1557 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 371534 T1557 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371908987762/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/
[junit4:junit4] 2> 371534 T1557 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@147c2e4
[junit4:junit4] 2> 371535 T1557 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 371535 T1557 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1
[junit4:junit4] 2> 371535 T1557 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index/
[junit4:junit4] 2> 371536 T1557 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 371536 T1557 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index
[junit4:junit4] 2> 371539 T1557 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 371539 T1557 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 371541 T1557 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 371542 T1557 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 371542 T1557 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 371543 T1557 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 371544 T1557 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 371544 T1557 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 371545 T1557 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 371545 T1557 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 371546 T1557 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 371548 T1557 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 371551 T1557 oass.SolrIndexSearcher.<init> Opening Searcher@1ff2e6e main
[junit4:junit4] 2> 371552 T1557 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 371552 T1557 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 371556 T1558 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ff2e6e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 371558 T1557 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 371558 T1557 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:43073/ebg/lc collection:collection1 shard:shard1
[junit4:junit4] 2> 371559 T1557 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 371565 T1557 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 371569 T1557 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 371569 T1557 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 371569 T1557 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:43073/ebg/lc/collection1/
[junit4:junit4] 2> 371569 T1557 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 371570 T1557 oasc.SyncStrategy.syncToMe http://127.0.0.1:43073/ebg/lc/collection1/ has no replicas
[junit4:junit4] 2> 371570 T1557 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:43073/ebg/lc/collection1/
[junit4:junit4] 2> 371570 T1557 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 371923 T1537 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 371931 T1536 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 371931 T1543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 371931 T1556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 371977 T1557 oasc.ZkController.register We are http://127.0.0.1:43073/ebg/lc/collection1/ and leader is http://127.0.0.1:43073/ebg/lc/collection1/
[junit4:junit4] 2> 371977 T1557 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43073/ebg/lc
[junit4:junit4] 2> 371977 T1557 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 371978 T1557 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 371978 T1557 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 371979 T1557 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 371980 T1514 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 371981 T1514 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 371981 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 372054 T1514 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 372056 T1514 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59164
[junit4:junit4] 2> 372056 T1514 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 372057 T1514 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 372057 T1514 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371908990820
[junit4:junit4] 2> 372057 T1514 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371908990820/solr.xml
[junit4:junit4] 2> 372058 T1514 oasc.CoreContainer.<init> New CoreContainer 3423626
[junit4:junit4] 2> 372058 T1514 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371908990820/'
[junit4:junit4] 2> 372058 T1514 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371908990820/'
[junit4:junit4] 2> 372102 T1514 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 372103 T1514 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 372103 T1514 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 372103 T1514 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 372104 T1514 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 372104 T1514 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 372104 T1514 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 372105 T1514 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 372105 T1514 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 372105 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 372108 T1514 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 372109 T1514 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33456/solr
[junit4:junit4] 2> 372109 T1514 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 372110 T1514 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 372112 T1570 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b4fb2 name:ZooKeeperConnection Watcher:127.0.0.1:33456 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 372112 T1514 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 372115 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 372118 T1514 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 372120 T1572 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17cd583 name:ZooKeeperConnection Watcher:127.0.0.1:33456/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 372120 T1514 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 372124 T1514 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 373127 T1514 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59164_ebg%2Flc
[junit4:junit4] 2> 373128 T1514 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59164_ebg%2Flc
[junit4:junit4] 2> 373131 T1543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 373132 T1556 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 373132 T1556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 373132 T1536 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 373132 T1536 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 373133 T1543 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 373134 T1572 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 373139 T1573 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 373139 T1573 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 373436 T1537 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 373437 T1537 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "node_name":"127.0.0.1:43073_ebg%2Flc",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43073/ebg/lc",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:43073_ebg%2Flc_collection1"}
[junit4:junit4] 2> 373440 T1537 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "node_name":"127.0.0.1:59164_ebg%2Flc",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:59164/ebg/lc",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":null}
[junit4:junit4] 2> 373441 T1537 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 373441 T1537 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 373445 T1556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 373445 T1572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 373445 T1536 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 373445 T1543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 374140 T1573 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371908990820/collection1
[junit4:junit4] 2> 374141 T1573 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 374141 T1573 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 374142 T1573 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 374143 T1573 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371908990820/collection1/'
[junit4:junit4] 2> 374144 T1573 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371908990820/collection1/lib/README' to classloader
[junit4:junit4] 2> 374144 T1573 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371908990820/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 374173 T1573 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 374206 T1573 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 374208 T1573 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 374211 T1573 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 374604 T1573 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 374604 T1573 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 374605 T1573 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 374612 T1573 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 374614 T1573 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 374627 T1573 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 374631 T1573 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 374635 T1573 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 374636 T1573 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 374637 T1573 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 374637 T1573 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 374638 T1573 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 374638 T1573 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 374639 T1573 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 374639 T1573 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371908990820/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/
[junit4:junit4] 2> 374639 T1573 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@147c2e4
[junit4:junit4] 2> 374640 T1573 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 374640 T1573 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2
[junit4:junit4] 2> 374640 T1573 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/index/
[junit4:junit4] 2> 374641 T1573 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 374641 T1573 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/index
[junit4:junit4] 2> 374645 T1573 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@32e520; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 374645 T1573 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 374648 T1573 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 374648 T1573 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 374649 T1573 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 374649 T1573 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 374650 T1573 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 374650 T1573 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 374651 T1573 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 374651 T1573 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 374652 T1573 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 374654 T1573 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 374658 T1573 oass.SolrIndexSearcher.<init> Opening Searcher@109bd51 main
[junit4:junit4] 2> 374659 T1573 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 374659 T1573 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 374663 T1574 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@109bd51 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 374665 T1573 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 374665 T1573 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59164/ebg/lc collection:collection1 shard:shard1
[junit4:junit4] 2> 374668 T1573 oasc.ZkController.register We are http://127.0.0.1:59164/ebg/lc/collection1/ and leader is http://127.0.0.1:43073/ebg/lc/collection1/
[junit4:junit4] 2> 374668 T1573 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59164/ebg/lc
[junit4:junit4] 2> 374668 T1573 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 374668 T1573 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C971 name=collection1 org.apache.solr.core.SolrCore@178a900 url=http://127.0.0.1:59164/ebg/lc/collection1 node=127.0.0.1:59164_ebg%2Flc C971_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:59164_ebg%2Flc, core=collection1, state=down, base_url=http://127.0.0.1:59164/ebg/lc}
[junit4:junit4] 2> 374669 T1575 C971 P59164 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 374669 T1573 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 374669 T1575 C971 P59164 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 374670 T1575 C971 P59164 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 374670 T1575 C971 P59164 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 374670 T1514 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 374671 T1514 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 374671 T1575 C971 P59164 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 374671 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 374677 T1550 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:59164_ebg%2Flc_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 374743 T1514 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 374745 T1514 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52252
[junit4:junit4] 2> 374745 T1514 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 374745 T1514 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 374746 T1514 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371908993511
[junit4:junit4] 2> 374746 T1514 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371908993511/solr.xml
[junit4:junit4] 2> 374746 T1514 oasc.CoreContainer.<init> New CoreContainer 5709614
[junit4:junit4] 2> 374747 T1514 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371908993511/'
[junit4:junit4] 2> 374747 T1514 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371908993511/'
[junit4:junit4] 2> 374789 T1514 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 374789 T1514 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 374790 T1514 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 374790 T1514 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 374790 T1514 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 374791 T1514 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 374791 T1514 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 374791 T1514 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 374791 T1514 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 374792 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 374795 T1514 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 374796 T1514 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33456/solr
[junit4:junit4] 2> 374796 T1514 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 374797 T1514 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 374799 T1587 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9d39bb name:ZooKeeperConnection Watcher:127.0.0.1:33456 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 374799 T1514 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 374801 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 374804 T1514 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 374806 T1589 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ddcb13 name:ZooKeeperConnection Watcher:127.0.0.1:33456/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 374806 T1514 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 374810 T1514 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 374951 T1537 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 374952 T1537 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "node_name":"127.0.0.1:59164_ebg%2Flc",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:59164/ebg/lc",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:59164_ebg%2Flc_collection1"}
[junit4:junit4] 2> 374955 T1536 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 374956 T1572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 374956 T1543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 374955 T1589 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 374956 T1556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 375678 T1550 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:59164_ebg%2Flc_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 375678 T1550 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={version=2&onlyIfLeader=true&nodeName=127.0.0.1:59164_ebg%252Flc&core=collection1&action=PREPRECOVERY&coreNodeName=127.0.0.1:59164_ebg%252Flc_collection1&state=recovering&wt=javabin&checkLive=true} status=0 QTime=1001
[junit4:junit4] 2> 375814 T1514 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52252_ebg%2Flc
[junit4:junit4] 2> 375815 T1514 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52252_ebg%2Flc
[junit4:junit4] 2> 375817 T1543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 375817 T1572 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 375817 T1556 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 375818 T1572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 375817 T1536 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 375818 T1589 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 375818 T1556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 375818 T1589 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 375818 T1536 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 375819 T1543 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 375825 T1590 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 375825 T1590 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 376461 T1537 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 376462 T1537 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "node_name":"127.0.0.1:52252_ebg%2Flc",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52252/ebg/lc",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":null}
[junit4:junit4] 2> 376462 T1537 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 376463 T1537 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 376468 T1543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 376468 T1572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 376468 T1536 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 376468 T1589 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 376468 T1556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 376827 T1590 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371908993511/collection1
[junit4:junit4] 2> 376827 T1590 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 376828 T1590 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 376828 T1590 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 376830 T1590 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371908993511/collection1/'
[junit4:junit4] 2> 376830 T1590 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371908993511/collection1/lib/README' to classloader
[junit4:junit4] 2> 376830 T1590 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371908993511/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 376863 T1590 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 376893 T1590 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 376895 T1590 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 376898 T1590 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 377226 T1590 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 377226 T1590 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 377227 T1590 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 377232 T1590 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 377235 T1590 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 377249 T1590 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 377251 T1590 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 377254 T1590 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 377255 T1590 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 377255 T1590 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 377255 T1590 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 377256 T1590 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 377256 T1590 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 377257 T1590 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 377257 T1590 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371908993511/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/
[junit4:junit4] 2> 377257 T1590 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@147c2e4
[junit4:junit4] 2> 377257 T1590 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 377258 T1590 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3
[junit4:junit4] 2> 377258 T1590 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/index/
[junit4:junit4] 2> 377258 T1590 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 377258 T1590 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/index
[junit4:junit4] 2> 377261 T1590 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fc965c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 377261 T1590 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 377263 T1590 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 377263 T1590 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 377264 T1590 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 377264 T1590 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 377265 T1590 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 377265 T1590 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 377265 T1590 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 377266 T1590 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 377266 T1590 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 377267 T1590 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 377270 T1590 oass.SolrIndexSearcher.<init> Opening Searcher@12dea9f main
[junit4:junit4] 2> 377271 T1590 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 377271 T1590 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 377276 T1591 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12dea9f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 377278 T1590 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 377279 T1590 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52252/ebg/lc collection:collection1 shard:shard1
[junit4:junit4] 2> 377282 T1590 oasc.ZkController.register We are http://127.0.0.1:52252/ebg/lc/collection1/ and leader is http://127.0.0.1:43073/ebg/lc/collection1/
[junit4:junit4] 2> 377282 T1590 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52252/ebg/lc
[junit4:junit4] 2> 377282 T1590 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 377283 T1590 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C972 name=collection1 org.apache.solr.core.SolrCore@eef1f5 url=http://127.0.0.1:52252/ebg/lc/collection1 node=127.0.0.1:52252_ebg%2Flc C972_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:52252_ebg%2Flc, core=collection1, state=down, base_url=http://127.0.0.1:52252/ebg/lc}
[junit4:junit4] 2> 377283 T1592 C972 P52252 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 377284 T1592 C972 P52252 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 377284 T1590 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 377284 T1592 C972 P52252 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 377284 T1592 C972 P52252 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 377285 T1514 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 377285 T1514 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 377285 T1592 C972 P52252 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 377286 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 377290 T1552 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:52252_ebg%2Flc_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 377403 T1514 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 377405 T1514 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:57495
[junit4:junit4] 2> 377406 T1514 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 377407 T1514 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 377407 T1514 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371908996125
[junit4:junit4] 2> 377407 T1514 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371908996125/solr.xml
[junit4:junit4] 2> 377408 T1514 oasc.CoreContainer.<init> New CoreContainer 32642067
[junit4:junit4] 2> 377408 T1514 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371908996125/'
[junit4:junit4] 2> 377409 T1514 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371908996125/'
[junit4:junit4] 2> 377468 T1514 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 377469 T1514 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 377469 T1514 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 377470 T1514 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 377470 T1514 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 377470 T1514 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 377471 T1514 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 377471 T1514 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 377471 T1514 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 377472 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 377475 T1514 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 377476 T1514 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33456/solr
[junit4:junit4] 2> 377476 T1514 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 377477 T1514 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 377478 T1604 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e4bc79 name:ZooKeeperConnection Watcher:127.0.0.1:33456 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 377479 T1514 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 377480 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 377483 T1514 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 377486 T1606 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11d07cb name:ZooKeeperConnection Watcher:127.0.0.1:33456/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 377487 T1514 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 377490 T1514 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C973 name=collection1 org.apache.solr.core.SolrCore@178a900 url=http://127.0.0.1:59164/ebg/lc/collection1 node=127.0.0.1:59164_ebg%2Flc C973_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:59164_ebg%2Flc, core=collection1, state=recovering, shard=shard1, base_url=http://127.0.0.1:59164/ebg/lc}
[junit4:junit4] 2> 377679 T1575 C973 P59164 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:43073/ebg/lc/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 377679 T1575 C973 P59164 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:59164/ebg/lc START replicas=[http://127.0.0.1:43073/ebg/lc/collection1/] nUpdates=100
[junit4:junit4] 2> 377680 T1575 C973 P59164 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 377680 T1575 C973 P59164 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 377680 T1575 C973 P59164 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 377680 T1575 C973 P59164 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 377680 T1575 C973 P59164 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 377680 T1575 C973 P59164 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:43073/ebg/lc/collection1/. core=collection1
[junit4:junit4] 2> 377681 T1575 C973 P59164 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C974 name=collection1 org.apache.solr.core.SolrCore@1144034 url=http://127.0.0.1:43073/ebg/lc/collection1 node=127.0.0.1:43073_ebg%2Flc C974_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:43073_ebg%2Flc, core=collection1, state=active, shard=shard1, base_url=http://127.0.0.1:43073/ebg/lc, leader=true}
[junit4:junit4] 2> 377684 T1549 C974 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/get params={distrib=false&version=2&qt=/get&wt=javabin&getVersions=100} status=0 QTime=0
[junit4:junit4] 2> 377690 T1551 C974 P43073 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 377692 T1551 C974 P43073 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 377693 T1551 C974 P43073 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 377705 T1551 C974 P43073 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 377705 T1551 C974 P43073 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 377706 T1551 C974 P43073 oass.SolrIndexSearcher.<init> Opening Searcher@11f8365 realtime
[junit4:junit4] 2> 377706 T1551 C974 P43073 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 377706 T1551 C974 P43073 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&softCommit=false&openSearcher=false&wt=javabin&commit=true&commit_end_point=true&waitSearcher=true} {commit=} 0 16
[junit4:junit4] 2> 377707 T1575 C973 P59164 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 377707 T1575 C973 P59164 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 377711 T1551 C974 P43073 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 377711 T1551 C974 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/replication params={version=2&command=indexversion&qt=/replication&wt=javabin} status=0 QTime=3
[junit4:junit4] 2> 377712 T1575 C973 P59164 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 377712 T1575 C973 P59164 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 377712 T1575 C973 P59164 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 377714 T1549 C974 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/replication params={version=2&command=filelist&generation=2&qt=/replication&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 377714 T1575 C973 P59164 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 377715 T1575 C973 P59164 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/index.20130622194956551
[junit4:junit4] 2> 377715 T1575 C973 P59164 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/index.20130622194956551 lockFactory=org.apache.lucene.store.NativeFSLockFactory@527368; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 377718 T1550 C974 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/replication params={checksum=true&file=segments_2&command=filecontent&generation=2&qt=/replication&wt=filestream} status=0 QTime=0
[junit4:junit4] 2> 377719 T1575 C973 P59164 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 377725 T1575 C973 P59164 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 377726 T1575 C973 P59164 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 377727 T1575 C973 P59164 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@32e520; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@32e520; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 377728 T1575 C973 P59164 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 377728 T1575 C973 P59164 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 377728 T1575 C973 P59164 oass.SolrIndexSearcher.<init> Opening Searcher@1b88331 main
[junit4:junit4] 2> 377729 T1574 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b88331 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 377729 T1575 C973 P59164 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/index.20130622194956551 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/index.20130622194956551;done=true>>]
[junit4:junit4] 2> 377730 T1575 C973 P59164 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/index.20130622194956551
[junit4:junit4] 2> 377730 T1575 C973 P59164 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/index.20130622194956551
[junit4:junit4] 2> 377730 T1575 C973 P59164 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 377730 T1575 C973 P59164 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 377730 T1575 C973 P59164 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 377731 T1575 C973 P59164 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 377733 T1575 C973 P59164 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 377974 T1537 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 377975 T1537 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "node_name":"127.0.0.1:52252_ebg%2Flc",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52252/ebg/lc",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:52252_ebg%2Flc_collection1"}
[junit4:junit4] 2> 377996 T1537 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "node_name":"127.0.0.1:59164_ebg%2Flc",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:59164/ebg/lc",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:59164_ebg%2Flc_collection1"}
[junit4:junit4] 2> 378001 T1536 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 378001 T1556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 378002 T1606 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 378002 T1543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 378002 T1572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 378001 T1589 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 378290 T1552 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:52252_ebg%2Flc_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 378291 T1552 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={version=2&onlyIfLeader=true&nodeName=127.0.0.1:52252_ebg%252Flc&core=collection1&action=PREPRECOVERY&coreNodeName=127.0.0.1:52252_ebg%252Flc_collection1&state=recovering&wt=javabin&checkLive=true} status=0 QTime=1001
[junit4:junit4] 2> 378493 T1514 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57495_ebg%2Flc
[junit4:junit4] 2> 378495 T1514 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57495_ebg%2Flc
[junit4:junit4] 2> 378496 T1589 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 378496 T1606 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 378497 T1556 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 378497 T1543 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 378497 T1536 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 378498 T1572 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 378498 T1572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 378497 T1543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 378497 T1556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 378499 T1589 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 378499 T1606 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 378502 T1536 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 378504 T1609 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 378504 T1609 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 379510 T1537 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 379510 T1537 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "node_name":"127.0.0.1:57495_ebg%2Flc",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57495/ebg/lc",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":null}
[junit4:junit4] 2> 379510 T1537 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 379511 T1537 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 379514 T1556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 379514 T1572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 379515 T1606 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 379515 T1543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 379514 T1536 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 379514 T1589 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> ASYNC NEW_CORE C975 name=collection1 org.apache.solr.core.SolrCore@eef1f5 url=http://127.0.0.1:52252/ebg/lc/collection1 node=127.0.0.1:52252_ebg%2Flc C975_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:52252_ebg%2Flc, core=collection1, state=recovering, shard=shard1, base_url=http://127.0.0.1:52252/ebg/lc}
[junit4:junit4] 2> 380292 T1592 C975 P52252 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:43073/ebg/lc/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 380292 T1592 C975 P52252 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:52252/ebg/lc START replicas=[http://127.0.0.1:43073/ebg/lc/collection1/] nUpdates=100
[junit4:junit4] 2> 380292 T1592 C975 P52252 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 380293 T1592 C975 P52252 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 380293 T1592 C975 P52252 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 380293 T1592 C975 P52252 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 380293 T1592 C975 P52252 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 380293 T1592 C975 P52252 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:43073/ebg/lc/collection1/. core=collection1
[junit4:junit4] 2> 380293 T1592 C975 P52252 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 380294 T1552 C974 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/get params={distrib=false&version=2&qt=/get&wt=javabin&getVersions=100} status=0 QTime=0
[junit4:junit4] 2> 380298 T1551 C974 P43073 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 380305 T1551 C974 P43073 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 380305 T1551 C974 P43073 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 380306 T1551 C974 P43073 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 380306 T1551 C974 P43073 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&softCommit=false&openSearcher=false&wt=javabin&commit=true&commit_end_point=true&waitSearcher=true} {commit=} 0 9
[junit4:junit4] 2> 380307 T1592 C975 P52252 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 380307 T1592 C975 P52252 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 380309 T1552 C974 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/replication params={version=2&command=indexversion&qt=/replication&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 380309 T1592 C975 P52252 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 380309 T1592 C975 P52252 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 380310 T1592 C975 P52252 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 380311 T1550 C974 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/replication params={version=2&command=filelist&generation=3&qt=/replication&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 380312 T1592 C975 P52252 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 380313 T1592 C975 P52252 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/index.20130622194959148
[junit4:junit4] 2> 380313 T1592 C975 P52252 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/index.20130622194959148 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a2996; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 380316 T1551 C974 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/replication params={checksum=true&file=segments_3&command=filecontent&generation=3&qt=/replication&wt=filestream} status=0 QTime=0
[junit4:junit4] 2> 380317 T1592 C975 P52252 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 380319 T1592 C975 P52252 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 380319 T1592 C975 P52252 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 380320 T1592 C975 P52252 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fc965c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fc965c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 380321 T1592 C975 P52252 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 380321 T1592 C975 P52252 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 380321 T1592 C975 P52252 oass.SolrIndexSearcher.<init> Opening Searcher@1b835ab main
[junit4:junit4] 2> 380322 T1591 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b835ab main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 380323 T1592 C975 P52252 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/index.20130622194959148 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/index.20130622194959148;done=true>>]
[junit4:junit4] 2> 380323 T1592 C975 P52252 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/index.20130622194959148
[junit4:junit4] 2> 380323 T1592 C975 P52252 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/index.20130622194959148
[junit4:junit4] 2> 380324 T1592 C975 P52252 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 380324 T1592 C975 P52252 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 380324 T1592 C975 P52252 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 380324 T1592 C975 P52252 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 380325 T1592 C975 P52252 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 380506 T1609 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371908996125/collection1
[junit4:junit4] 2> 380507 T1609 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 380507 T1609 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 380508 T1609 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 380509 T1609 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371908996125/collection1/'
[junit4:junit4] 2> 380509 T1609 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371908996125/collection1/lib/README' to classloader
[junit4:junit4] 2> 380510 T1609 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371908996125/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 380546 T1609 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 380576 T1609 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 380577 T1609 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 380581 T1609 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 380930 T1609 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 380930 T1609 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 380930 T1609 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 380936 T1609 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 380938 T1609 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 380950 T1609 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 380953 T1609 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 380956 T1609 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 380957 T1609 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 380957 T1609 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 380958 T1609 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 380959 T1609 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 380959 T1609 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 380959 T1609 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 380959 T1609 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371908996125/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/
[junit4:junit4] 2> 380959 T1609 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@147c2e4
[junit4:junit4] 2> 380960 T1609 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 380960 T1609 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4
[junit4:junit4] 2> 380960 T1609 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/index/
[junit4:junit4] 2> 380960 T1609 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 380961 T1609 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/index
[junit4:junit4] 2> 380976 T1609 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@10bb693; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 380977 T1609 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 380979 T1609 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 380979 T1609 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 380979 T1609 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 380980 T1609 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 380980 T1609 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 380980 T1609 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 380982 T1609 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 380982 T1609 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 380983 T1609 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 380985 T1609 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 380988 T1609 oass.SolrIndexSearcher.<init> Opening Searcher@981dfc main
[junit4:junit4] 2> 380988 T1609 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 380989 T1609 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 380992 T1612 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@981dfc main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 380994 T1609 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 380994 T1609 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57495/ebg/lc collection:collection1 shard:shard1
[junit4:junit4] 2> 380997 T1609 oasc.ZkController.register We are http://127.0.0.1:57495/ebg/lc/collection1/ and leader is http://127.0.0.1:43073/ebg/lc/collection1/
[junit4:junit4] 2> 380997 T1609 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57495/ebg/lc
[junit4:junit4] 2> 380997 T1609 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 380997 T1609 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C976 name=collection1 org.apache.solr.core.SolrCore@d20b8d url=http://127.0.0.1:57495/ebg/lc/collection1 node=127.0.0.1:57495_ebg%2Flc C976_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:57495_ebg%2Flc, core=collection1, state=down, base_url=http://127.0.0.1:57495/ebg/lc}
[junit4:junit4] 2> 380997 T1613 C976 P57495 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 380998 T1613 C976 P57495 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 380998 T1609 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 380998 T1613 C976 P57495 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 380998 T1613 C976 P57495 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 380999 T1514 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 381000 T1514 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 380999 T1613 C976 P57495 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 381000 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 381004 T1550 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:57495_ebg%2Flc_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 381004 T1514 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 381006 T1514 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 381006 T1514 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 381007 T1514 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 381018 T1537 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 381019 T1537 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "node_name":"127.0.0.1:52252_ebg%2Flc",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52252/ebg/lc",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:52252_ebg%2Flc_collection1"}
[junit4:junit4] 2> 381022 T1537 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "node_name":"127.0.0.1:57495_ebg%2Flc",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57495/ebg/lc",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:57495_ebg%2Flc_collection1"}
[junit4:junit4] 2> 381025 T1556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 381026 T1543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 381026 T1536 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 381026 T1606 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 381026 T1572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 381026 T1589 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 382005 T1550 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:57495_ebg%2Flc_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 382005 T1550 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={version=2&onlyIfLeader=true&nodeName=127.0.0.1:57495_ebg%252Flc&core=collection1&action=PREPRECOVERY&coreNodeName=127.0.0.1:57495_ebg%252Flc_collection1&state=recovering&wt=javabin&checkLive=true} status=0 QTime=1001
[junit4:junit4] 2> 382008 T1514 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383010 T1514 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C976_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:57495_ebg%2Flc, core=collection1, state=recovering, shard=shard1, base_url=http://127.0.0.1:57495/ebg/lc}
[junit4:junit4] 2> 384006 T1613 C976 P57495 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:43073/ebg/lc/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 384006 T1613 C976 P57495 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57495/ebg/lc START replicas=[http://127.0.0.1:43073/ebg/lc/collection1/] nUpdates=100
[junit4:junit4] 2> 384007 T1613 C976 P57495 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 384007 T1613 C976 P57495 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 384007 T1613 C976 P57495 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 384007 T1613 C976 P57495 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 384007 T1613 C976 P57495 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 384008 T1613 C976 P57495 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:43073/ebg/lc/collection1/. core=collection1
[junit4:junit4] 2> 384008 T1613 C976 P57495 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C977 name=collection1 org.apache.solr.core.SolrCore@1144034 url=http://127.0.0.1:43073/ebg/lc/collection1 node=127.0.0.1:43073_ebg%2Flc C977_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:43073_ebg%2Flc, core=collection1, state=active, shard=shard1, base_url=http://127.0.0.1:43073/ebg/lc, leader=true}
[junit4:junit4] 2> 384009 T1550 C977 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/get params={distrib=false&version=2&qt=/get&wt=javabin&getVersions=100} status=0 QTime=0
[junit4:junit4] 2> 384012 T1514 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 384013 T1552 C977 P43073 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 384025 T1552 C977 P43073 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 384025 T1552 C977 P43073 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 384026 T1552 C977 P43073 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 384027 T1552 C977 P43073 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&softCommit=false&openSearcher=false&wt=javabin&commit=true&commit_end_point=true&waitSearcher=true} {commit=} 0 15
[junit4:junit4] 2> 384027 T1613 C976 P57495 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 384028 T1613 C976 P57495 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 384030 T1550 C977 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/replication params={version=2&command=indexversion&qt=/replication&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 384030 T1613 C976 P57495 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 384030 T1613 C976 P57495 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 384031 T1613 C976 P57495 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 384033 T1549 C977 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/replication params={version=2&command=filelist&generation=4&qt=/replication&wt=javabin} status=0 QTime=1
[junit4:junit4] 2> 384033 T1613 C976 P57495 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 384034 T1613 C976 P57495 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/index.20130622195002869
[junit4:junit4] 2> 384034 T1613 C976 P57495 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/index.20130622195002869 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f6c06a; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 384037 T1552 C977 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/replication params={checksum=true&file=segments_4&command=filecontent&generation=4&qt=/replication&wt=filestream} status=0 QTime=0
[junit4:junit4] 2> 384038 T1613 C976 P57495 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 384040 T1613 C976 P57495 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 384040 T1613 C976 P57495 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 384042 T1613 C976 P57495 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@10bb693; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@10bb693; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 384043 T1613 C976 P57495 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 384043 T1613 C976 P57495 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 384044 T1613 C976 P57495 oass.SolrIndexSearcher.<init> Opening Searcher@1c7c9f6 main
[junit4:junit4] 2> 384045 T1612 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c7c9f6 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 384045 T1613 C976 P57495 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/index.20130622195002869 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/index.20130622195002869;done=true>>]
[junit4:junit4] 2> 384046 T1613 C976 P57495 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/index.20130622195002869
[junit4:junit4] 2> 384046 T1613 C976 P57495 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/index.20130622195002869
[junit4:junit4] 2> 384046 T1613 C976 P57495 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 384047 T1613 C976 P57495 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 384047 T1613 C976 P57495 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 384047 T1613 C976 P57495 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 384048 T1613 C976 P57495 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 385014 T1514 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 385539 T1537 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 385540 T1537 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "node_name":"127.0.0.1:57495_ebg%2Flc",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57495/ebg/lc",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:57495_ebg%2Flc_collection1"}
[junit4:junit4] 2> 385550 T1556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 385550 T1606 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 385550 T1572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 385550 T1543 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 385550 T1589 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 385550 T1536 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 386016 T1514 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 386017 T1514 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C978 name=collection1 org.apache.solr.core.SolrCore@1a7df9f url=http://127.0.0.1:50855/ebg/lc/collection1 node=127.0.0.1:50855_ebg%2Flc C978_STATE=coll:control_collection core:collection1 props:{collection=control_collection, node_name=127.0.0.1:50855_ebg%2Flc, core=collection1, state=active, shard=shard1, base_url=http://127.0.0.1:50855/ebg/lc, leader=true}
[junit4:junit4] 2> 386024 T1532 C978 P50855 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 386027 T1532 C978 P50855 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c33b7f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 386028 T1532 C978 P50855 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 386035 T1532 C978 P50855 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c33b7f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c33b7f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 386035 T1532 C978 P50855 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 386036 T1532 C978 P50855 oass.SolrIndexSearcher.<init> Opening Searcher@4526e0 main
[junit4:junit4] 2> 386036 T1532 C978 P50855 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 386037 T1540 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4526e0 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 386038 T1532 C978 P50855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&softCommit=false&wt=javabin&commit=true&waitSearcher=true} {commit=} 0 14
[junit4:junit4] 2> C976_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:57495_ebg%2Flc, core=collection1, state=active, shard=shard1, base_url=http://127.0.0.1:57495/ebg/lc}
[junit4:junit4] 2> 386048 T1600 C976 P57495 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:43073/ebg/lc/collection1/, StdNode: http://127.0.0.1:59164/ebg/lc/collection1/, StdNode: http://127.0.0.1:52252/ebg/lc/collection1/, StdNode: http://127.0.0.1:57495/ebg/lc/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 386053 T1549 C977 P43073 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 386054 T1602 C976 P57495 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C979 name=collection1 org.apache.solr.core.SolrCore@eef1f5 url=http://127.0.0.1:52252/ebg/lc/collection1 node=127.0.0.1:52252_ebg%2Flc C979_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:52252_ebg%2Flc, core=collection1, state=active, shard=shard1, base_url=http://127.0.0.1:52252/ebg/lc}
[junit4:junit4] 2> 386059 T1583 C979 P52252 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C980 name=collection1 org.apache.solr.core.SolrCore@178a900 url=http://127.0.0.1:59164/ebg/lc/collection1 node=127.0.0.1:59164_ebg%2Flc C980_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:59164_ebg%2Flc, core=collection1, state=active, shard=shard1, base_url=http://127.0.0.1:59164/ebg/lc}
[junit4:junit4] 2> 386060 T1565 C980 P59164 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 386085 T1602 C976 P57495 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@10bb693; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@10bb693; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 386085 T1549 C977 P43073 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 386085 T1602 C976 P57495 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 386086 T1549 C977 P43073 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 386087 T1602 C976 P57495 oass.SolrIndexSearcher.<init> Opening Searcher@3f236b main
[junit4:junit4] 2> 386087 T1549 C977 P43073 oass.SolrIndexSearcher.<init> Opening Searcher@72a35c main
[junit4:junit4] 2> 386087 T1602 C976 P57495 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 386088 T1549 C977 P43073 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 386088 T1558 oasc.JmxMonitoredMap.put WARN Failed to register info bean: searcher javax.management.InstanceAlreadyExistsException: solr/collection1:type=searcher,id=org.apache.solr.search.SolrIndexSearcher
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:298)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1846)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1723)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2>
[junit4:junit4] 2> 386088 T1612 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3f236b main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 386089 T1558 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@72a35c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 386089 T1602 C976 P57495 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&expungeDeletes=false&softCommit=false&wt=javabin&commit=true&commit_end_point=true&waitSearcher=true} {commit=} 0 35
[junit4:junit4] 2> 386090 T1549 C977 P43073 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&expungeDeletes=false&softCommit=false&wt=javabin&commit=true&commit_end_point=true&waitSearcher=true} {commit=} 0 37
[junit4:junit4] 2> 386105 T1565 C980 P59164 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@32e520; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@32e520; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 386105 T1583 C979 P52252 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fc965c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fc965c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 386106 T1583 C979 P52252 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 386105 T1565 C980 P59164 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 386107 T1583 C979 P52252 oass.SolrIndexSearcher.<init> Opening Searcher@121217f main
[junit4:junit4] 2> 386107 T1565 C980 P59164 oass.SolrIndexSearcher.<init> Opening Searcher@41962d main
[junit4:junit4] 2> 386108 T1591 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@121217f main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 386107 T1583 C979 P52252 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 386109 T1565 C980 P59164 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 386109 T1574 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@41962d main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 386109 T1583 C979 P52252 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&expungeDeletes=false&softCommit=false&wt=javabin&commit=true&commit_end_point=true&waitSearcher=true} {commit=} 0 50
[junit4:junit4] 2> 386110 T1565 C980 P59164 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&expungeDeletes=false&softCommit=false&wt=javabin&commit=true&commit_end_point=true&waitSearcher=true} {commit=} 0 50
[junit4:junit4] 2> 386110 T1600 C976 P57495 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&softCommit=false&wt=javabin&commit=true&waitSearcher=true} {commit=} 0 62
[junit4:junit4] 2> 386111 T1514 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 386113 T1550 C977 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/select params={distrib=false&version=2&rows=0&tests=checkShardConsistency&q=*:*&wt=javabin} hits=0 status=0 QTime=0
[junit4:junit4] 2> 386115 T1566 C980 P59164 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/select params={distrib=false&version=2&rows=0&tests=checkShardConsistency&q=*:*&wt=javabin} hits=0 status=0 QTime=0
[junit4:junit4] 2> 386116 T1584 C979 P52252 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/select params={distrib=false&version=2&rows=0&tests=checkShardConsistency&q=*:*&wt=javabin} hits=0 status=0 QTime=0
[junit4:junit4] 2> 386118 T1599 C976 P57495 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/select params={distrib=false&version=2&rows=0&tests=checkShardConsistency&q=*:*&wt=javabin} hits=0 status=0 QTime=0
[junit4:junit4] 2> 388121 T1530 C978 P50855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&wt=javabin} {deleteByQuery=*:* (-1438550858878943232)} 0 1
[junit4:junit4] 2> 388129 T1567 C980 P59164 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=FROMLEADER&_version_=-1438550858883137536&wt=javabin&update.from=http://127.0.0.1:43073/ebg/lc/collection1/} {deleteByQuery=*:* (-1438550858883137536)} 0 0
[junit4:junit4] 2> 388129 T1582 C979 P52252 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=FROMLEADER&_version_=-1438550858883137536&wt=javabin&update.from=http://127.0.0.1:43073/ebg/lc/collection1/} {deleteByQuery=*:* (-1438550858883137536)} 0 0
[junit4:junit4] 2> 388129 T1601 C976 P57495 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=FROMLEADER&_version_=-1438550858883137536&wt=javabin&update.from=http://127.0.0.1:43073/ebg/lc/collection1/} {deleteByQuery=*:* (-1438550858883137536)} 0 0
[junit4:junit4] 2> 388131 T1551 C977 P43073 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=TOLEADER&wt=javabin} {deleteByQuery=*:* (-1438550858883137536)} 0 6
[junit4:junit4] 2> 388131 T1600 C976 P57495 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&wt=javabin} {deleteByQuery=*:*} 0 8
[junit4:junit4] 2> 388135 T1529 C978 P50855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&wt=javabin} {add=[0 (1438550858892574720)]} 0 2
[junit4:junit4] 2> 388145 T1584 C979 P52252 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=FROMLEADER&wt=javabin&distrib.from=http://127.0.0.1:43073/ebg/lc/collection1/} {add=[0 (1438550858899914752)]} 0 1
[junit4:junit4] 2> 388145 T1566 C980 P59164 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=FROMLEADER&wt=javabin&distrib.from=http://127.0.0.1:43073/ebg/lc/collection1/} {add=[0 (1438550858899914752)]} 0 1
[junit4:junit4] 2> 388145 T1599 C976 P57495 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=FROMLEADER&wt=javabin&distrib.from=http://127.0.0.1:43073/ebg/lc/collection1/} {add=[0 (1438550858899914752)]} 0 1
[junit4:junit4] 2> 388146 T1552 C977 P43073 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=TOLEADER&wt=javabin&distrib.from=http://127.0.0.1:57495/ebg/lc/collection1/} {add=[0 (1438550858899914752)]} 0 6
[junit4:junit4] 2> 388147 T1602 C976 P57495 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&wt=javabin} {add=[0]} 0 10
[junit4:junit4] 2> 388150 T1532 C978 P50855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&wt=javabin} {add=[1 (1438550858908303360)]} 0 1
[junit4:junit4] 2> 388157 T1585 C979 P52252 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=FROMLEADER&wt=javabin&distrib.from=http://127.0.0.1:43073/ebg/lc/collection1/} {add=[1 (1438550858912497664)]} 0 1
[junit4:junit4] 2> 388157 T1601 C976 P57495 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=FROMLEADER&wt=javabin&distrib.from=http://127.0.0.1:43073/ebg/lc/collection1/} {add=[1 (1438550858912497664)]} 0 1
[junit4:junit4] 2> 388157 T1568 C980 P59164 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=FROMLEADER&wt=javabin&distrib.from=http://127.0.0.1:43073/ebg/lc/collection1/} {add=[1 (1438550858912497664)]} 0 1
[junit4:junit4] 2> 388158 T1549 C977 P43073 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=TOLEADER&wt=javabin&distrib.from=http://127.0.0.1:59164/ebg/lc/collection1/} {add=[1 (1438550858912497664)]} 0 5
[junit4:junit4] 2> 388158 T1565 C980 P59164 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&wt=javabin} {add=[1]} 0 7
[junit4:junit4] 2> 388162 T1531 C978 P50855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&wt=javabin} {add=[2 (1438550858920886272)]} 0 1
[junit4:junit4] 2> 388164 T1551 C977 P43073 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:59164/ebg/lc/collection1/]
[junit4:junit4] 2> 388164 T1551 C977 P43073 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:59164/ebg/lc/collection1/ against:[http://127.0.0.1:59164/ebg/lc/collection1/] result:true
[junit4:junit4] 2> 388164 T1551 C977 P43073 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:52252/ebg/lc/collection1/ against:[http://127.0.0.1:59164/ebg/lc/collection1/] result:false
[junit4:junit4] 2> 388165 T1551 C977 P43073 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:57495/ebg/lc/collection1/ against:[http://127.0.0.1:59164/ebg/lc/collection1/] result:false
[junit4:junit4] 2> 388169 T1600 C976 P57495 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:59164/ebg/lc/collection1/&wt=javabin&distrib.from=http://127.0.0.1:43073/ebg/lc/collection1/} {add=[2 (1438550858925080576)]} 0 1
[junit4:junit4] 2> 388169 T1583 C979 P52252 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:59164/ebg/lc/collection1/&wt=javabin&distrib.from=http://127.0.0.1:43073/ebg/lc/collection1/} {add=[2 (1438550858925080576)]} 0 1
[junit4:junit4] 2> 388170 T1551 C977 P43073 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&test.distrib.skip.servers=http://127.0.0.1:59164/ebg/lc/collection1/&wt=javabin} {add=[2 (1438550858925080576)]} 0 6
[junit4:junit4] 2> 388173 T1530 C978 P50855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&wt=javabin} {add=[3 (1438550858932420608)]} 0 1
[junit4:junit4] 2> 388176 T1552 C977 P43073 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:59164/ebg/lc/collection1/]
[junit4:junit4] 2> 388177 T1552 C977 P43073 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:59164/ebg/lc/collection1/ against:[http://127.0.0.1:59164/ebg/lc/collection1/] result:true
[junit4:junit4] 2> 388177 T1552 C977 P43073 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:52252/ebg/lc/collection1/ against:[http://127.0.0.1:59164/ebg/lc/collection1/] result:false
[junit4:junit4] 2> 388177 T1552 C977 P43073 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:57495/ebg/lc/collection1/ against:[http://127.0.0.1:59164/ebg/lc/collection1/] result:false
[junit4:junit4] 2> 388181 T1599 C976 P57495 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:59164/ebg/lc/collection1/&wt=javabin&distrib.from=http://127.0.0.1:43073/ebg/lc/collection1/} {add=[3 (1438550858938712064)]} 0 1
[junit4:junit4] 2> 388181 T1585 C979 P52252 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:59164/ebg/lc/collection1/&wt=javabin&distrib.from=http://127.0.0.1:43073/ebg/lc/collection1/} {add=[3 (1438550858938712064)]} 0 1
[junit4:junit4] 2> 388182 T1552 C977 P43073 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:59164/ebg/lc/collection1/&wt=javabin&distrib.from=http://127.0.0.1:52252/ebg/lc/collection1/} {add=[3 (1438550858938712064)]} 0 6
[junit4:junit4] 2> 388182 T1584 C979 P52252 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&test.distrib.skip.servers=http://127.0.0.1:59164/ebg/lc/collection1/&test.distrib.skip.servers=http://127.0.0.1:52252/ebg/lc/collection1/&wt=javabin} {add=[3]} 0 7
[junit4:junit4] 2> 388184 T1532 C978 P50855 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 388225 T1532 C978 P50855 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c33b7f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c33b7f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 388225 T1532 C978 P50855 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 388227 T1532 C978 P50855 oass.SolrIndexSearcher.<init> Opening Searcher@1185a82 main
[junit4:junit4] 2> 388228 T1532 C978 P50855 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 388228 T1540 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1185a82 main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 388228 T1532 C978 P50855 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&softCommit=false&wt=javabin&commit=true&waitSearcher=true} {commit=} 0 44
[junit4:junit4] 2> 388231 T1549 C977 P43073 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:43073/ebg/lc/collection1/, StdNode: http://127.0.0.1:59164/ebg/lc/collection1/, StdNode: http://127.0.0.1:52252/ebg/lc/collection1/, StdNode: http://127.0.0.1:57495/ebg/lc/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 388233 T1550 C977 P43073 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 388233 T1567 C980 P59164 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 388233 T1583 C979 P52252 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 388237 T1602 C976 P57495 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 388285 T1583 C979 P52252 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fc965c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@fc965c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 388285 T1550 C977 P43073 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1dceeb7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 388285 T1567 C980 P59164 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@32e520; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@32e520; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 388286 T1550 C977 P43073 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 388285 T1583 C979 P52252 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 388286 T1567 C980 P59164 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 388288 T1550 C977 P43073 oass.SolrIndexSearcher.<init> Opening Searcher@25fcf7 main
[junit4:junit4] 2> 388288 T1583 C979 P52252 oass.SolrIndexSearcher.<init> Opening Searcher@a0a30d main
[junit4:junit4] 2> 388289 T1550 C977 P43073 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 388289 T1591 oasc.JmxMonitoredMap.put WARN Failed to register info bean: searcher javax.management.InstanceAlreadyExistsException: solr/collection1:type=searcher,id=org.apache.solr.search.SolrIndexSearcher
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:298)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1846)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1723)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2>
[junit4:junit4] 2> 388289 T1558 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@25fcf7 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 388289 T1567 C980 P59164 oass.SolrIndexSearcher.<init> Opening Searcher@1e423f8 main
[junit4:junit4] 2> 388289 T1583 C979 P52252 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 388290 T1567 C980 P59164 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 388290 T1550 C977 P43073 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&expungeDeletes=false&softCommit=false&wt=javabin&commit=true&commit_end_point=true&waitSearcher=true} {commit=} 0 58
[junit4:junit4] 2> 388290 T1591 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@a0a30d main{StandardDirectoryReader(segments_5:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 388291 T1574 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1e423f8 main{StandardDirectoryReader(segments_4:3:nrt _0(4.4):C2)}
[junit4:junit4] 2> 388292 T1583 C979 P52252 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&expungeDeletes=false&softCommit=false&wt=javabin&commit=true&commit_end_point=true&waitSearcher=true} {commit=} 0 59
[junit4:junit4] 2> 388292 T1567 C980 P59164 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&expungeDeletes=false&softCommit=false&wt=javabin&commit=true&commit_end_point=true&waitSearcher=true} {commit=} 0 59
[junit4:junit4] 2> 388295 T1602 C976 P57495 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@10bb693; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@10bb693; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 388295 T1602 C976 P57495 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 388297 T1602 C976 P57495 oass.SolrIndexSearcher.<init> Opening Searcher@1f8d24c main
[junit4:junit4] 2> 388298 T1602 C976 P57495 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 388299 T1612 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1f8d24c main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 388299 T1602 C976 P57495 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&expungeDeletes=false&softCommit=false&wt=javabin&commit=true&commit_end_point=true&waitSearcher=true} {commit=} 0 62
[junit4:junit4] 2> 388300 T1549 C977 P43073 oasup.LogUpdateProcessor.finish [collection1] webapp=/ebg/lc path=/update params={version=2&softCommit=false&wt=javabin&commit=true&waitSearcher=true} {commit=} 0 69
[junit4:junit4] 2> 388300 T1514 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 388301 T1514 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 388302 T1514 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 388304 T1551 C977 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/select params={distrib=false&version=2&rows=0&tests=checkShardConsistency&q=*:*&wt=javabin} hits=4 status=0 QTime=0
[junit4:junit4] 2> 388306 T1566 C980 P59164 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/select params={distrib=false&version=2&rows=0&tests=checkShardConsistency&q=*:*&wt=javabin} hits=2 status=0 QTime=0
[junit4:junit4] 2> 388309 T1582 C979 P52252 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/select params={distrib=false&version=2&rows=0&tests=checkShardConsistency&q=*:*&wt=javabin} hits=4 status=0 QTime=0
[junit4:junit4] 2> 388311 T1601 C976 P57495 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/select params={distrib=false&version=2&rows=0&tests=checkShardConsistency&q=*:*&wt=javabin} hits=4 status=0 QTime=0
[junit4:junit4] 2> 388312 T1514 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 388316 T1585 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : version=2&collection=collection1&action=SYNCSHARD&wt=javabin&shard=shard1
[junit4:junit4] 2> 388316 T1585 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 388319 T1552 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 388319 T1552 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:43073/ebg/lc/collection1/
[junit4:junit4] 2> 388320 T1552 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:43073/ebg/lc START replicas=[http://127.0.0.1:59164/ebg/lc/collection1/, http://127.0.0.1:52252/ebg/lc/collection1/, http://127.0.0.1:57495/ebg/lc/collection1/] nUpdates=100
[junit4:junit4] 2> 388321 T1568 C980 P59164 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/get params={distrib=false&version=2&qt=/get&wt=javabin&getVersions=100} status=0 QTime=0
[junit4:junit4] 2> 388322 T1600 C976 P57495 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/get params={distrib=false&version=2&qt=/get&wt=javabin&getVersions=100} status=0 QTime=0
[junit4:junit4] 2> 388322 T1552 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:43073/ebg/lc Received 3 versions from 127.0.0.1:59164/ebg/lc/collection1/
[junit4:junit4] 2> 388321 T1584 C979 P52252 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/get params={distrib=false&version=2&qt=/get&wt=javabin&getVersions=100} status=0 QTime=0
[junit4:junit4] 2> 388322 T1552 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:43073/ebg/lc Our versions are newer. ourLowThreshold=1438550858883137536 otherHigh=1438550858912497664
[junit4:junit4] 2> 388323 T1552 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:43073/ebg/lc Received 5 versions from 127.0.0.1:57495/ebg/lc/collection1/
[junit4:junit4] 2> 388323 T1552 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:43073/ebg/lc Our versions are newer. ourLowThreshold=1438550858883137536 otherHigh=1438550858925080576
[junit4:junit4] 2> 388323 T1552 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:43073/ebg/lc Received 5 versions from 127.0.0.1:52252/ebg/lc/collection1/
[junit4:junit4] 2> 388323 T1552 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:43073/ebg/lc Our versions are newer. ourLowThreshold=1438550858883137536 otherHigh=1438550858925080576
[junit4:junit4] 2> 388324 T1552 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:43073/ebg/lc DONE. sync succeeded
[junit4:junit4] 2> 388324 T1552 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 388324 T1552 oasc.SyncStrategy.syncToMe http://127.0.0.1:43073/ebg/lc/collection1/: try and ask http://127.0.0.1:59164/ebg/lc/collection1/ to sync
[junit4:junit4] 2> 388325 T1552 oasc.SyncStrategy.syncToMe http://127.0.0.1:43073/ebg/lc/collection1/: try and ask http://127.0.0.1:52252/ebg/lc/collection1/ to sync
[junit4:junit4] 2> 388325 T1552 oasc.SyncStrategy.syncToMe http://127.0.0.1:43073/ebg/lc/collection1/: try and ask http://127.0.0.1:57495/ebg/lc/collection1/ to sync
[junit4:junit4] 2> 388326 T1565 C980 P59164 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:59164/ebg/lc START replicas=[http://127.0.0.1:43073/ebg/lc/collection1/] nUpdates=100
[junit4:junit4] 2> 388327 T1583 C979 P52252 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:52252/ebg/lc START replicas=[http://127.0.0.1:43073/ebg/lc/collection1/] nUpdates=100
[junit4:junit4] 2> 388327 T1602 C976 P57495 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57495/ebg/lc START replicas=[http://127.0.0.1:43073/ebg/lc/collection1/] nUpdates=100
[junit4:junit4] 2> 388328 T1550 C977 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/get params={distrib=false&version=2&qt=/get&wt=javabin&getVersions=100} status=0 QTime=0
[junit4:junit4] 2> 388328 T1549 C977 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/get params={distrib=false&version=2&qt=/get&wt=javabin&getVersions=100} status=0 QTime=0
[junit4:junit4] 2> 388329 T1551 C977 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/get params={distrib=false&version=2&qt=/get&wt=javabin&getVersions=100} status=0 QTime=1
[junit4:junit4] 2> 388329 T1565 C980 P59164 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:59164/ebg/lc Received 5 versions from 127.0.0.1:43073/ebg/lc/collection1/
[junit4:junit4] 2> 388329 T1583 C979 P52252 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:52252/ebg/lc Received 5 versions from 127.0.0.1:43073/ebg/lc/collection1/
[junit4:junit4] 2> 388329 T1565 C980 P59164 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:59164/ebg/lc Requesting updates from 127.0.0.1:43073/ebg/lc/collection1/n=2 versions=[1438550858938712064, 1438550858925080576]
[junit4:junit4] 2> 388329 T1602 C976 P57495 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:57495/ebg/lc Received 5 versions from 127.0.0.1:43073/ebg/lc/collection1/
[junit4:junit4] 2> 388329 T1583 C979 P52252 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:52252/ebg/lc Our versions are newer. ourLowThreshold=1438550858883137536 otherHigh=1438550858925080576
[junit4:junit4] 2> 388330 T1602 C976 P57495 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:57495/ebg/lc Our versions are newer. ourLowThreshold=1438550858883137536 otherHigh=1438550858925080576
[junit4:junit4] 2> 388330 T1583 C979 P52252 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:52252/ebg/lc DONE. sync succeeded
[junit4:junit4] 2> 388331 T1602 C976 P57495 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57495/ebg/lc DONE. sync succeeded
[junit4:junit4] 2> 388331 T1583 C979 P52252 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/get params={distrib=false&version=2&qt=/get&wt=javabin&getVersions=100&sync=http://127.0.0.1:43073/ebg/lc/collection1/} status=0 QTime=4
[junit4:junit4] 2> 388331 T1602 C976 P57495 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/get params={distrib=false&version=2&qt=/get&wt=javabin&getVersions=100&sync=http://127.0.0.1:43073/ebg/lc/collection1/} status=0 QTime=4
[junit4:junit4] 2> 388332 T1552 oasc.SyncStrategy.syncToMe http://127.0.0.1:43073/ebg/lc/collection1/: sync completed with http://127.0.0.1:52252/ebg/lc/collection1/
[junit4:junit4] 2> 388332 T1550 C977 P43073 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/get params={distrib=false&version=2&getUpdates=1438550858938712064,1438550858925080576&qt=/get&wt=javabin} status=0 QTime=1
[junit4:junit4] 2> 388332 T1552 oasc.SyncStrategy.syncToMe http://127.0.0.1:43073/ebg/lc/collection1/: sync completed with http://127.0.0.1:57495/ebg/lc/collection1/
[junit4:junit4] 2> 388335 T1565 C980 P59164 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1438550858925080576), 3 (1438550858938712064)]} 0 3
[junit4:junit4] 2> 388335 T1565 C980 P59164 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:59164/ebg/lc DONE. sync succeeded
[junit4:junit4] 2> 388335 T1565 C980 P59164 oasc.SolrCore.execute [collection1] webapp=/ebg/lc path=/get params={distrib=false&version=2&qt=/get&wt=javabin&getVersions=100&sync=http://127.0.0.1:43073/ebg/lc/collection1/} status=0 QTime=9
[junit4:junit4] 2> 388336 T1552 oasc.SyncStrategy.syncToMe http://127.0.0.1:43073/ebg/lc/collection1/: sync completed with http://127.0.0.1:59164/ebg/lc/collection1/
[junit4:junit4] 2> 388336 T1552 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={version=2&collection=collection1&core=collection1&action=REQUESTSYNCSHARD&wt=javabin&shard=shard1} status=0 QTime=17
[junit4:junit4] 2> 388336 T1585 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={version=2&collection=collection1&action=SYNCSHARD&wt=javabin&shard=shard1} status=0 QTime=21
[junit4:junit4] 2> 388337 T1514 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 388337 T1514 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 388338 T1514 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 388339 T1514 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 388341 T1531 C978 P50855 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 388405 T1531 C978 P50855 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371908984249/control/data/index lockFactory=org.apache.lucene.store.Nativ
[...truncated too long message...]
lection/leader_elect/shard1/election/89909427191939075-127.0.0.1:50855_ebg%2Flc_collection1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "node_name":"127.0.0.1:52252_ebg%2Flc",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:52252/ebg/lc"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89909427191939084-127.0.0.1:57495_ebg%2Flc_collection1-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89909427191939082-127.0.0.1:52252_ebg%2Flc_collection1-n_0000000002 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89909427191939086-127.0.0.1:43073_ebg%2Flc_collection1-n_0000000004 (0)
[junit4:junit4] 1> /solr/clusterstate.json (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "control_collection":{
[junit4:junit4] 1> "router":"compositeId",
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"127.0.0.1:50855_ebg%2Flc_collection1":{
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "node_name":"127.0.0.1:50855_ebg%2Flc",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:50855/ebg/lc",
[junit4:junit4] 1> "leader":"true"}}}}},
[junit4:junit4] 1> "collection1":{
[junit4:junit4] 1> "router":"compositeId",
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{
[junit4:junit4] 1> "127.0.0.1:43073_ebg%2Flc_collection1":{
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:43073_ebg%2Flc",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:43073/ebg/lc"},
[junit4:junit4] 1> "127.0.0.1:59164_ebg%2Flc_collection1":{
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:59164_ebg%2Flc",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:59164/ebg/lc"},
[junit4:junit4] 1> "127.0.0.1:52252_ebg%2Flc_collection1":{
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:52252_ebg%2Flc",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:52252/ebg/lc",
[junit4:junit4] 1> "leader":"true"},
[junit4:junit4] 1> "127.0.0.1:57495_ebg%2Flc_collection1":{
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:57495_ebg%2Flc",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:57495/ebg/lc"}}}}}}
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89909427191939075-127.0.0.1:50855_ebg%2Flc-n_0000000000"}
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89909427191939084-127.0.0.1:57495_ebg%2Flc-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89909427191939082-127.0.0.1:52252_ebg%2Flc-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89909427191939086-127.0.0.1:43073_ebg%2Flc-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89909427191939075-127.0.0.1:50855_ebg%2Flc-n_0000000000 (0)
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=FD6B0656D09C0FB9 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ko_KR -Dtests.timezone=Antarctica/Vostok -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 101s J0 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:43073/ebg/lc/collection1lastClient and got 246 from http://127.0.0.1:52252/ebg/lc/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([FD6B0656D09C0FB9:7C8D884EA7C36F85]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2> 466642 T1514 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 101236 T1513 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4] 2> 467803 T1606 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 467803 T1606 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 467803 T1606 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Asserting, sim=DefaultSimilarity, locale=ko_KR, timezone=Antarctica/Vostok
[junit4:junit4] 2> NOTE: Linux 3.2.0-48-generic i386/Oracle Corporation 1.8.0-ea (32-bit)/cpus=8,threads=3,free=118563880,total=220200960
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestWriterPerf, ExternalFileFieldSortTest, MBeansHandlerTest, SolrInfoMBeanTest, TestSolrXmlPersistence, TestValueSourceCache, TestPluginEnable, TestXIncludeConfig, RecoveryZkTest, SolrCoreTest, TestFieldResource, ReturnFieldsTest, DocValuesTest, TestStressRecovery, ClusterStateUpdateTest, TestSurroundQueryParser, CoreAdminHandlerTest, PreAnalyzedUpdateProcessorTest, BinaryUpdateRequestHandlerTest, OverseerCollectionProcessorTest, DistributedSpellCheckComponentTest, DistributedTermsComponentTest, PeerSyncTest, TestCharFilters, TermsComponentTest, TestFieldTypeCollectionResource, CacheHeaderTest, UpdateRequestProcessorFactoryTest, TestCoreContainer, AutoCommitTest, SoftAutoCommitTest, TestJoin, SliceStateTest, TestGroupingSearch, TestFastWriter, TestReplicationHandler, IndexSchemaTest, SimpleFacetsTest, TermVectorComponentTest, TestDynamicFieldCollectionResource, TestSolrDeletionPolicy2, SyncSliceTest]
[junit4:junit4] Completed on J0 in 102.51s, 1 test, 1 failure <<< FAILURES!
[...truncated 681 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:392: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:372: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1246: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:890: There were test failures: 298 suites, 1256 tests, 1 failure, 16 ignored (7 assumptions)
Total time: 39 minutes 46 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0-ea-b93 -server -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.7.0_25) - Build # 6183
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6183/
Java: 64bit/jdk1.7.0_25 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:44824/e_t/collection1lastClient and got 249 from http://127.0.0.1:39874/e_t/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:44824/e_t/collection1lastClient and got 249 from http://127.0.0.1:39874/e_t/collection1
at __randomizedtesting.SeedInfo.seed([BC174470A104C5EC:3DF1CA68D65BA5D0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
Build Log:
[...truncated 9694 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 780879 T2179 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /e_t/
[junit4:junit4] 2> 780883 T2179 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1371904103237
[junit4:junit4] 2> 780884 T2179 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 780884 T2180 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 780984 T2179 oasc.ZkTestServer.run start zk server on port:35303
[junit4:junit4] 2> 780985 T2179 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 780988 T2186 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@479e1bf5 name:ZooKeeperConnection Watcher:127.0.0.1:35303 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 780988 T2179 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 780988 T2179 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 780990 T2179 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 780991 T2188 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d551e98 name:ZooKeeperConnection Watcher:127.0.0.1:35303/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 780992 T2179 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 780992 T2179 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 780997 T2179 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 780999 T2179 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 781000 T2179 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 781002 T2179 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 781002 T2179 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 781004 T2179 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 781005 T2179 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 781007 T2179 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 781007 T2179 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 781008 T2179 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 781009 T2179 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 781010 T2179 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 781011 T2179 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 781012 T2179 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 781013 T2179 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 781014 T2179 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 781014 T2179 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 781016 T2179 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 781016 T2179 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 781018 T2179 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 781018 T2179 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 781081 T2179 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 781099 T2179 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:58087
[junit4:junit4] 2> 781100 T2179 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 781100 T2179 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 781100 T2179 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371904103373
[junit4:junit4] 2> 781101 T2179 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371904103373/solr.xml
[junit4:junit4] 2> 781101 T2179 oasc.CoreContainer.<init> New CoreContainer 977602485
[junit4:junit4] 2> 781101 T2179 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371904103373/'
[junit4:junit4] 2> 781101 T2179 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371904103373/'
[junit4:junit4] 2> 781147 T2179 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 781147 T2179 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 781147 T2179 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 781148 T2179 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 781148 T2179 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 781148 T2179 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 781148 T2179 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 781149 T2179 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 781149 T2179 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 781149 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 781152 T2179 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 781152 T2179 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35303/solr
[junit4:junit4] 2> 781153 T2179 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 781153 T2179 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 781154 T2199 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3170b49b name:ZooKeeperConnection Watcher:127.0.0.1:35303 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 781155 T2179 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 781156 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 781159 T2179 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 781160 T2201 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5f826513 name:ZooKeeperConnection Watcher:127.0.0.1:35303/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 781160 T2179 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 781161 T2179 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 781163 T2179 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 781164 T2179 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 781165 T2179 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58087_e_t
[junit4:junit4] 2> 781166 T2179 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58087_e_t
[junit4:junit4] 2> 781168 T2179 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 781171 T2179 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 781172 T2179 oasc.Overseer.start Overseer (id=89909107309871107-127.0.0.1:58087_e_t-n_0000000000) starting
[junit4:junit4] 2> 781177 T2179 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 781179 T2203 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 781180 T2179 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 781181 T2179 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 781182 T2179 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 781184 T2202 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 781186 T2204 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 781186 T2204 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 782686 T2202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 782687 T2202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:58087_e_t",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58087/e_t"}
[junit4:junit4] 2> 782687 T2202 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 782687 T2202 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 782689 T2201 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 783187 T2204 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371904103373/collection1
[junit4:junit4] 2> 783188 T2204 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 783188 T2204 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 783188 T2204 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 783190 T2204 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371904103373/collection1/'
[junit4:junit4] 2> 783190 T2204 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371904103373/collection1/lib/README' to classloader
[junit4:junit4] 2> 783190 T2204 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371904103373/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 783215 T2204 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 783256 T2204 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 783257 T2204 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 783263 T2204 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 783625 T2204 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 783625 T2204 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 783625 T2204 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 783631 T2204 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 783633 T2204 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 783643 T2204 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 783645 T2204 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 783648 T2204 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 783648 T2204 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 783649 T2204 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 783649 T2204 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 783650 T2204 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 783650 T2204 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 783650 T2204 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 783656 T2204 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371904103373/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371904103236/control/data/
[junit4:junit4] 2> 783656 T2204 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e229394
[junit4:junit4] 2> 783657 T2204 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 783657 T2204 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/control/data
[junit4:junit4] 2> 783657 T2204 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371904103236/control/data/index/
[junit4:junit4] 2> 783657 T2204 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371904103236/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 783658 T2204 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/control/data/index
[junit4:junit4] 2> 783660 T2204 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 783661 T2204 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 783662 T2204 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 783663 T2204 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 783663 T2204 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 783663 T2204 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 783664 T2204 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 783664 T2204 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 783664 T2204 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 783665 T2204 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 783665 T2204 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 783667 T2204 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 783670 T2204 oass.SolrIndexSearcher.<init> Opening Searcher@7aa51b70 main
[junit4:junit4] 2> 783670 T2204 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 783670 T2204 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 783674 T2205 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7aa51b70 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 783675 T2204 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 783675 T2204 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58087/e_t collection:control_collection shard:shard1
[junit4:junit4] 2> 783676 T2204 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 783680 T2204 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 783682 T2204 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 783682 T2204 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 783682 T2204 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:58087/e_t/collection1/
[junit4:junit4] 2> 783682 T2204 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 783682 T2204 oasc.SyncStrategy.syncToMe http://127.0.0.1:58087/e_t/collection1/ has no replicas
[junit4:junit4] 2> 783683 T2204 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:58087/e_t/collection1/
[junit4:junit4] 2> 783683 T2204 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 784192 T2202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 784202 T2201 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 784238 T2204 oasc.ZkController.register We are http://127.0.0.1:58087/e_t/collection1/ and leader is http://127.0.0.1:58087/e_t/collection1/
[junit4:junit4] 2> 784238 T2204 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58087/e_t
[junit4:junit4] 2> 784238 T2204 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 784239 T2204 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 784239 T2204 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 784245 T2204 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 784246 T2179 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 784246 T2179 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 784247 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 784249 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 784250 T2179 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 784252 T2208 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6bd2c9b4 name:ZooKeeperConnection Watcher:127.0.0.1:35303/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 784253 T2179 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 784254 T2179 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 784255 T2179 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 784315 T2179 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 784317 T2179 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:44824
[junit4:junit4] 2> 784318 T2179 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 784319 T2179 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 784319 T2179 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371904106609
[junit4:junit4] 2> 784319 T2179 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371904106609/solr.xml
[junit4:junit4] 2> 784320 T2179 oasc.CoreContainer.<init> New CoreContainer 1410430286
[junit4:junit4] 2> 784320 T2179 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371904106609/'
[junit4:junit4] 2> 784320 T2179 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371904106609/'
[junit4:junit4] 2> 784365 T2179 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 784365 T2179 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 784366 T2179 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 784366 T2179 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 784366 T2179 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 784366 T2179 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 784367 T2179 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 784367 T2179 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 784367 T2179 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 784367 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 784370 T2179 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 784370 T2179 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35303/solr
[junit4:junit4] 2> 784371 T2179 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 784371 T2179 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 784373 T2219 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@311e4da1 name:ZooKeeperConnection Watcher:127.0.0.1:35303 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 784373 T2179 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 784374 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 784377 T2179 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 784379 T2221 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ec0cefe name:ZooKeeperConnection Watcher:127.0.0.1:35303/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 784379 T2179 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 784381 T2179 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 785383 T2179 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44824_e_t
[junit4:junit4] 2> 785385 T2179 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44824_e_t
[junit4:junit4] 2> 785387 T2208 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 785387 T2221 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 785387 T2201 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 785388 T2201 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 785391 T2222 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 785392 T2222 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 785710 T2202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 785711 T2202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:58087_e_t_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:58087_e_t",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58087/e_t"}
[junit4:junit4] 2> 785713 T2202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44824_e_t",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44824/e_t"}
[junit4:junit4] 2> 785714 T2202 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 785714 T2202 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 785716 T2201 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 785716 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 785716 T2208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 786393 T2222 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371904106609/collection1
[junit4:junit4] 2> 786393 T2222 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 786394 T2222 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 786394 T2222 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 786395 T2222 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371904106609/collection1/'
[junit4:junit4] 2> 786395 T2222 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371904106609/collection1/lib/README' to classloader
[junit4:junit4] 2> 786395 T2222 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371904106609/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 786420 T2222 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 786454 T2222 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 786455 T2222 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 786459 T2222 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 787952 T3 oasc.CoreContainer.finalize ERROR CoreContainer was not shutdown prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=530158209
[junit4:junit4] 2> 787953 T3 oasc.CoreContainer.finalize ERROR CoreContainer was not shutdown prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=192869350
[junit4:junit4] 2> 787953 T3 oasc.CoreContainer.finalize ERROR CoreContainer was not shutdown prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=39188805
[junit4:junit4] 2> 788228 T2222 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 788229 T2222 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 788229 T2222 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 788235 T2222 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 788237 T2222 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 788246 T2222 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 788248 T2222 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 788251 T2222 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 788251 T2222 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 788251 T2222 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 788252 T2222 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 788252 T2222 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 788252 T2222 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 788253 T2222 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 788253 T2222 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371904106609/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/
[junit4:junit4] 2> 788253 T2222 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e229394
[junit4:junit4] 2> 788253 T2222 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 788254 T2222 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1
[junit4:junit4] 2> 788254 T2222 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index/
[junit4:junit4] 2> 788254 T2222 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 788255 T2222 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index
[junit4:junit4] 2> 788257 T2222 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 788257 T2222 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 788259 T2222 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 788259 T2222 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 788260 T2222 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 788260 T2222 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 788261 T2222 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 788261 T2222 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 788261 T2222 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 788262 T2222 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 788262 T2222 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 788264 T2222 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 788267 T2222 oass.SolrIndexSearcher.<init> Opening Searcher@3ccc0576 main
[junit4:junit4] 2> 788267 T2222 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 788268 T2222 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 788271 T2223 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3ccc0576 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 788273 T2222 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 788273 T2222 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44824/e_t collection:collection1 shard:shard1
[junit4:junit4] 2> 788274 T2222 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 788278 T2222 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 788279 T2222 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 788279 T2222 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 788279 T2222 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44824/e_t/collection1/
[junit4:junit4] 2> 788279 T2222 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 788280 T2222 oasc.SyncStrategy.syncToMe http://127.0.0.1:44824/e_t/collection1/ has no replicas
[junit4:junit4] 2> 788280 T2222 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44824/e_t/collection1/
[junit4:junit4] 2> 788280 T2222 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 789447 T2202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 789455 T2201 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 789455 T2208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 789455 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 789486 T2222 oasc.ZkController.register We are http://127.0.0.1:44824/e_t/collection1/ and leader is http://127.0.0.1:44824/e_t/collection1/
[junit4:junit4] 2> 789486 T2222 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44824/e_t
[junit4:junit4] 2> 789487 T2222 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 789487 T2222 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 789487 T2222 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 789488 T2222 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 789489 T2179 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 789490 T2179 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 789490 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 789554 T2179 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 789556 T2179 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:56367
[junit4:junit4] 2> 789557 T2179 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 789557 T2179 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 789557 T2179 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371904111845
[junit4:junit4] 2> 789557 T2179 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371904111845/solr.xml
[junit4:junit4] 2> 789558 T2179 oasc.CoreContainer.<init> New CoreContainer 1779347200
[junit4:junit4] 2> 789558 T2179 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371904111845/'
[junit4:junit4] 2> 789558 T2179 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371904111845/'
[junit4:junit4] 2> 789601 T2179 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 789601 T2179 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 789602 T2179 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 789602 T2179 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 789602 T2179 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 789602 T2179 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 789602 T2179 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 789603 T2179 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 789603 T2179 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 789603 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 789606 T2179 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 789606 T2179 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35303/solr
[junit4:junit4] 2> 789606 T2179 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 789607 T2179 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 789609 T2235 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6844a2ba name:ZooKeeperConnection Watcher:127.0.0.1:35303 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 789609 T2179 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 789610 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 789613 T2179 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 789614 T2237 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@312e374d name:ZooKeeperConnection Watcher:127.0.0.1:35303/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 789614 T2179 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 789617 T2179 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 790619 T2179 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56367_e_t
[junit4:junit4] 2> 790620 T2179 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56367_e_t
[junit4:junit4] 2> 790622 T2201 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 790623 T2208 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 790623 T2237 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 790623 T2208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 790623 T2201 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 790623 T2221 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 790623 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 790628 T2238 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 790628 T2238 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 790960 T2202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 790961 T2202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:44824_e_t_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44824_e_t",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44824/e_t"}
[junit4:junit4] 2> 790963 T2202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56367_e_t",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56367/e_t"}
[junit4:junit4] 2> 790964 T2202 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 790964 T2202 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 790967 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 790967 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 790967 T2208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 790967 T2201 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 791629 T2238 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371904111845/collection1
[junit4:junit4] 2> 791629 T2238 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 791630 T2238 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 791630 T2238 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 791631 T2238 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371904111845/collection1/'
[junit4:junit4] 2> 791632 T2238 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371904111845/collection1/lib/README' to classloader
[junit4:junit4] 2> 791632 T2238 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371904111845/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 791654 T2238 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 791681 T2238 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 791682 T2238 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 791686 T2238 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 791971 T2238 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 791971 T2238 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 791972 T2238 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 791977 T2238 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 791979 T2238 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 791988 T2238 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 791991 T2238 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 791993 T2238 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 791994 T2238 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 791994 T2238 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 791994 T2238 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 791995 T2238 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 791995 T2238 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 791995 T2238 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 791995 T2238 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371904111845/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/
[junit4:junit4] 2> 791995 T2238 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e229394
[junit4:junit4] 2> 791996 T2238 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 791996 T2238 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2
[junit4:junit4] 2> 791996 T2238 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/index/
[junit4:junit4] 2> 791996 T2238 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 791997 T2238 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/index
[junit4:junit4] 2> 791999 T2238 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 791999 T2238 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 792001 T2238 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 792001 T2238 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 792002 T2238 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 792002 T2238 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 792003 T2238 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 792003 T2238 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 792003 T2238 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 792003 T2238 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 792004 T2238 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 792006 T2238 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 792008 T2238 oass.SolrIndexSearcher.<init> Opening Searcher@39d53a3 main
[junit4:junit4] 2> 792009 T2238 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 792009 T2238 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 792013 T2239 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@39d53a3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 792015 T2238 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 792015 T2238 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56367/e_t collection:collection1 shard:shard1
[junit4:junit4] 2> 792018 T2238 oasc.ZkController.register We are http://127.0.0.1:56367/e_t/collection1/ and leader is http://127.0.0.1:44824/e_t/collection1/
[junit4:junit4] 2> 792018 T2238 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56367/e_t
[junit4:junit4] 2> 792018 T2238 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 792018 T2238 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C3020 name=collection1 org.apache.solr.core.SolrCore@dcd56b4 url=http://127.0.0.1:56367/e_t/collection1 node=127.0.0.1:56367_e_t C3020_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:56367_e_t, base_url=http://127.0.0.1:56367/e_t}
[junit4:junit4] 2> 792018 T2240 C3020 P56367 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 792019 T2238 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 792019 T2240 C3020 P56367 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 792019 T2240 C3020 P56367 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 792019 T2240 C3020 P56367 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 792020 T2179 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 792020 T2179 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 792020 T2240 C3020 P56367 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 792020 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 792024 T2216 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:56367_e_t_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 792095 T2179 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 792098 T2179 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:39874
[junit4:junit4] 2> 792098 T2179 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 792099 T2179 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 792099 T2179 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371904114376
[junit4:junit4] 2> 792099 T2179 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371904114376/solr.xml
[junit4:junit4] 2> 792099 T2179 oasc.CoreContainer.<init> New CoreContainer 426062260
[junit4:junit4] 2> 792100 T2179 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371904114376/'
[junit4:junit4] 2> 792100 T2179 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371904114376/'
[junit4:junit4] 2> 792141 T2179 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 792141 T2179 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 792142 T2179 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 792142 T2179 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 792142 T2179 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 792142 T2179 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 792142 T2179 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 792143 T2179 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 792143 T2179 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 792143 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 792146 T2179 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 792146 T2179 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35303/solr
[junit4:junit4] 2> 792146 T2179 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 792147 T2179 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 792149 T2252 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d77a747 name:ZooKeeperConnection Watcher:127.0.0.1:35303 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 792150 T2179 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 792151 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 792154 T2179 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 792155 T2254 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18df98cc name:ZooKeeperConnection Watcher:127.0.0.1:35303/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 792155 T2179 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 792157 T2179 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 792497 T2202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 792498 T2202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:56367_e_t_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56367_e_t",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56367/e_t"}
[junit4:junit4] 2> 792501 T2254 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 792501 T2208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 792501 T2201 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 792501 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 792501 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 793024 T2216 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:56367_e_t_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 793025 T2216 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:56367_e_t_collection1&state=recovering&nodeName=127.0.0.1:56367_e_t&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 793159 T2179 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39874_e_t
[junit4:junit4] 2> 793160 T2179 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39874_e_t
[junit4:junit4] 2> 793162 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 793162 T2201 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 793162 T2221 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 793162 T2201 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 793162 T2208 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 793162 T2254 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 793162 T2208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 793162 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 793168 T2237 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 793168 T2254 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 793173 T2255 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 793173 T2255 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 794005 T2202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 794005 T2202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:39874_e_t",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39874/e_t"}
[junit4:junit4] 2> 794006 T2202 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 794006 T2202 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 794009 T2201 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 794009 T2254 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 794010 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 794009 T2208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 794009 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 794174 T2255 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371904114376/collection1
[junit4:junit4] 2> 794174 T2255 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 794175 T2255 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 794175 T2255 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 794176 T2255 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371904114376/collection1/'
[junit4:junit4] 2> 794177 T2255 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371904114376/collection1/lib/README' to classloader
[junit4:junit4] 2> 794177 T2255 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371904114376/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 794212 T2255 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 794249 T2255 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 794250 T2255 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 794254 T2255 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 794572 T2255 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 794572 T2255 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 794572 T2255 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 794578 T2255 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 794580 T2255 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 794590 T2255 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 794593 T2255 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 794595 T2255 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 794596 T2255 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 794596 T2255 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 794597 T2255 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 794597 T2255 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 794598 T2255 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 794598 T2255 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 794598 T2255 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371904114376/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/
[junit4:junit4] 2> 794598 T2255 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e229394
[junit4:junit4] 2> 794599 T2255 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 794599 T2255 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3
[junit4:junit4] 2> 794599 T2255 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index/
[junit4:junit4] 2> 794599 T2255 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 794600 T2255 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index
[junit4:junit4] 2> 794603 T2255 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 794603 T2255 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 794605 T2255 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 794605 T2255 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 794606 T2255 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 794606 T2255 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 794606 T2255 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 794607 T2255 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 794607 T2255 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 794607 T2255 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 794608 T2255 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 794610 T2255 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 794612 T2255 oass.SolrIndexSearcher.<init> Opening Searcher@7ae6a5c5 main
[junit4:junit4] 2> 794613 T2255 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 794613 T2255 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 794618 T2256 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7ae6a5c5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 794620 T2255 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 794621 T2255 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39874/e_t collection:collection1 shard:shard1
[junit4:junit4] 2> 794623 T2255 oasc.ZkController.register We are http://127.0.0.1:39874/e_t/collection1/ and leader is http://127.0.0.1:44824/e_t/collection1/
[junit4:junit4] 2> 794623 T2255 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39874/e_t
[junit4:junit4] 2> 794624 T2255 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 794624 T2255 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C3021 name=collection1 org.apache.solr.core.SolrCore@1196c7b6 url=http://127.0.0.1:39874/e_t/collection1 node=127.0.0.1:39874_e_t C3021_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:39874_e_t, base_url=http://127.0.0.1:39874/e_t}
[junit4:junit4] 2> 794624 T2257 C3021 P39874 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 794625 T2255 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 794625 T2257 C3021 P39874 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 794625 T2257 C3021 P39874 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 794625 T2257 C3021 P39874 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 794626 T2179 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 794626 T2179 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 794626 T2257 C3021 P39874 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 794626 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 794630 T2216 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:39874_e_t_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 794690 T2179 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 794692 T2179 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47823
[junit4:junit4] 2> 794693 T2179 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 794693 T2179 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 794694 T2179 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371904116982
[junit4:junit4] 2> 794694 T2179 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371904116982/solr.xml
[junit4:junit4] 2> 794694 T2179 oasc.CoreContainer.<init> New CoreContainer 1742386999
[junit4:junit4] 2> 794694 T2179 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371904116982/'
[junit4:junit4] 2> 794695 T2179 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371904116982/'
[junit4:junit4] 2> 794732 T2179 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 794733 T2179 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 794733 T2179 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 794733 T2179 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 794733 T2179 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 794733 T2179 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 794734 T2179 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 794734 T2179 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 794734 T2179 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 794734 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 794737 T2179 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 794737 T2179 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:35303/solr
[junit4:junit4] 2> 794738 T2179 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 794738 T2179 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 794740 T2269 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6789b136 name:ZooKeeperConnection Watcher:127.0.0.1:35303 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 794741 T2179 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 794742 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 794745 T2179 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 794746 T2271 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@606a0554 name:ZooKeeperConnection Watcher:127.0.0.1:35303/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 794746 T2179 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 794749 T2179 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C3022 name=collection1 org.apache.solr.core.SolrCore@dcd56b4 url=http://127.0.0.1:56367/e_t/collection1 node=127.0.0.1:56367_e_t C3022_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:56367_e_t, base_url=http://127.0.0.1:56367/e_t}
[junit4:junit4] 2> 795026 T2240 C3022 P56367 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:44824/e_t/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 795026 T2240 C3022 P56367 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56367/e_t START replicas=[http://127.0.0.1:44824/e_t/collection1/] nUpdates=100
[junit4:junit4] 2> 795026 T2240 C3022 P56367 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 795027 T2240 C3022 P56367 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 795027 T2240 C3022 P56367 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 795027 T2240 C3022 P56367 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 795027 T2240 C3022 P56367 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 795027 T2240 C3022 P56367 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:44824/e_t/collection1/. core=collection1
[junit4:junit4] 2> 795027 T2240 C3022 P56367 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C3023 name=collection1 org.apache.solr.core.SolrCore@7e463875 url=http://127.0.0.1:44824/e_t/collection1 node=127.0.0.1:44824_e_t C3023_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44824_e_t, base_url=http://127.0.0.1:44824/e_t, leader=true}
[junit4:junit4] 2> 795031 T2217 C3023 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 795033 T2214 C3023 P44824 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 795036 T2214 C3023 P44824 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 795036 T2214 C3023 P44824 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 795045 T2214 C3023 P44824 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 795045 T2214 C3023 P44824 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 795046 T2214 C3023 P44824 oass.SolrIndexSearcher.<init> Opening Searcher@6b4f4a68 realtime
[junit4:junit4] 2> 795046 T2214 C3023 P44824 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 795046 T2214 C3023 P44824 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 795047 T2240 C3022 P56367 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 795047 T2240 C3022 P56367 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 795049 T2214 C3023 P44824 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 795049 T2214 C3023 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 795050 T2240 C3022 P56367 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 795050 T2240 C3022 P56367 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 795050 T2240 C3022 P56367 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 795052 T2215 C3023 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 795053 T2240 C3022 P56367 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 795053 T2240 C3022 P56367 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/index.20130622092837406
[junit4:junit4] 2> 795054 T2240 C3022 P56367 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/index.20130622092837406 lockFactory=org.apache.lucene.store.NativeFSLockFactory@48e11e4a fullCopy=false
[junit4:junit4] 2> 795056 T2217 C3023 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 795057 T2240 C3022 P56367 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 795065 T2240 C3022 P56367 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 795065 T2240 C3022 P56367 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 795067 T2240 C3022 P56367 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 795067 T2240 C3022 P56367 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 795068 T2240 C3022 P56367 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 795068 T2240 C3022 P56367 oass.SolrIndexSearcher.<init> Opening Searcher@1b5f14d main
[junit4:junit4] 2> 795069 T2239 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b5f14d main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 795069 T2240 C3022 P56367 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/index.20130622092837406 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/index.20130622092837406;done=true>>]
[junit4:junit4] 2> 795069 T2240 C3022 P56367 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/index.20130622092837406
[junit4:junit4] 2> 795070 T2240 C3022 P56367 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/index.20130622092837406
[junit4:junit4] 2> 795070 T2240 C3022 P56367 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 795070 T2240 C3022 P56367 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 795070 T2240 C3022 P56367 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 795070 T2240 C3022 P56367 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 795072 T2240 C3022 P56367 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 795513 T2202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 795514 T2202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:39874_e_t_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:39874_e_t",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39874/e_t"}
[junit4:junit4] 2> 795517 T2202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:56367_e_t_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:56367_e_t",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:56367/e_t"}
[junit4:junit4] 2> 795538 T2201 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 795538 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 795538 T2208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 795538 T2254 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 795538 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 795538 T2271 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 795630 T2216 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:39874_e_t_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 795631 T2216 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:39874_e_t_collection1&state=recovering&nodeName=127.0.0.1:39874_e_t&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 795751 T2179 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47823_e_t
[junit4:junit4] 2> 795752 T2179 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47823_e_t
[junit4:junit4] 2> 795753 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 795754 T2201 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 795754 T2201 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 795754 T2271 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 795754 T2254 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 795754 T2221 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 795754 T2254 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 795754 T2271 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 795754 T2208 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 795755 T2237 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 795755 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 795755 T2208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 795760 T2274 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 795760 T2274 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 797043 T2202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 797043 T2202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:47823_e_t",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:47823/e_t"}
[junit4:junit4] 2> 797044 T2202 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 797044 T2202 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 797047 T2201 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 797047 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 797047 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 797047 T2254 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 797047 T2208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 797047 T2271 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> C3021_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:39874_e_t, base_url=http://127.0.0.1:39874/e_t}
[junit4:junit4] 2> 797632 T2257 C3021 P39874 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:44824/e_t/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 797632 T2257 C3021 P39874 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:39874/e_t START replicas=[http://127.0.0.1:44824/e_t/collection1/] nUpdates=100
[junit4:junit4] 2> 797632 T2257 C3021 P39874 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 797632 T2257 C3021 P39874 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 797632 T2257 C3021 P39874 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 797633 T2257 C3021 P39874 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 797633 T2257 C3021 P39874 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 797633 T2257 C3021 P39874 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:44824/e_t/collection1/. core=collection1
[junit4:junit4] 2> 797633 T2257 C3021 P39874 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 797634 T2216 C3023 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 797636 T2217 C3023 P44824 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 797645 T2217 C3023 P44824 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 797645 T2217 C3023 P44824 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 797646 T2217 C3023 P44824 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 797646 T2217 C3023 P44824 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 797647 T2257 C3021 P39874 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 797647 T2257 C3021 P39874 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 797648 T2215 C3023 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 797649 T2257 C3021 P39874 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 797649 T2257 C3021 P39874 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 797649 T2257 C3021 P39874 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 797651 T2216 C3023 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=0
[junit4:junit4] 2> 797651 T2257 C3021 P39874 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 797652 T2257 C3021 P39874 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index.20130622092840005
[junit4:junit4] 2> 797653 T2257 C3021 P39874 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index.20130622092840005 lockFactory=org.apache.lucene.store.NativeFSLockFactory@492778a6 fullCopy=false
[junit4:junit4] 2> ASYNC NEW_CORE C3024 name=collection1 org.apache.solr.core.SolrCore@7e463875 url=http://127.0.0.1:44824/e_t/collection1 node=127.0.0.1:44824_e_t C3024_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44824_e_t, base_url=http://127.0.0.1:44824/e_t, leader=true}
[junit4:junit4] 2> 797661 T2217 C3024 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/replication params={file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C3025 name=collection1 org.apache.solr.core.SolrCore@1196c7b6 url=http://127.0.0.1:39874/e_t/collection1 node=127.0.0.1:39874_e_t C3025_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:39874_e_t, base_url=http://127.0.0.1:39874/e_t}
[junit4:junit4] 2> 797662 T2257 C3025 P39874 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 797666 T2257 C3025 P39874 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 797666 T2257 C3025 P39874 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 797668 T2257 C3025 P39874 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 797668 T2257 C3025 P39874 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 797669 T2257 C3025 P39874 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 797669 T2257 C3025 P39874 oass.SolrIndexSearcher.<init> Opening Searcher@3ca1cdcb main
[junit4:junit4] 2> 797670 T2256 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3ca1cdcb main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 797671 T2257 C3025 P39874 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index.20130622092840005 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index.20130622092840005;done=true>>]
[junit4:junit4] 2> 797671 T2257 C3025 P39874 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index.20130622092840005
[junit4:junit4] 2> 797671 T2257 C3025 P39874 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index.20130622092840005
[junit4:junit4] 2> 797671 T2257 C3025 P39874 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 797671 T2257 C3025 P39874 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 797671 T2257 C3025 P39874 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 797672 T2257 C3025 P39874 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 797673 T2257 C3025 P39874 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 797761 T2274 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371904116982/collection1
[junit4:junit4] 2> 797762 T2274 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 797763 T2274 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 797763 T2274 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 797764 T2274 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371904116982/collection1/'
[junit4:junit4] 2> 797765 T2274 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371904116982/collection1/lib/README' to classloader
[junit4:junit4] 2> 797765 T2274 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371904116982/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 797794 T2274 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 797823 T2274 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 797824 T2274 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 797829 T2274 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 798137 T2274 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 798137 T2274 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 798138 T2274 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 798143 T2274 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 798145 T2274 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 798156 T2274 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 798158 T2274 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 798161 T2274 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 798161 T2274 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 798162 T2274 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 798162 T2274 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 798163 T2274 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 798163 T2274 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 798163 T2274 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 798163 T2274 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371904116982/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/
[junit4:junit4] 2> 798163 T2274 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e229394
[junit4:junit4] 2> 798164 T2274 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 798164 T2274 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4
[junit4:junit4] 2> 798164 T2274 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/index/
[junit4:junit4] 2> 798165 T2274 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 798165 T2274 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/index
[junit4:junit4] 2> 798186 T2274 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 798187 T2274 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 798188 T2274 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 798189 T2274 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 798189 T2274 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 798190 T2274 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 798190 T2274 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 798190 T2274 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 798191 T2274 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 798192 T2274 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 798192 T2274 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 798194 T2274 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 798196 T2274 oass.SolrIndexSearcher.<init> Opening Searcher@1563a3fc main
[junit4:junit4] 2> 798197 T2274 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 798197 T2274 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 798203 T2277 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1563a3fc main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 798205 T2274 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 798205 T2274 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:47823/e_t collection:collection1 shard:shard1
[junit4:junit4] 2> 798208 T2274 oasc.ZkController.register We are http://127.0.0.1:47823/e_t/collection1/ and leader is http://127.0.0.1:44824/e_t/collection1/
[junit4:junit4] 2> 798208 T2274 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:47823/e_t
[junit4:junit4] 2> 798208 T2274 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 798209 T2274 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C3026 name=collection1 org.apache.solr.core.SolrCore@dbadbae url=http://127.0.0.1:47823/e_t/collection1 node=127.0.0.1:47823_e_t C3026_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:47823_e_t, base_url=http://127.0.0.1:47823/e_t}
[junit4:junit4] 2> 798209 T2278 C3026 P47823 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 798209 T2274 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 798210 T2278 C3026 P47823 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 798210 T2278 C3026 P47823 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 798210 T2278 C3026 P47823 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 798211 T2179 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 798211 T2179 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 798211 T2278 C3026 P47823 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 798211 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 798214 T2179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 798215 T2216 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:47823_e_t_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 798215 T2179 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 798215 T2179 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 798216 T2179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 798551 T2202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 798551 T2202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:39874_e_t_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:39874_e_t",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39874/e_t"}
[junit4:junit4] 2> 798554 T2202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:47823_e_t_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:47823_e_t",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:47823/e_t"}
[junit4:junit4] 2> 798556 T2201 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 798557 T2254 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 798557 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 798556 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 798556 T2208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 798556 T2271 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 799215 T2216 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:47823_e_t_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 799216 T2216 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:47823_e_t_collection1&state=recovering&nodeName=127.0.0.1:47823_e_t&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 799217 T2179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 800219 T2179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C3026_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:47823_e_t, base_url=http://127.0.0.1:47823/e_t}
[junit4:junit4] 2> 801216 T2278 C3026 P47823 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:44824/e_t/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 801217 T2278 C3026 P47823 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:47823/e_t START replicas=[http://127.0.0.1:44824/e_t/collection1/] nUpdates=100
[junit4:junit4] 2> 801217 T2278 C3026 P47823 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 801217 T2278 C3026 P47823 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 801217 T2278 C3026 P47823 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 801217 T2278 C3026 P47823 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 801217 T2278 C3026 P47823 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 801218 T2278 C3026 P47823 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:44824/e_t/collection1/. core=collection1
[junit4:junit4] 2> 801218 T2278 C3026 P47823 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C3027 name=collection1 org.apache.solr.core.SolrCore@7e463875 url=http://127.0.0.1:44824/e_t/collection1 node=127.0.0.1:44824_e_t C3027_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44824_e_t, base_url=http://127.0.0.1:44824/e_t, leader=true}
[junit4:junit4] 2> 801219 T2214 C3027 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 801220 T2179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 801221 T2217 C3027 P44824 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 801355 T2217 C3027 P44824 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 801355 T2217 C3027 P44824 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 801355 T2217 C3027 P44824 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 801356 T2217 C3027 P44824 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 135
[junit4:junit4] 2> 801357 T2278 C3026 P47823 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 801357 T2278 C3026 P47823 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 801359 T2214 C3027 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 801359 T2278 C3026 P47823 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 801359 T2278 C3026 P47823 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 801359 T2278 C3026 P47823 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 801361 T2215 C3027 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=0
[junit4:junit4] 2> 801362 T2278 C3026 P47823 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 801363 T2278 C3026 P47823 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/index.20130622092843715
[junit4:junit4] 2> 801363 T2278 C3026 P47823 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/index.20130622092843715 lockFactory=org.apache.lucene.store.NativeFSLockFactory@39defc07 fullCopy=false
[junit4:junit4] 2> 801365 T2217 C3027 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/replication params={file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0
[junit4:junit4] 2> 801366 T2278 C3026 P47823 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 801368 T2278 C3026 P47823 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 801368 T2278 C3026 P47823 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 801370 T2278 C3026 P47823 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 801370 T2278 C3026 P47823 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 801370 T2278 C3026 P47823 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 801371 T2278 C3026 P47823 oass.SolrIndexSearcher.<init> Opening Searcher@4f95600e main
[junit4:junit4] 2> 801372 T2277 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4f95600e main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 801373 T2278 C3026 P47823 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/index.20130622092843715 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/index.20130622092843715;done=true>>]
[junit4:junit4] 2> 801373 T2278 C3026 P47823 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/index.20130622092843715
[junit4:junit4] 2> 801373 T2278 C3026 P47823 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/index.20130622092843715
[junit4:junit4] 2> 801373 T2278 C3026 P47823 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 801373 T2278 C3026 P47823 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 801373 T2278 C3026 P47823 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 801374 T2278 C3026 P47823 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 801375 T2278 C3026 P47823 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 801562 T2202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 801563 T2202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:47823_e_t_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:47823_e_t",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:47823/e_t"}
[junit4:junit4] 2> 801566 T2201 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 801566 T2254 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 801566 T2208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 801566 T2237 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 801566 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 801566 T2271 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 802221 T2179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 802222 T2179 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C3028 name=collection1 org.apache.solr.core.SolrCore@67fe6888 url=http://127.0.0.1:58087/e_t/collection1 node=127.0.0.1:58087_e_t C3028_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:58087_e_t, base_url=http://127.0.0.1:58087/e_t, leader=true}
[junit4:junit4] 2> 802228 T2195 C3028 P58087 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 802230 T2195 C3028 P58087 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 802230 T2195 C3028 P58087 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 802255 T2195 C3028 P58087 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/control/data/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 802255 T2195 C3028 P58087 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 802256 T2195 C3028 P58087 oass.SolrIndexSearcher.<init> Opening Searcher@7f7293a1 main
[junit4:junit4] 2> 802256 T2195 C3028 P58087 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 802257 T2205 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7f7293a1 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 802258 T2195 C3028 P58087 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 30
[junit4:junit4] 2> 802262 T2217 C3027 P44824 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:44824/e_t/collection1/, StdNode: http://127.0.0.1:56367/e_t/collection1/, StdNode: http://127.0.0.1:39874/e_t/collection1/, StdNode: http://127.0.0.1:47823/e_t/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 802265 T2214 C3027 P44824 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C3029 name=collection1 org.apache.solr.core.SolrCore@dcd56b4 url=http://127.0.0.1:56367/e_t/collection1 node=127.0.0.1:56367_e_t C3029_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:56367_e_t, base_url=http://127.0.0.1:56367/e_t}
[junit4:junit4] 2> 802269 T2230 C3029 P56367 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C3030 name=collection1 org.apache.solr.core.SolrCore@1196c7b6 url=http://127.0.0.1:39874/e_t/collection1 node=127.0.0.1:39874_e_t C3030_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:39874_e_t, base_url=http://127.0.0.1:39874/e_t}
[junit4:junit4] 2> 802269 T2247 C3030 P39874 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C3026_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:47823_e_t, base_url=http://127.0.0.1:47823/e_t}
[junit4:junit4] 2> 802269 T2267 C3026 P47823 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 802285 T2214 C3027 P44824 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 802285 T2214 C3027 P44824 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 802286 T2214 C3027 P44824 oass.SolrIndexSearcher.<init> Opening Searcher@46c3397c main
[junit4:junit4] 2> 802286 T2214 C3027 P44824 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 802287 T2223 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@46c3397c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 802287 T2214 C3027 P44824 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 22
[junit4:junit4] 2> 802315 T2247 C3030 P39874 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 802315 T2230 C3029 P56367 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 802315 T2267 C3026 P47823 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 802315 T2230 C3029 P56367 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 802315 T2247 C3030 P39874 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 802316 T2267 C3026 P47823 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 802317 T2230 C3029 P56367 oass.SolrIndexSearcher.<init> Opening Searcher@13aa07b1 main
[junit4:junit4] 2> 802317 T2247 C3030 P39874 oass.SolrIndexSearcher.<init> Opening Searcher@95b81ea main
[junit4:junit4] 2> 802317 T2267 C3026 P47823 oass.SolrIndexSearcher.<init> Opening Searcher@351c7e4a main
[junit4:junit4] 2> 802317 T2230 C3029 P56367 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 802318 T2267 C3026 P47823 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 802318 T2239 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@13aa07b1 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 802318 T2256 oasc.JmxMonitoredMap.put WARN Failed to register info bean: searcher javax.management.InstanceAlreadyExistsException: solr/collection1:type=searcher,id=org.apache.solr.search.SolrIndexSearcher
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:298)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1846)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1723)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2>
[junit4:junit4] 2> 802318 T2247 C3030 P39874 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 802319 T2230 C3029 P56367 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 50
[junit4:junit4] 2> 802318 T2277 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@351c7e4a main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 802319 T2256 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@95b81ea main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 802320 T2267 C3026 P47823 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 51
[junit4:junit4] 2> 802320 T2247 C3030 P39874 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 51
[junit4:junit4] 2> 802320 T2217 C3027 P44824 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 58
[junit4:junit4] 2> 802321 T2179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 802323 T2215 C3027 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 802324 T2231 C3029 P56367 oasc.SolrCore.execute [collection1] webapp=/e_t path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 802326 T2249 C3030 P39874 oasc.SolrCore.execute [collection1] webapp=/e_t path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 802327 T2265 C3026 P47823 oasc.SolrCore.execute [collection1] webapp=/e_t path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 804332 T2197 C3028 P58087 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438545741541801984)} 0 2
[junit4:junit4] 2> 804339 T2265 C3026 P47823 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={update.distrib=FROMLEADER&_version_=-1438545741547044864&update.from=http://127.0.0.1:44824/e_t/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438545741547044864)} 0 1
[junit4:junit4] 2> 804339 T2250 C3030 P39874 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={update.distrib=FROMLEADER&_version_=-1438545741547044864&update.from=http://127.0.0.1:44824/e_t/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438545741547044864)} 0 1
[junit4:junit4] 2> 804339 T2230 C3029 P56367 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={update.distrib=FROMLEADER&_version_=-1438545741547044864&update.from=http://127.0.0.1:44824/e_t/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438545741547044864)} 0 0
[junit4:junit4] 2> 804340 T2215 C3027 P44824 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438545741547044864)} 0 4
[junit4:junit4] 2> 804340 T2249 C3030 P39874 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 6
[junit4:junit4] 2> 804345 T2194 C3028 P58087 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={wt=javabin&version=2} {add=[0 (1438545741554384896)]} 0 3
[junit4:junit4] 2> 804355 T2233 C3029 P56367 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={distrib.from=http://127.0.0.1:44824/e_t/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438545741559627776)]} 0 2
[junit4:junit4] 2> 804355 T2247 C3030 P39874 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={distrib.from=http://127.0.0.1:44824/e_t/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438545741559627776)]} 0 2
[junit4:junit4] 2> 804355 T2264 C3026 P47823 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={distrib.from=http://127.0.0.1:44824/e_t/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438545741559627776)]} 0 2
[junit4:junit4] 2> 804356 T2216 C3027 P44824 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={distrib.from=http://127.0.0.1:56367/e_t/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1438545741559627776)]} 0 8
[junit4:junit4] 2> 804357 T2232 C3029 P56367 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={wt=javabin&version=2} {add=[0]} 0 11
[junit4:junit4] 2> 804360 T2195 C3028 P58087 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={wt=javabin&version=2} {add=[1 (1438545741571162112)]} 0 1
[junit4:junit4] 2> 804367 T2248 C3030 P39874 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={distrib.from=http://127.0.0.1:44824/e_t/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438545741575356416)]} 0 1
[junit4:junit4] 2> 804367 T2266 C3026 P47823 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={distrib.from=http://127.0.0.1:44824/e_t/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438545741575356416)]} 0 1
[junit4:junit4] 2> 804367 T2231 C3029 P56367 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={distrib.from=http://127.0.0.1:44824/e_t/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438545741575356416)]} 0 1
[junit4:junit4] 2> 804368 T2214 C3027 P44824 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={distrib.from=http://127.0.0.1:47823/e_t/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1438545741575356416)]} 0 5
[junit4:junit4] 2> 804368 T2267 C3026 P47823 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={wt=javabin&version=2} {add=[1]} 0 7
[junit4:junit4] 2> 804371 T2197 C3028 P58087 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={wt=javabin&version=2} {add=[2 (1438545741583745024)]} 0 0
[junit4:junit4] 2> 804376 T2217 C3027 P44824 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:56367/e_t/collection1/]
[junit4:junit4] 2> 804376 T2217 C3027 P44824 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:56367/e_t/collection1/ against:[http://127.0.0.1:56367/e_t/collection1/] result:true
[junit4:junit4] 2> 804376 T2217 C3027 P44824 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:39874/e_t/collection1/ against:[http://127.0.0.1:56367/e_t/collection1/] result:false
[junit4:junit4] 2> 804376 T2217 C3027 P44824 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:47823/e_t/collection1/ against:[http://127.0.0.1:56367/e_t/collection1/] result:false
[junit4:junit4] 2> 804381 T2265 C3026 P47823 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={distrib.from=http://127.0.0.1:44824/e_t/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:56367/e_t/collection1/&wt=javabin&version=2} {add=[2 (1438545741590036480)]} 0 2
[junit4:junit4] 2> 804381 T2249 C3030 P39874 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={distrib.from=http://127.0.0.1:44824/e_t/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:56367/e_t/collection1/&wt=javabin&version=2} {add=[2 (1438545741590036480)]} 0 2
[junit4:junit4] 2> 804381 T2217 C3027 P44824 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={distrib.from=http://127.0.0.1:39874/e_t/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:56367/e_t/collection1/&wt=javabin&version=2} {add=[2 (1438545741590036480)]} 0 6
[junit4:junit4] 2> 804382 T2250 C3030 P39874 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={test.distrib.skip.servers=http://127.0.0.1:56367/e_t/collection1/&wt=javabin&version=2} {add=[2]} 0 8
[junit4:junit4] 2> 804385 T2194 C3028 P58087 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={wt=javabin&version=2} {add=[3 (1438545741597376512)]} 0 1
[junit4:junit4] 2> 804389 T2215 C3027 P44824 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:56367/e_t/collection1/]
[junit4:junit4] 2> 804389 T2215 C3027 P44824 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:56367/e_t/collection1/ against:[http://127.0.0.1:56367/e_t/collection1/] result:true
[junit4:junit4] 2> 804390 T2215 C3027 P44824 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:39874/e_t/collection1/ against:[http://127.0.0.1:56367/e_t/collection1/] result:false
[junit4:junit4] 2> 804390 T2215 C3027 P44824 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:47823/e_t/collection1/ against:[http://127.0.0.1:56367/e_t/collection1/] result:false
[junit4:junit4] 2> 804393 T2247 C3030 P39874 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={distrib.from=http://127.0.0.1:44824/e_t/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:56367/e_t/collection1/&wt=javabin&version=2} {add=[3 (1438545741603667968)]} 0 1
[junit4:junit4] 2> 804393 T2266 C3026 P47823 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={distrib.from=http://127.0.0.1:44824/e_t/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:56367/e_t/collection1/&wt=javabin&version=2} {add=[3 (1438545741603667968)]} 0 1
[junit4:junit4] 2> 804394 T2215 C3027 P44824 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={distrib.from=http://127.0.0.1:47823/e_t/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:56367/e_t/collection1/&wt=javabin&version=2} {add=[3 (1438545741603667968)]} 0 6
[junit4:junit4] 2> 804394 T2264 C3026 P47823 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={test.distrib.skip.servers=http://127.0.0.1:56367/e_t/collection1/&test.distrib.skip.servers=http://127.0.0.1:39874/e_t/collection1/&wt=javabin&version=2} {add=[3]} 0 7
[junit4:junit4] 2> 804396 T2195 C3028 P58087 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 804415 T2195 C3028 P58087 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/control/data/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/control/data/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> 804415 T2195 C3028 P58087 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 804417 T2195 C3028 P58087 oass.SolrIndexSearcher.<init> Opening Searcher@3f7e792e main
[junit4:junit4] 2> 804418 T2195 C3028 P58087 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 804418 T2205 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3f7e792e main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 804419 T2195 C3028 P58087 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 23
[junit4:junit4] 2> 804421 T2267 C3026 P47823 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:44824/e_t/collection1/, StdNode: http://127.0.0.1:56367/e_t/collection1/, StdNode: http://127.0.0.1:39874/e_t/collection1/, StdNode: http://127.0.0.1:47823/e_t/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 804423 T2248 C3030 P39874 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 804423 T2266 C3026 P47823 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 804423 T2216 C3027 P44824 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 804423 T2233 C3029 P56367 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 804465 T2233 C3029 P56367 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty2/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 804465 T2233 C3029 P56367 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 804465 T2216 C3027 P44824 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 804466 T2216 C3027 P44824 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 804468 T2233 C3029 P56367 oass.SolrIndexSearcher.<init> Opening Searcher@50facf4e main
[junit4:junit4] 2> 804468 T2233 C3029 P56367 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 804468 T2216 C3027 P44824 oass.SolrIndexSearcher.<init> Opening Searcher@23c3754e main
[junit4:junit4] 2> 804469 T2216 C3027 P44824 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 804469 T2239 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@50facf4e main{StandardDirectoryReader(segments_4:3:nrt _0(4.4):c2)}
[junit4:junit4] 2> 804469 T2223 oasc.JmxMonitoredMap.put WARN Failed to register info bean: fieldValueCache javax.management.InstanceAlreadyExistsException: solr/collection1:type=fieldValueCache,id=org.apache.solr.search.FastLRUCache
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:302)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1846)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1723)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2>
[junit4:junit4] 2> 804470 T2233 C3029 P56367 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 47
[junit4:junit4] 2> 804470 T2223 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@23c3754e main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 804470 T2216 C3027 P44824 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 47
[junit4:junit4] 2> 804485 T2266 C3026 P47823 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty4/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 804485 T2266 C3026 P47823 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 804487 T2266 C3026 P47823 oass.SolrIndexSearcher.<init> Opening Searcher@3f89f8eb main
[junit4:junit4] 2> 804488 T2266 C3026 P47823 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 804488 T2277 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3f89f8eb main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 804489 T2266 C3026 P47823 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 66
[junit4:junit4] 2> 804505 T2248 C3030 P39874 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> 804505 T2248 C3030 P39874 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 804507 T2248 C3030 P39874 oass.SolrIndexSearcher.<init> Opening Searcher@5323f4ad main
[junit4:junit4] 2> 804508 T2248 C3030 P39874 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 804509 T2256 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5323f4ad main{StandardDirectoryReader(segments_5:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 804509 T2248 C3030 P39874 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 86
[junit4:junit4] 2> 804510 T2267 C3026 P47823 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 89
[junit4:junit4] 2> 804510 T2179 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 804511 T2179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 804511 T2179 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 804513 T2214 C3027 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 804516 T2232 C3029 P56367 oasc.SolrCore.execute [collection1] webapp=/e_t path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=2 status=0 QTime=0
[junit4:junit4] 2> 804518 T2249 C3030 P39874 oasc.SolrCore.execute [collection1] webapp=/e_t path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 804520 T2264 C3026 P47823 oasc.SolrCore.execute [collection1] webapp=/e_t path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 804521 T2179 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 804524 T2250 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2
[junit4:junit4] 2> 804524 T2250 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 804527 T2217 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 804527 T2217 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44824/e_t/collection1/
[junit4:junit4] 2> 804528 T2217 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44824/e_t START replicas=[http://127.0.0.1:56367/e_t/collection1/, http://127.0.0.1:39874/e_t/collection1/, http://127.0.0.1:47823/e_t/collection1/] nUpdates=100
[junit4:junit4] 2> 804529 T2231 C3029 P56367 oasc.SolrCore.execute [collection1] webapp=/e_t path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 804529 T2266 C3026 P47823 oasc.SolrCore.execute [collection1] webapp=/e_t path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 804530 T2217 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:44824/e_t Received 3 versions from 127.0.0.1:56367/e_t/collection1/
[junit4:junit4] 2> 804529 T2247 C3030 P39874 oasc.SolrCore.execute [collection1] webapp=/e_t path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 804530 T2217 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:44824/e_t Our versions are newer. ourLowThreshold=1438545741547044864 otherHigh=1438545741575356416
[junit4:junit4] 2> 804530 T2217 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:44824/e_t Received 5 versions from 127.0.0.1:47823/e_t/collection1/
[junit4:junit4] 2> 804531 T2217 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:44824/e_t Our versions are newer. ourLowThreshold=1438545741547044864 otherHigh=1438545741590036480
[junit4:junit4] 2> 804531 T2217 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:44824/e_t Received 5 versions from 127.0.0.1:39874/e_t/collection1/
[junit4:junit4] 2> 804531 T2217 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:44824/e_t Our versions are newer. ourLowThreshold=1438545741547044864 otherHigh=1438545741590036480
[junit4:junit4] 2> 804531 T2217 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44824/e_t DONE. sync succeeded
[junit4:junit4] 2> 804531 T2217 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 804531 T2217 oasc.SyncStrategy.syncToMe http://127.0.0.1:44824/e_t/collection1/: try and ask http://127.0.0.1:56367/e_t/collection1/ to sync
[junit4:junit4] 2> 804532 T2217 oasc.SyncStrategy.syncToMe http://127.0.0.1:44824/e_t/collection1/: try and ask http://127.0.0.1:39874/e_t/collection1/ to sync
[junit4:junit4] 2> 804532 T2217 oasc.SyncStrategy.syncToMe http://127.0.0.1:44824/e_t/collection1/: try and ask http://127.0.0.1:47823/e_t/collection1/ to sync
[junit4:junit4] 2> 804533 T2230 C3029 P56367 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56367/e_t START replicas=[http://127.0.0.1:44824/e_t/collection1/] nUpdates=100
[junit4:junit4] 2> 804533 T2248 C3030 P39874 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:39874/e_t START replicas=[http://127.0.0.1:44824/e_t/collection1/] nUpdates=100
[junit4:junit4] 2> 804534 T2267 C3026 P47823 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:47823/e_t START replicas=[http://127.0.0.1:44824/e_t/collection1/] nUpdates=100
[junit4:junit4] 2> 804535 T2216 C3027 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 804535 T2248 C3030 P39874 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:39874/e_t Received 5 versions from 127.0.0.1:44824/e_t/collection1/
[junit4:junit4] 2> 804535 T2214 C3027 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 804535 T2248 C3030 P39874 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:39874/e_t Our versions are newer. ourLowThreshold=1438545741547044864 otherHigh=1438545741590036480
[junit4:junit4] 2> 804536 T2248 C3030 P39874 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:39874/e_t DONE. sync succeeded
[junit4:junit4] 2> 804536 T2230 C3029 P56367 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:56367/e_t Received 5 versions from 127.0.0.1:44824/e_t/collection1/
[junit4:junit4] 2> 804536 T2215 C3027 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 804536 T2230 C3029 P56367 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:56367/e_t Requesting updates from 127.0.0.1:44824/e_t/collection1/n=2 versions=[1438545741603667968, 1438545741590036480]
[junit4:junit4] 2> 804537 T2267 C3026 P47823 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:47823/e_t Received 5 versions from 127.0.0.1:44824/e_t/collection1/
[junit4:junit4] 2> 804536 T2248 C3030 P39874 oasc.SolrCore.execute [collection1] webapp=/e_t path=/get params={sync=http://127.0.0.1:44824/e_t/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=3
[junit4:junit4] 2> 804537 T2267 C3026 P47823 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:47823/e_t Our versions are newer. ourLowThreshold=1438545741547044864 otherHigh=1438545741590036480
[junit4:junit4] 2> 804538 T2267 C3026 P47823 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:47823/e_t DONE. sync succeeded
[junit4:junit4] 2> 804537 T2217 oasc.SyncStrategy.syncToMe http://127.0.0.1:44824/e_t/collection1/: sync completed with http://127.0.0.1:39874/e_t/collection1/
[junit4:junit4] 2> 804538 T2267 C3026 P47823 oasc.SolrCore.execute [collection1] webapp=/e_t path=/get params={sync=http://127.0.0.1:44824/e_t/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=4
[junit4:junit4] 2> 804538 T2217 oasc.SyncStrategy.syncToMe http://127.0.0.1:44824/e_t/collection1/: sync completed with http://127.0.0.1:47823/e_t/collection1/
[junit4:junit4] 2> 804539 T2216 C3027 P44824 oasc.SolrCore.execute [collection1] webapp=/e_t path=/get params={distrib=false&getUpdates=1438545741603667968,1438545741590036480&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 804541 T2230 C3029 P56367 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1438545741590036480), 3 (1438545741603667968)]} 0 2
[junit4:junit4] 2> 804542 T2230 C3029 P56367 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56367/e_t DONE. sync succeeded
[junit4:junit4] 2> 804542 T2230 C3029 P56367 oasc.SolrCore.execute [collection1] webapp=/e_t path=/get params={sync=http://127.0.0.1:44824/e_t/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=9
[junit4:junit4] 2> 804542 T2217 oasc.SyncStrategy.syncToMe http://127.0.0.1:44824/e_t/collection1/: sync completed with http://127.0.0.1:56367/e_t/collection1/
[junit4:junit4] 2> 804542 T2217 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={shard=shard1&action=REQUESTSYNCSHARD&core=collection1&collection=collection1&wt=javabin&version=2} status=0 QTime=15
[junit4:junit4] 2> 804543 T2250 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2} status=0 QTime=19
[junit4:junit4] 2> 804543 T2179 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 804543 T2179 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 804544 T2179 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 804545 T2179 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 804547 T2196 C3028 P58087 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 804575 T2196 C3028 P58087 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/control/data/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/control/data/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> 804576 T2196 C3028 P58087 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 804577 T2196 C3028 P58087 oass.SolrIndexSearcher.<init> Opening Searcher@47ad06a6 main
[junit4:junit4] 2> 804577 T2196 C3028 P58087 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 804578 T2205 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@47ad06a6 main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 804579 T2196 C3028 P58087 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 32
[junit4:junit4] 2> 804581 T2233 C3029 P56367 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:44824/e_t/collection1/, StdNode: http://127.0.0.1:56367/e_t/collection1/, StdNode: http://127.0.0.1:39874/e_t/collection1/, StdNode: http://127.0.0.1:47823/e_t/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 804584 T2264 C3026 P47823 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 804584 T2249 C3030 P39874 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 804584 T2214 C3027 P44824 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 804586 T2231 C3029 P56367 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C3031 name=collection1 org.apache.solr.core.SolrCore@1196c7b6 url=http://127.0.0.1:39874/e_t/collection1 node=127.0.0.1:39874_e_t C3031_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:39874_e_t, base_url=http://127.0.0.1:39874/e_t}
[junit4:junit4] 2> 804605 T2249 C3031 P39874 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index,segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty3/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> 804605 T2249 C3031 P39874 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> ASYNC NEW_CORE C3032 name=collection1 org.apache.solr.core.SolrCore@7e463875 url=http://127.0.0.1:44824/e_t/collection1 node=127.0.0.1:44824_e_t C3032_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:44824_e_t, base_url=http://127.0.0.1:44824/e_t, leader=true}
[junit4:junit4] 2> 804605 T2214 C3032 P44824 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_6,generation=6}
[junit4:junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371904103236/jetty1/index,segFN=segments_7,generation=7}
[junit4:junit4] 2> 804606 T2214 C3032 P44824 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
[junit4:junit4] 2> 804606 T2249 C3031 P39874 oass.SolrIndexSearcher.<init> Opening Searcher@5f440211 main
[junit4:junit4] 2> 804607 T2249 C3031 P39874 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 804607 T2214 C3032 P44824 oass.SolrIndexSearcher.<init> Opening Searcher@1a055558 main
[junit4:junit4] 2> 804608 T2214 C3032 P44824 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 804608 T2256 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5f440211 main{StandardDirectoryReader(segments_5:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 804609 T2223 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1a055558 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 804609 T2249 C3031 P39874 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 26
[junit4:junit4] 2> 804609 T2214 C3032 P44824 oasup.LogUpdateProcessor.finish [collection1] webapp=/e_t path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 26
[junit4:junit4] 2> ASYNC NEW_CORE C3033 name=collection1 org.apache.solr.core.SolrCore@dbadbae url=http://127.0.0.1:47823/e_t/collection1 node=127.0.0.1:47823_e_t C3033_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:47823_e_t, base_url=http://127.0.0.1:47823/e_t}
[junit4:junit4] 2> 804615 T2264 C3033 P47823 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/mnt
[...truncated too long message...]
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:44824_e_t (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:58087_e_t (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:47823_e_t (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:39874_e_t (0)
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/collections (2)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89909107309871114-127.0.0.1:39874_e_t_collection1-n_0000000005 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89909107309871118-127.0.0.1:44824_e_t_collection1-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89909107309871116-127.0.0.1:47823_e_t_collection1-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:47823_e_t",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:47823/e_t"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89909107309871107-127.0.0.1:58087_e_t_collection1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:58087_e_t",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:58087/e_t"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89909107309871114-127.0.0.1:39874_e_t-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89909107309871118-127.0.0.1:44824_e_t-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89909107309871116-127.0.0.1:47823_e_t-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89909107309871107-127.0.0.1:58087_e_t-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89909107309871107-127.0.0.1:58087_e_t-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> 882001 T2254 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 882012 T2254 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 882012 T2254 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=BC174470A104C5EC -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=is_IS -Dtests.timezone=America/Halifax -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 101s J0 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:44824/e_t/collection1lastClient and got 249 from http://127.0.0.1:39874/e_t/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([BC174470A104C5EC:3DF1CA68D65BA5D0]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2> 882012 T2179 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 101137 T2178 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4] 2> 883089 T2271 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 883089 T2271 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 883089 T2271 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_l=Pulsing41(freqCutoff=6 minBlockSize=83 maxBlockSize=179), text=MockFixedIntBlock(blockSize=347), _version_=PostingsFormat(name=Direct), rnd_b=PostingsFormat(name=TestBloomFilteredLucene41Postings), intDefault=PostingsFormat(name=Direct), timestamp=Pulsing41(freqCutoff=6 minBlockSize=83 maxBlockSize=179), id=PostingsFormat(name=TestBloomFilteredLucene41Postings), a_t=Pulsing41(freqCutoff=6 minBlockSize=83 maxBlockSize=179), range_facet_sl=PostingsFormat(name=TestBloomFilteredLucene41Postings), range_facet_si=Pulsing41(freqCutoff=6 minBlockSize=83 maxBlockSize=179), other_tl1=PostingsFormat(name=Direct), multiDefault=Pulsing41(freqCutoff=6 minBlockSize=83 maxBlockSize=179), a_si=PostingsFormat(name=Direct)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=is_IS, timezone=America/Halifax
[junit4:junit4] 2> NOTE: Linux 3.2.0-48-generic amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=8,threads=1,free=305432760,total=518979584
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestReversedWildcardFilterFactory, TestRemoteStreaming, TestLFUCache, TestSolrXmlPersistence, TermVectorComponentTest, TestQuerySenderListener, TestPseudoReturnFields, TestJoin, TestClassNameShortening, AnalysisAfterCoreReloadTest, SliceStateUpdateTest, PreAnalyzedFieldTest, SuggesterTest, TestDocSet, DateFieldTest, IndexSchemaTest, MoreLikeThisHandlerTest, TestQueryUtils, TestReloadAndDeleteDocs, SpellingQueryConverterTest, FastVectorHighlighterTest, TestPerFieldSimilarity, OverseerCollectionProcessorTest, ClusterStateUpdateTest, SpatialFilterTest, TestRandomDVFaceting, PeerSyncTest, SolrIndexSplitterTest, TestDocumentBuilder, QueryEqualityTest, TestFastLRUCache, QueryElevationComponentTest, UnloadDistributedZkTest, TestFieldCollectionResource, DocumentBuilderTest, TestPropInjectDefaults, DocValuesTest, OverseerTest, BasicFunctionalityTest, BadCopyFieldTest, LegacyHTMLStripCharFilterTest, SOLR749Test, TestGroupingSearch, TestSolrDeletionPolicy2, ResourceLoaderTest, MinimalSchemaTest, TestLRUCache, TestSolrDeletionPolicy1, ExternalFileFieldSortTest, TestFunctionQuery, PrimUtilsTest, FileBasedSpellCheckerTest, SoftAutoCommitTest, DistributedTermsComponentTest, TestSerializedLuceneMatchVersion, TestSolrQueryParser, TestSweetSpotSimilarityFactory, TestSolrCoreProperties, TestFaceting, AliasIntegrationTest, TestIBSimilarityFactory, TestSolrIndexConfig, StatelessScriptUpdateProcessorFactoryTest, ShardSplitTest, TestBadConfig, PluginInfoTest, TestCloudManagedSchema, TestSchemaVersionResource, UpdateRequestProcessorFactoryTest, TestLMDirichletSimilarityFactory, ShardRoutingTest, SyncSliceTest]
[junit4:junit4] Completed on J0 in 102.48s, 1 test, 1 failure <<< FAILURES!
[...truncated 443 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:392: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:372: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1246: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:890: There were test failures: 298 suites, 1256 tests, 1 failure, 13 ignored (7 assumptions)
Total time: 37 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_25 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux (32bit/ibm-j9-jdk6) - Build # 6182
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6182/
Java: 32bit/ibm-j9-jdk6 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
3 tests failed.
REGRESSION: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
Server at http://127.0.0.1:57565/o_/onenodecollectioncore returned non ok status:404, message:Can not find: /o_/onenodecollectioncore/update
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:57565/o_/onenodecollectioncore returned non ok status:404, message:Can not find: /o_/onenodecollectioncore/update
at __randomizedtesting.SeedInfo.seed([6AD2A1B1ADE97BA9:EB342FA9DAB61B95]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:198)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:90)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:738)
REGRESSION: org.apache.solr.cloud.ShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:43941/ikgk/sv returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:43941/ikgk/sv returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([6AD2A1B1ADE97BA9:EB342FA9DAB61B95]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:738)
REGRESSION: org.apache.solr.core.TestJmxIntegration.testJmxRegistration
Error Message:
No SolrDynamicMBeans found
Stack Trace:
java.lang.AssertionError: No SolrDynamicMBeans found
at __randomizedtesting.SeedInfo.seed([6AD2A1B1ADE97BA9:E403C58BC0A823CC]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.core.TestJmxIntegration.testJmxRegistration(TestJmxIntegration.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:738)
Build Log:
[...truncated 8591 lines...]
[junit4:junit4] Suite: org.apache.solr.core.TestJmxIntegration
[junit4:junit4] 2> log4j:WARN No such property [conversionPattern] in org.apache.solr.util.SolrLogLayout.
[junit4:junit4] 2> 332 T18 oas.SolrTestCaseJ4.initCore ####initCore
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-TestJmxIntegration-1371897914400
[junit4:junit4] 2> 371 T18 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4] 2> 376 T18 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4] 2> 378 T18 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 895 T18 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 1013 T18 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1018 T18 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1065 T18 oass.IndexSchema.readSchema [null] Schema name=test
[junit4:junit4] 2> 2007 T18 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 2009 T18 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 2023 T18 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2046 T18 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2050 T18 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2161 T18 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2168 T18 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2178 T18 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2183 T18 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2185 T18 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2186 T18 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2188 T18 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2189 T18 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2209 T18 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2210 T18 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr
[junit4:junit4] 2> 2215 T18 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/'
[junit4:junit4] 2> 2238 T18 oasc.CoreContainer.<init> New CoreContainer 982465167
[junit4:junit4] 2> 2326 T18 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2331 T18 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-TestJmxIntegration-1371897914400/
[junit4:junit4] 2> 2333 T18 oasc.JmxMonitoredMap.<init> No JMX servers found, not exposing Solr information with JMX.
[junit4:junit4] 2> 2336 T18 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2353 T18 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-TestJmxIntegration-1371897914400
[junit4:junit4] 2> 2354 T18 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-TestJmxIntegration-1371897914400/index/
[junit4:junit4] 2> 2355 T18 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-TestJmxIntegration-1371897914400/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2359 T18 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-TestJmxIntegration-1371897914400/index
[junit4:junit4] 2> 2426 T18 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@6080608 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4dd24dd2)),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2428 T18 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2433 T18 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4] 2> 2526 T18 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2527 T18 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4] 2> 2528 T18 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2530 T18 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4] 2> 2546 T18 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2548 T18 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2549 T18 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2563 T18 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2566 T18 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2572 T18 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4] 2> 2574 T18 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4] 2> 2576 T18 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2577 T18 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4] 2> 2578 T18 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4] 2> 2579 T18 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4] 2> 2583 T18 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2584 T18 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4] 2> 2585 T18 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4] 2> 2586 T18 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4] 2> 2587 T18 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4] 2> 2588 T18 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4] 2> 2588 T18 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4] 2> 2589 T18 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4] 2> 2590 T18 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4] 2> 2594 T18 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4] 2> 2596 T18 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4] 2> 2616 T18 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2621 T18 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4] 2> 2628 T18 oasc.SolrCore.initDeprecatedSupport WARN adding ShowFileRequestHandler with hidden files: [SOLRCONFIG-MANAGED-SCHEMA.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SCHEMA-REQUIRED-FIELDS.XML, BAD-SOLRCONFIG-MANAGED-SCHEMA-NAMED-SCHEMA.XML.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, SCHEMA-BEHAVIOR.XML, SOLRCONFIG-MERGEPOLICY-DEFAULTS.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, STOP-2.TXT, SCHEMA11.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, SCHEMA15.XML, BAD-SOLRCONFIG-UNEXPECTED-SCHEMA-ATTRIBUTE.XML, SOLRCONFIG-NATIVELOCK.XML, SCHEMA-SIM.XML, CONDITIONAL.UPDATEPROCESSOR.JS, SCHEMA-DOCVALUES.XML, SCHEMA-ID-AND-VERSION-FIELDS-ONLY.XML, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SOLRCONFIG-REPEATER.XML, BAD-SCHEMA-DOCVALUES-NOT-REQUIRED-NO-DEFAULT.XML, SOLRCONFIG-SPELLCHECKER.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-EFF.XML, SCHEMA-CHARFILTERS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, SOLRCONFIG_CODEC.XML, SOLRCONFIG-MUTABLE-MANAGED-SCHEMA.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML, HYPHENATION.DTD, TRIVIAL.UPDATEPROCESSOR1.JS, SOLRCONFIG-MERGEPOLICY.XML, SOLRCONFIG-PHRASESUGGEST.XML, SOLRCONFIG-SLAVE1.XML, SOLRCONFIG-BASIC.XML, SCHEMA-LMJELINEKMERCER.XML, BAD-MP-SOLRCONFIG.XML, JASUGGEST.TXT, BAD-SCHEMA-DUP-FIELD.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SOLRCONFIG-MASTER.XML, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-TFIDF.XML, SCHEMA-REPLICATION1.XML, STOPWITHBOM.TXT, SOLRCONFIG-ELEVATE.XML, SYNONYMS.TXT, SOLRCONFIG-QUERYSENDER.XML, SOLRCONFIG-COMPONENTS-NAME.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, WDFTYPES.TXT, BAD-SCHEMA-CURRENCY-FT-OER-NORATES.XML, SOLRCONFIG-DELPOLICY1.XML, BAD_SOLRCONFIG.XML, BAD-SCHEMA-UNSUPPORTED-DOCVALUES.XML, HUNSPELL-TEST.DIC, SCHEMA-NUMERIC.XML, SCHEMA-LUCENEMATCHVERSION.XML, KEEP-2.TXT, SCHEMA-STOP-KEEP.XML, SCHEMA-IB.XML, STOP-SNOWBALL.TXT, SCHEMA-ONE-FIELD-NO-DYNAMIC-FIELD.XML, BAD-SCHEMA-MULTIPLE-ASTERISK-COPYFIELD-DEST-SHOULD-FAIL-TEST.XML, SCHEMA-DOCVALUESMULTI.XML, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG-SLAVE.XML, SCHEMA-CLASS-NAME-SHORTENING-ON-SERIALIZATION.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, STOPWORDSWRONGENCODING.TXT, BAD-SCHEMA-MISPLACED-ASTERISK-COPYFIELD-SOURCE-SHOULD-FAIL-TEST.XML, THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, STOPTYPES-1.TXT, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, SOLRCONFIG.SNIPPET.RANDOMINDEXCONFIG.XML, REGEX-BOOST-PROCESSOR-TEST.TXT, SOLRCONFIG-MINIMAL.XML, SCHEMA-DOCVALUESFACETING.XML, SOLRCONFIG-TLOG.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-POSTINGSHIGHLIGHT.XML, DA_COMPOUNDDICTIONARY.TXT, SOLRCONFIG-TERMINDEX.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, BAD-SCHEMA-NON-GLOB-COPYFIELD-SOURCE-MATCHING-NOTHING-SHOULD-FAIL-TEST.XML, SCHEMASURROUND.XML, OPEN-EXCHANGE-RATES.JSON, BAD-SCHEMA-DUP-FIELDTYPE.XML, PHRASESUGGEST.TXT, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-MASTER2.XML, OLD_SYNONYMS.TXT, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SCHEMA-LMDIRICHLET.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, SCHEMA-REVERSED.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, SCHEMA-FOLDING.XML, SOLRCONFIG-CACHING.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD-CURRENCY.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SCHEMA-BM25.XML, SOLRCONFIG-ENABLEPLUGIN.XML, STOP-1.TXT, BAD-SCHEMA-SWEETSPOT-BOTH-TF.XML, SCHEMA-REST.XML, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, SCHEMA12.XML, SOLRCONFIG-TRANSFORMERS.XML, SCHEMA-COPYFIELD-TEST.XML, SOLRCONFIG-BASIC-LUCENEVERSION31.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, BAD-SCHEMA-SWEETSPOT-PARTIAL-BASELINE.XML, CURRENCY.XML, DA_UTF8.XML, FUZZYSUGGEST.TXT, SCHEMA-POSTINGSHIGHLIGHT.XML, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SOLRCONFIG-SOLR-749.XML, STOPWORDS.TXT, BAD-SCHEMA-BOGUS-ANALYSIS-PARAMETERS.XML, HUNSPELL-TEST.AFF, SOLRCONFIG-TLOG-MANAGED-SCHEMA.XML, SOLRCONFIG_PERF.XML, SCHEMA-SWEETSPOT.XML, SOLRCONFIG-SIMPLELOCK.XML, BAD-SCHEMA-MISPLACED-ASTERISK-COPYFIELD-DEST-SHOULD-FAIL-TEST.XML, SCHEMA-TINY.XML, BAD-SCHEMA-SWEETSPOT-PARTIAL-NORMS.XML, FRENCHARTICLES.TXT, SOLRCONFIG-LAZYWRITER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SCHEMA-REPLICATION2.XML, SCHEMA-SNIPPET-TYPE.XML, SCHEMA-SPATIAL.XML, BAD-SCHEMA-SWEETSPOT-PARTIAL-HYPERBOLIC.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-XINCLUDE.XML, XSLT, SOLRCONFIG-WARMER.XML, SOLRCONFIG-DELPOLICY2.XML, KEEP-1.TXT, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA-TRIE.XML, SOLRCONFIG-TLOG-MUTABLE-MANAGED-SCHEMA.XML, SCHEMA-BINARYFIELD.XML, SCHEMA-REST-LUCENE-MATCH-VERSION.XML, SCHEMA-COLLATEFILTER.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, BAD-SCHEMA-CURRENCY-FT-BOGUS-CODE-IN-XML.XML, SCHEMA-MINIMAL.XML, STOPTYPES-2.TXT, SCHEMA_CODEC.XML, STEMDICT.TXT, SCHEMA-SYNONYM-TOKENIZER.XML, ELEVATE.XML, SCHEMA-COLLATE.XML, PROTWORDS.TXT, SOLRCONFIG-DEFAULTS.XML, SCHEMA-XINCLUDE.XML, SCHEMA-SNIPPET-FIELD.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMA-PHRASESUGGEST.XML, SCHEMA-DFR.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, SOLRCONFIG-MASTER1.XML, SOLRCONFIG-FUNCTIONQUERY.XML, BAD-SOLRCONFIG-SCHEMA-MUTABLE-BUT-NOT-MANAGED.XML, SOLRCONFIG-MASTER3.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, SOLRCONFIG-ALTDIRECTORY.XML, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, BAD-SCHEMA-MULTIPLE-ASTERISK-COPYFIELD-SOURCE-SHOULD-FAIL-TEST.XML, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, BAD-SCHEMA-CURRENCY-FT-BOGUS-DEFAULT-CODE.XML, SOLRCONFIG-MERGEPOLICY-LEGACY.XML, BAD-SCHEMA-BOGUS-FIELD-PARAMETERS.XML]
[junit4:junit4] 2> 2646 T18 oass.SolrIndexSearcher.<init> Opening Searcher@26492649 main
[junit4:junit4] 2> 2657 T18 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-TestJmxIntegration-1371897914400/tlog
[junit4:junit4] 2> 2665 T18 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2666 T18 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2666 T18 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4] 2> 2703 T18 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4] 2> 2809 T19 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@26492649 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2809 T18 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2812 T18 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4:junit4] 2> 2842 T18 oas.SolrTestCaseJ4.setUp ###Starting testJmxRegistration
[junit4:junit4] 2> 2843 T18 oasc.TestJmxIntegration.testJmxRegistration Servers in testJmxRegistration: [com.sun.jmx.mbeanserver.JmxMBeanServer@1200120]
[junit4:junit4] 2> 2848 T18 oas.SolrTestCaseJ4.tearDown ###Ending testJmxRegistration
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestJmxIntegration -Dtests.method=testJmxRegistration -Dtests.seed=6AD2A1B1ADE97BA9 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=da_DK_EURO -Dtests.timezone=ROK -Dtests.file.encoding=US-ASCII
[junit4:junit4] FAILURE 0.23s J1 | TestJmxIntegration.testJmxRegistration <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: No SolrDynamicMBeans found
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([6AD2A1B1ADE97BA9:E403C58BC0A823CC]:0)
[junit4:junit4] > at org.apache.solr.core.TestJmxIntegration.testJmxRegistration(TestJmxIntegration.java:94)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:738)
[junit4:junit4] 2> 3054 T18 oas.SolrTestCaseJ4.setUp ###Starting testJmxUpdate
[junit4:junit4] 2> 3057 T18 oas.SolrTestCaseJ4.tearDown ###Ending testJmxUpdate
[junit4:junit4] IGNOR/A 0.04s J1 | TestJmxIntegration.testJmxUpdate
[junit4:junit4] > Assumption #1: got: <false>, expected: is <true>
[junit4:junit4] 2> 3095 T18 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 3096 T18 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=982465167
[junit4:junit4] 2> 3097 T18 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@27972797
[junit4:junit4] 2> 3099 T18 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 3101 T18 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 3102 T18 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 3103 T18 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 3107 T18 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 3109 T18 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 3111 T18 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-TestJmxIntegration-1371897914400/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-TestJmxIntegration-1371897914400/index;done=false>>]
[junit4:junit4] 2> 3112 T18 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-TestJmxIntegration-1371897914400/index
[junit4:junit4] 2> 3114 T18 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-TestJmxIntegration-1371897914400 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-TestJmxIntegration-1371897914400;done=false>>]
[junit4:junit4] 2> 3115 T18 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-TestJmxIntegration-1371897914400
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=da_DK_EURO, timezone=ROK
[junit4:junit4] 2> NOTE: Linux 3.2.0-48-generic x86/IBM Corporation 1.6.0 (32-bit)/cpus=8,threads=2,free=920192,total=19858944
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestJmxIntegration]
[junit4:junit4] Completed on J1 in 3.26s, 3 tests, 1 failure, 2 skipped <<< FAILURES!
[...truncated 263 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4] 2> 316942 T1168 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /o_/
[junit4:junit4] 2> 316945 T1168 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1371898231009
[junit4:junit4] 2> 316947 T1168 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 316947 T1169 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 317047 T1168 oasc.ZkTestServer.run start zk server on port:39763
[junit4:junit4] 2> 317049 T1168 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 317177 T1175 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7dde7dde name:ZooKeeperConnection Watcher:127.0.0.1:39763 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 317178 T1168 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 317179 T1168 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 317184 T1168 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 317186 T1177 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5f4f5f4f name:ZooKeeperConnection Watcher:127.0.0.1:39763/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 317187 T1168 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 317188 T1168 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 317207 T1168 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 317213 T1168 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 317217 T1168 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 317223 T1168 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 317224 T1168 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 317230 T1168 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 317231 T1168 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 317235 T1168 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 317236 T1168 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 317240 T1168 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 317241 T1168 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 317245 T1168 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 317246 T1168 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 317251 T1168 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 317251 T1168 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 317256 T1168 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 317256 T1168 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 317261 T1168 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 317262 T1168 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 317265 T1168 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 317266 T1168 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 317463 T1168 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 317466 T1168 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:33322
[junit4:junit4] 2> 317467 T1168 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 317467 T1168 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 317468 T1168 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1371898231334
[junit4:junit4] 2> 317469 T1168 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1371898231334/solr.xml
[junit4:junit4] 2> 317470 T1168 oasc.CoreContainer.<init> New CoreContainer 1552112771
[junit4:junit4] 2> 317470 T1168 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1371898231334/'
[junit4:junit4] 2> 317471 T1168 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1371898231334/'
[junit4:junit4] 2> 317518 T1168 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 317518 T1168 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 317519 T1168 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 317520 T1168 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 317521 T1168 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 317521 T1168 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 317522 T1168 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 317523 T1168 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 317524 T1168 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 317524 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 317531 T1168 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 317532 T1168 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39763/solr
[junit4:junit4] 2> 317533 T1168 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 317535 T1168 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 317537 T1188 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a024a02 name:ZooKeeperConnection Watcher:127.0.0.1:39763 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 317538 T1168 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 317541 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 317548 T1168 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 317550 T1190 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e8f7e8f name:ZooKeeperConnection Watcher:127.0.0.1:39763/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 317550 T1168 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 317553 T1168 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 317559 T1168 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 317564 T1168 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 317568 T1168 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33322_o_
[junit4:junit4] 2> 317569 T1168 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33322_o_
[junit4:junit4] 2> 317575 T1168 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 317583 T1168 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 317596 T1168 oasc.Overseer.start Overseer (id=89908722467733507-127.0.0.1:33322_o_-n_0000000000) starting
[junit4:junit4] 2> 317601 T1168 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 317608 T1192 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 317609 T1168 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 317614 T1168 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 317617 T1168 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 317623 T1191 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 317625 T1193 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 317625 T1193 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 319129 T1191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 319131 T1191 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:33322/o_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:33322_o_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 319131 T1191 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 319132 T1191 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 319140 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 319628 T1193 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1371898231334/collection1
[junit4:junit4] 2> 319628 T1193 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 319630 T1193 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 319631 T1193 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 319634 T1193 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1371898231334/collection1/'
[junit4:junit4] 2> 319636 T1193 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1371898231334/collection1/lib/README' to classloader
[junit4:junit4] 2> 319637 T1193 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1371898231334/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 319669 T1193 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 319711 T1193 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 319714 T1193 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 319719 T1193 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 320092 T1193 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 320093 T1193 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 320094 T1193 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 320099 T1193 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 320102 T1193 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 320121 T1193 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 320126 T1193 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 320131 T1193 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 320133 T1193 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 320133 T1193 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 320134 T1193 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 320136 T1193 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 320137 T1193 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 320137 T1193 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 320138 T1193 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1371898231334/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/control/data/
[junit4:junit4] 2> 320138 T1193 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1200120
[junit4:junit4] 2> 320139 T1193 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 320140 T1193 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/control/data
[junit4:junit4] 2> 320141 T1193 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/control/data/index/
[junit4:junit4] 2> 320141 T1193 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 320142 T1193 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/control/data/index
[junit4:junit4] 2> 320144 T1193 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@38323832 lockFactory=org.apache.lucene.store.NativeFSLockFactory@22e322e3),segFN=segments_1,generation=1}
[junit4:junit4] 2> 320145 T1193 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 320147 T1193 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 320148 T1193 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 320149 T1193 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 320150 T1193 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 320150 T1193 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 320151 T1193 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 320152 T1193 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 320152 T1193 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 320153 T1193 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 320155 T1193 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 320157 T1193 oass.SolrIndexSearcher.<init> Opening Searcher@2a9f2a9f main
[junit4:junit4] 2> 320157 T1193 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/control/data/tlog
[junit4:junit4] 2> 320158 T1193 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 320159 T1193 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 320165 T1194 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2a9f2a9f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 320170 T1193 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 320171 T1193 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:33322/o_ collection:control_collection shard:shard1
[junit4:junit4] 2> 320173 T1193 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 320193 T1193 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 320199 T1193 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 320199 T1193 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 320200 T1193 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:33322/o_/collection1/
[junit4:junit4] 2> 320200 T1193 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 320201 T1193 oasc.SyncStrategy.syncToMe http://127.0.0.1:33322/o_/collection1/ has no replicas
[junit4:junit4] 2> 320201 T1193 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:33322/o_/collection1/
[junit4:junit4] 2> 320202 T1193 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 320647 T1191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 320661 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 320717 T1193 oasc.ZkController.register We are http://127.0.0.1:33322/o_/collection1/ and leader is http://127.0.0.1:33322/o_/collection1/
[junit4:junit4] 2> 320717 T1193 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33322/o_
[junit4:junit4] 2> 320718 T1193 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 320719 T1193 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 320719 T1193 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 320722 T1193 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 320724 T1168 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 320725 T1168 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 320727 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 320736 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 320738 T1168 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 320740 T1197 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11761176 name:ZooKeeperConnection Watcher:127.0.0.1:39763/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 320740 T1168 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 320742 T1168 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 320747 T1168 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 320908 T1168 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 320910 T1168 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52718
[junit4:junit4] 2> 320911 T1168 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 320912 T1168 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 320912 T1168 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1371898234811
[junit4:junit4] 2> 320913 T1168 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1371898234811/solr.xml
[junit4:junit4] 2> 320914 T1168 oasc.CoreContainer.<init> New CoreContainer 1057898254
[junit4:junit4] 2> 320915 T1168 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1371898234811/'
[junit4:junit4] 2> 320915 T1168 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1371898234811/'
[junit4:junit4] 2> 320970 T1168 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 320971 T1168 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 320972 T1168 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 320973 T1168 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 320973 T1168 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 320974 T1168 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 320975 T1168 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 320976 T1168 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 320976 T1168 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 320977 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 320984 T1168 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 320985 T1168 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39763/solr
[junit4:junit4] 2> 320986 T1168 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 320987 T1168 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 320989 T1208 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@600d600d name:ZooKeeperConnection Watcher:127.0.0.1:39763 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 320990 T1168 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 320993 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 321000 T1168 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 321001 T1210 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a014a01 name:ZooKeeperConnection Watcher:127.0.0.1:39763/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 321002 T1168 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 321009 T1168 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 322014 T1168 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52718_o_
[junit4:junit4] 2> 322016 T1168 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52718_o_
[junit4:junit4] 2> 322021 T1197 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 322021 T1210 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 322022 T1190 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 322023 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 322029 T1211 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 322030 T1211 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 322171 T1191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 322172 T1191 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:33322/o_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:33322_o_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 322178 T1191 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52718/o_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:52718_o_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 322179 T1191 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 322179 T1191 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 322194 T1197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 322194 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 322194 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 323032 T1211 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1371898234811/collection1
[junit4:junit4] 2> 323033 T1211 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 323034 T1211 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 323035 T1211 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 323037 T1211 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1371898234811/collection1/'
[junit4:junit4] 2> 323038 T1211 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1371898234811/collection1/lib/README' to classloader
[junit4:junit4] 2> 323039 T1211 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1371898234811/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 323069 T1211 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 323103 T1211 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 323106 T1211 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 323111 T1211 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 323496 T1211 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 323497 T1211 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 323498 T1211 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 323503 T1211 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 323506 T1211 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 323532 T1211 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 323539 T1211 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 323544 T1211 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 323547 T1211 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 323547 T1211 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 323548 T1211 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 323551 T1211 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 323561 T1211 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 323561 T1211 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 323562 T1211 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1371898234811/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty1/
[junit4:junit4] 2> 323563 T1211 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1200120
[junit4:junit4] 2> 323563 T1211 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 323564 T1211 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty1
[junit4:junit4] 2> 323565 T1211 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty1/index/
[junit4:junit4] 2> 323565 T1211 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 323566 T1211 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty1/index
[junit4:junit4] 2> 323568 T1211 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3e9c3e9c lockFactory=org.apache.lucene.store.NativeFSLockFactory@66186618),segFN=segments_1,generation=1}
[junit4:junit4] 2> 323569 T1211 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 323572 T1211 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 323572 T1211 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 323573 T1211 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 323574 T1211 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 323575 T1211 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 323575 T1211 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 323576 T1211 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 323577 T1211 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 323578 T1211 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 323579 T1211 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 323582 T1211 oass.SolrIndexSearcher.<init> Opening Searcher@e000e main
[junit4:junit4] 2> 323582 T1211 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty1/tlog
[junit4:junit4] 2> 323583 T1211 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 323584 T1211 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 323592 T1212 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e000e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 323595 T1211 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 323596 T1211 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52718/o_ collection:collection1 shard:shard1
[junit4:junit4] 2> 323597 T1211 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 323610 T1211 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 323614 T1211 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 323614 T1211 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 323615 T1211 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52718/o_/collection1/
[junit4:junit4] 2> 323615 T1211 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 323616 T1211 oasc.SyncStrategy.syncToMe http://127.0.0.1:52718/o_/collection1/ has no replicas
[junit4:junit4] 2> 323616 T1211 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52718/o_/collection1/
[junit4:junit4] 2> 323617 T1211 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 323704 T1191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 323715 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 323715 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 323718 T1197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 323730 T1211 oasc.ZkController.register We are http://127.0.0.1:52718/o_/collection1/ and leader is http://127.0.0.1:52718/o_/collection1/
[junit4:junit4] 2> 323731 T1211 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52718/o_
[junit4:junit4] 2> 323731 T1211 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 323732 T1211 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 323733 T1211 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 323735 T1211 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 323738 T1168 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 323738 T1168 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 323739 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 323903 T1168 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 323908 T1168 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:57565
[junit4:junit4] 2> 323910 T1168 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 323911 T1168 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 323912 T1168 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1371898237808
[junit4:junit4] 2> 323913 T1168 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1371898237808/solr.xml
[junit4:junit4] 2> 323913 T1168 oasc.CoreContainer.<init> New CoreContainer 1497585987
[junit4:junit4] 2> 323914 T1168 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1371898237808/'
[junit4:junit4] 2> 323915 T1168 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1371898237808/'
[junit4:junit4] 2> 323972 T1168 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 323973 T1168 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 323974 T1168 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 323975 T1168 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 323975 T1168 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 323976 T1168 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 323977 T1168 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 323978 T1168 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 323978 T1168 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 323979 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 323986 T1168 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 323988 T1168 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39763/solr
[junit4:junit4] 2> 323988 T1168 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 323990 T1168 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 323992 T1224 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@465c465c name:ZooKeeperConnection Watcher:127.0.0.1:39763 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 323992 T1168 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 323995 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 324003 T1168 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 324005 T1226 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5aec5aec name:ZooKeeperConnection Watcher:127.0.0.1:39763/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 324005 T1168 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 324013 T1168 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 325019 T1168 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57565_o_
[junit4:junit4] 2> 325021 T1168 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57565_o_
[junit4:junit4] 2> 325025 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 325025 T1197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 325027 T1190 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 325028 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 325029 T1226 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 325030 T1197 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 325031 T1210 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 325036 T1227 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 325037 T1227 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 325235 T1191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 325236 T1191 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52718/o_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:52718_o_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 325266 T1191 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57565/o_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "node_name":"127.0.0.1:57565_o_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 325267 T1191 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 325267 T1191 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 325273 T1197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 325273 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 325274 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 325273 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 326039 T1227 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1371898237808/collection1
[junit4:junit4] 2> 326039 T1227 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 326041 T1227 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 326041 T1227 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 326043 T1227 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1371898237808/collection1/'
[junit4:junit4] 2> 326045 T1227 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1371898237808/collection1/lib/README' to classloader
[junit4:junit4] 2> 326045 T1227 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1371898237808/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 326076 T1227 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 326118 T1227 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 326120 T1227 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 326125 T1227 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 326529 T1227 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 326529 T1227 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 326530 T1227 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 326535 T1227 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 326538 T1227 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 326568 T1227 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 326574 T1227 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 326579 T1227 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 326581 T1227 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 326582 T1227 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 326582 T1227 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 326584 T1227 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 326585 T1227 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 326585 T1227 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 326586 T1227 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1371898237808/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty2/
[junit4:junit4] 2> 326587 T1227 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1200120
[junit4:junit4] 2> 326587 T1227 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 326588 T1227 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty2
[junit4:junit4] 2> 326589 T1227 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty2/index/
[junit4:junit4] 2> 326589 T1227 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 326590 T1227 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty2/index
[junit4:junit4] 2> 326592 T1227 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261c261c lockFactory=org.apache.lucene.store.NativeFSLockFactory@19d819d8),segFN=segments_1,generation=1}
[junit4:junit4] 2> 326593 T1227 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 326596 T1227 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 326596 T1227 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 326597 T1227 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 326598 T1227 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 326599 T1227 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 326599 T1227 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 326600 T1227 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 326601 T1227 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 326601 T1227 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 326603 T1227 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 326605 T1227 oass.SolrIndexSearcher.<init> Opening Searcher@69f569f5 main
[junit4:junit4] 2> 326606 T1227 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty2/tlog
[junit4:junit4] 2> 326607 T1227 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 326608 T1227 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 326616 T1228 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@69f569f5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 326621 T1227 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 326622 T1227 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57565/o_ collection:collection1 shard:shard2
[junit4:junit4] 2> 326623 T1227 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 327234 T1227 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 327239 T1227 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 327239 T1227 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 327240 T1227 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57565/o_/collection1/
[junit4:junit4] 2> 327241 T1227 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 327242 T1227 oasc.SyncStrategy.syncToMe http://127.0.0.1:57565/o_/collection1/ has no replicas
[junit4:junit4] 2> 327243 T1227 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57565/o_/collection1/
[junit4:junit4] 2> 327243 T1227 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 328736 T1191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 328748 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 328748 T1197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 328749 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 328748 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 328756 T1227 oasc.ZkController.register We are http://127.0.0.1:57565/o_/collection1/ and leader is http://127.0.0.1:57565/o_/collection1/
[junit4:junit4] 2> 328757 T1227 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57565/o_
[junit4:junit4] 2> 328757 T1227 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 328758 T1227 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 328758 T1227 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 328761 T1227 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 328763 T1168 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 328764 T1168 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 328765 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 328952 T1168 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 328954 T1168 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50938
[junit4:junit4] 2> 328955 T1168 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 328956 T1168 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 328957 T1168 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1371898242835
[junit4:junit4] 2> 328957 T1168 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1371898242835/solr.xml
[junit4:junit4] 2> 328958 T1168 oasc.CoreContainer.<init> New CoreContainer 1715562049
[junit4:junit4] 2> 328959 T1168 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1371898242835/'
[junit4:junit4] 2> 328960 T1168 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1371898242835/'
[junit4:junit4] 2> 329045 T1168 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 329046 T1168 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 329048 T1168 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 329049 T1168 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 329050 T1168 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 329051 T1168 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 329052 T1168 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 329053 T1168 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 329054 T1168 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 329055 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 329062 T1168 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 329064 T1168 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39763/solr
[junit4:junit4] 2> 329065 T1168 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 329066 T1168 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 329069 T1240 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21a621a6 name:ZooKeeperConnection Watcher:127.0.0.1:39763 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 329069 T1168 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 329073 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 329080 T1168 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 329082 T1242 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e2b2e2b name:ZooKeeperConnection Watcher:127.0.0.1:39763/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 329082 T1168 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 329092 T1168 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 330100 T1168 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50938_o_
[junit4:junit4] 2> 330102 T1168 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50938_o_
[junit4:junit4] 2> 330237 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 330237 T1197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 330237 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 330239 T1190 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 330256 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 330257 T1197 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 330259 T1242 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 330261 T1210 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 330262 T1226 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 330266 T1191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 330268 T1191 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57565/o_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "node_name":"127.0.0.1:57565_o_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 330270 T1243 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 330270 T1243 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 330279 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 330279 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 330281 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 330282 T1197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 330286 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 331788 T1191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 331789 T1191 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50938/o_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:50938_o_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 331790 T1191 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 331790 T1191 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 331907 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 331907 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 331907 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 331907 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 331907 T1197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 332279 T1243 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1371898242835/collection1
[junit4:junit4] 2> 332279 T1243 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 332280 T1243 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 332281 T1243 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 332283 T1243 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1371898242835/collection1/'
[junit4:junit4] 2> 332285 T1243 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1371898242835/collection1/lib/README' to classloader
[junit4:junit4] 2> 332286 T1243 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1371898242835/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 332330 T1243 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 332386 T1243 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 332389 T1243 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 332394 T1243 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 332832 T1243 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 332833 T1243 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 332834 T1243 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 332839 T1243 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 332842 T1243 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 332865 T1243 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 332870 T1243 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 332875 T1243 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 332878 T1243 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 332879 T1243 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 332879 T1243 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 332882 T1243 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 332882 T1243 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 332883 T1243 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 332884 T1243 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1371898242835/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty3/
[junit4:junit4] 2> 332884 T1243 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1200120
[junit4:junit4] 2> 332885 T1243 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 332886 T1243 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty3
[junit4:junit4] 2> 332887 T1243 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty3/index/
[junit4:junit4] 2> 332887 T1243 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 332888 T1243 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty3/index
[junit4:junit4] 2> 332890 T1243 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3b403b4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8a408a4),segFN=segments_1,generation=1}
[junit4:junit4] 2> 332891 T1243 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 332894 T1243 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 332895 T1243 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 332895 T1243 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 332896 T1243 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 332897 T1243 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 332898 T1243 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 332899 T1243 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 332909 T1243 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 332910 T1243 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 332912 T1243 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 332914 T1243 oass.SolrIndexSearcher.<init> Opening Searcher@40a740a7 main
[junit4:junit4] 2> 332915 T1243 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty3/tlog
[junit4:junit4] 2> 332916 T1243 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 332916 T1243 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 332923 T1244 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@40a740a7 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 332927 T1243 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 332927 T1243 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50938/o_ collection:collection1 shard:shard1
[junit4:junit4] 2> 332932 T1243 oasc.ZkController.register We are http://127.0.0.1:50938/o_/collection1/ and leader is http://127.0.0.1:52718/o_/collection1/
[junit4:junit4] 2> 332933 T1243 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50938/o_
[junit4:junit4] 2> 332933 T1243 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 332934 T1243 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C421 name=collection1 org.apache.solr.core.SolrCore@345d345d url=http://127.0.0.1:50938/o_/collection1 node=127.0.0.1:50938_o_ C421_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:50938/o_, state=down, node_name=127.0.0.1:50938_o_, collection=collection1, core=collection1}
[junit4:junit4] 2> 332935 T1245 C421 P50938 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 332935 T1245 C421 P50938 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 332935 T1243 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 332936 T1245 C421 P50938 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 332937 T1245 C421 P50938 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 332938 T1168 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 332939 T1245 C421 P50938 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 332939 T1168 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 332941 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 332950 T1204 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 333126 T1168 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 333128 T1168 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59604
[junit4:junit4] 2> 333129 T1168 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 333130 T1168 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 333131 T1168 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1371898247013
[junit4:junit4] 2> 333132 T1168 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1371898247013/solr.xml
[junit4:junit4] 2> 333132 T1168 oasc.CoreContainer.<init> New CoreContainer 1527536396
[junit4:junit4] 2> 333133 T1168 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1371898247013/'
[junit4:junit4] 2> 333134 T1168 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1371898247013/'
[junit4:junit4] 2> 333204 T1168 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 333205 T1168 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 333206 T1168 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 333207 T1168 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 333207 T1168 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 333208 T1168 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 333209 T1168 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 333210 T1168 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 333210 T1168 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 333211 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 333218 T1168 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 333219 T1168 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:39763/solr
[junit4:junit4] 2> 333220 T1168 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 333222 T1168 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 333224 T1257 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@73667366 name:ZooKeeperConnection Watcher:127.0.0.1:39763 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 333225 T1168 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 333228 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 333235 T1168 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 333238 T1259 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@196a196a name:ZooKeeperConnection Watcher:127.0.0.1:39763/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 333243 T1168 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 333251 T1168 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 333428 T1191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 333429 T1191 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50938/o_",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:50938_o_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 333435 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 333435 T1197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 333446 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 333447 T1259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 333447 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 333451 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 333951 T1204 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 333952 T1204 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={state=recovering&onlyIfLeader=true&checkLive=true&action=PREPRECOVERY&version=2&coreNodeName=4&wt=javabin&nodeName=127.0.0.1:50938_o_&core=collection1} status=0 QTime=1002
[junit4:junit4] 2> 334257 T1168 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59604_o_
[junit4:junit4] 2> 334259 T1168 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59604_o_
[junit4:junit4] 2> 334263 T1259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 334263 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 334264 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 334264 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 334264 T1190 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 334266 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 334263 T1197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 334267 T1226 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 334269 T1242 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 334269 T1210 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 334272 T1197 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 334275 T1259 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 334281 T1260 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 334282 T1260 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 334957 T1191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 334958 T1191 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:59604/o_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "node_name":"127.0.0.1:59604_o_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 334959 T1191 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 334959 T1191 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 334980 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 334981 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 334981 T1197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 334980 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 334981 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 334981 T1259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 335285 T1260 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1371898247013/collection1
[junit4:junit4] 2> 335286 T1260 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 335287 T1260 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 335288 T1260 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 335291 T1260 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1371898247013/collection1/'
[junit4:junit4] 2> 335292 T1260 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1371898247013/collection1/lib/README' to classloader
[junit4:junit4] 2> 335293 T1260 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1371898247013/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 335336 T1260 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 335384 T1260 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 335387 T1260 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 335392 T1260 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 335841 T1260 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 335842 T1260 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 335842 T1260 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 335848 T1260 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 335851 T1260 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 335883 T1260 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 335889 T1260 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 335896 T1260 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 335898 T1260 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 335899 T1260 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 335899 T1260 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 335901 T1260 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 335902 T1260 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 335902 T1260 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 335903 T1260 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1371898247013/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty4/
[junit4:junit4] 2> 335904 T1260 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1200120
[junit4:junit4] 2> 335904 T1260 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 335905 T1260 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty4
[junit4:junit4] 2> 335906 T1260 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty4/index/
[junit4:junit4] 2> 335907 T1260 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 335907 T1260 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty4/index
[junit4:junit4] 2> 335909 T1260 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@31dc31dc lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fb85fb8),segFN=segments_1,generation=1}
[junit4:junit4] 2> 335910 T1260 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 335913 T1260 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 335914 T1260 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 335915 T1260 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 335917 T1260 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 335918 T1260 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 335919 T1260 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 335920 T1260 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 335921 T1260 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 335922 T1260 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 335923 T1260 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 335925 T1260 oass.SolrIndexSearcher.<init> Opening Searcher@65cc65cc main
[junit4:junit4] 2> 335926 T1260 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty4/tlog
[junit4:junit4] 2> 335927 T1260 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 335928 T1260 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 335936 T1261 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@65cc65cc main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 335942 T1260 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 335943 T1260 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59604/o_ collection:collection1 shard:shard2
[junit4:junit4] 2> 335950 T1260 oasc.ZkController.register We are http://127.0.0.1:59604/o_/collection1/ and leader is http://127.0.0.1:57565/o_/collection1/
[junit4:junit4] 2> 335951 T1260 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59604/o_
[junit4:junit4] 2> 335951 T1260 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 335952 T1260 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C422 name=collection1 org.apache.solr.core.SolrCore@15bb15bb url=http://127.0.0.1:59604/o_/collection1 node=127.0.0.1:59604_o_ C422_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:59604/o_, state=down, node_name=127.0.0.1:59604_o_, collection=collection1, core=collection1}
[junit4:junit4] 2> 335953 T1262 C422 P59604 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> ASYNC NEW_CORE C423 name=collection1 org.apache.solr.core.SolrCore@345d345d url=http://127.0.0.1:50938/o_/collection1 node=127.0.0.1:50938_o_ C423_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:50938/o_, state=recovering, node_name=127.0.0.1:50938_o_, collection=collection1, shard=shard1, core=collection1}
[junit4:junit4] 2> 335954 T1245 C423 P50938 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:52718/o_/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 335954 T1262 C422 P59604 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 335954 T1260 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 335955 T1262 C422 P59604 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 335956 T1262 C422 P59604 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 335954 T1245 C423 P50938 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50938/o_ START replicas=[http://127.0.0.1:52718/o_/collection1/] nUpdates=100
[junit4:junit4] 2> 335957 T1168 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 335958 T1168 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 335957 T1245 C423 P50938 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 335960 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 335960 T1245 C423 P50938 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 335962 T1245 C423 P50938 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 335958 T1262 C422 P59604 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 335962 T1245 C423 P50938 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 335964 T1245 C423 P50938 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 335964 T1245 C423 P50938 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:52718/o_/collection1/. core=collection1
[junit4:junit4] 2> 335965 T1245 C423 P50938 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C424 name=collection1 org.apache.solr.core.SolrCore@46944694 url=http://127.0.0.1:52718/o_/collection1 node=127.0.0.1:52718_o_ C424_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:52718/o_, state=active, node_name=127.0.0.1:52718_o_, collection=collection1, shard=shard1, core=collection1, leader=true}
[junit4:junit4] 2> 335965 T1203 C424 P52718 oasc.SolrCore.execute [collection1] webapp=/o_ path=/get params={getVersions=100&qt=/get&distrib=false&version=2&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 335984 T1220 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> ASYNC NEW_CORE C425 name=collection1 org.apache.solr.core.SolrCore@46944694 url=http://127.0.0.1:52718/o_/collection1 node=127.0.0.1:52718_o_ C425_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:52718/o_, state=active, node_name=127.0.0.1:52718_o_, collection=collection1, shard=shard1, core=collection1, leader=true}
[junit4:junit4] 2> 335986 T1204 C425 P52718 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 335986 T1168 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 335989 T1204 C425 P52718 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3e9c3e9c lockFactory=org.apache.lucene.store.NativeFSLockFactory@66186618),segFN=segments_1,generation=1}
[junit4:junit4] 2> 335990 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 335990 T1204 C425 P52718 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 335993 T1204 C425 P52718 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3e9c3e9c lockFactory=org.apache.lucene.store.NativeFSLockFactory@66186618),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3e9c3e9c lockFactory=org.apache.lucene.store.NativeFSLockFactory@66186618),segFN=segments_2,generation=2}
[junit4:junit4] 2> 335994 T1204 C425 P52718 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 335995 T1204 C425 P52718 oass.SolrIndexSearcher.<init> Opening Searcher@1090109 realtime
[junit4:junit4] 2> 335997 T1204 C425 P52718 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 335998 T1204 C425 P52718 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_ path=/update params={softCommit=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin&openSearcher=false} {commit=} 0 13
[junit4:junit4] 2> ASYNC NEW_CORE C426 name=collection1 org.apache.solr.core.SolrCore@345d345d url=http://127.0.0.1:50938/o_/collection1 node=127.0.0.1:50938_o_ C426_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:50938/o_, state=recovering, node_name=127.0.0.1:50938_o_, collection=collection1, shard=shard1, core=collection1}
[junit4:junit4] 2> 336000 T1245 C426 P50938 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 336000 T1245 C426 P50938 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 336001 T1203 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 336003 T1205 C425 P52718 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 336004 T1205 C425 P52718 oasc.SolrCore.execute [collection1] webapp=/o_ path=/replication params={qt=/replication&version=2&command=indexversion&wt=javabin} status=0 QTime=1
[junit4:junit4] 2> 336006 T1245 C426 P50938 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 336007 T1245 C426 P50938 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 336007 T1245 C426 P50938 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 336010 T1204 C425 P52718 oasc.SolrCore.execute [collection1] webapp=/o_ path=/replication params={qt=/replication&version=2&generation=2&command=filelist&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 336011 T1245 C426 P50938 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 336013 T1245 C426 P50938 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty3/index.20130622135050075
[junit4:junit4] 2> 336013 T1245 C426 P50938 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@1eea1eea lockFactory=org.apache.lucene.store.NativeFSLockFactory@14b814b8) fullCopy=false
[junit4:junit4] 2> ASYNC NEW_CORE C427 name=collection1 org.apache.solr.core.SolrCore@46944694 url=http://127.0.0.1:52718/o_/collection1 node=127.0.0.1:52718_o_ C427_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:52718/o_, state=active, node_name=127.0.0.1:52718_o_, collection=collection1, shard=shard1, core=collection1, leader=true}
[junit4:junit4] 2> 336060 T1206 C427 P52718 oasc.SolrCore.execute [collection1] webapp=/o_ path=/replication params={checksum=true&qt=/replication&generation=2&command=filecontent&file=segments_2&wt=filestream} status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C428 name=collection1 org.apache.solr.core.SolrCore@345d345d url=http://127.0.0.1:50938/o_/collection1 node=127.0.0.1:50938_o_ C428_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:50938/o_, state=recovering, node_name=127.0.0.1:50938_o_, collection=collection1, shard=shard1, core=collection1}
[junit4:junit4] 2> 336062 T1245 C428 P50938 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 336064 T1245 C428 P50938 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 336064 T1245 C428 P50938 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 336066 T1245 C428 P50938 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3b403b4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8a408a4),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3b403b4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8a408a4),segFN=segments_2,generation=2}
[junit4:junit4] 2> 336067 T1245 C428 P50938 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 336068 T1245 C428 P50938 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 336068 T1245 C428 P50938 oass.SolrIndexSearcher.<init> Opening Searcher@3cc63cc6 main
[junit4:junit4] 2> 336070 T1244 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3cc63cc6 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 336071 T1245 C428 P50938 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty3/index.20130622135050075 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty3/index.20130622135050075;done=true>>]
[junit4:junit4] 2> 336072 T1245 C428 P50938 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty3/index.20130622135050075
[junit4:junit4] 2> 336073 T1245 C428 P50938 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty3/index.20130622135050075
[junit4:junit4] 2> 336073 T1245 C428 P50938 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 336074 T1245 C428 P50938 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 336075 T1245 C428 P50938 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 336075 T1245 C428 P50938 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 336084 T1245 C428 P50938 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 336492 T1191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 336493 T1191 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:59604/o_",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "node_name":"127.0.0.1:59604_o_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 336592 T1191 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52718/o_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:52718_o_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 336592 T1191 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4:junit4] 2> 336593 T1191 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 336599 T1191 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50938/o_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:50938_o_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 336605 T1259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 336606 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 336606 T1197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 336606 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 336606 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 336606 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 336985 T1220 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 336986 T1220 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={state=recovering&onlyIfLeader=true&checkLive=true&action=PREPRECOVERY&version=2&coreNodeName=5&wt=javabin&nodeName=127.0.0.1:59604_o_&core=collection1} status=0 QTime=1003
[junit4:junit4] 2> 337004 T1203 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1371898234811/onenodecollectioncore
[junit4:junit4] 2> 337005 T1203 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4:junit4] 2> 337007 T1203 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4:junit4] 2> 337008 T1203 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 337011 T1203 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 337012 T1203 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4:junit4] 2> 337018 T1203 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4:junit4] 2> 337021 T1203 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1371898234811/onenodecollectioncore/'
[junit4:junit4] 2> 337075 T1203 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 337110 T1203 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 337113 T1203 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 337119 T1203 oass.IndexSchema.readSchema [onenodecollectioncore] Schema name=test
[junit4:junit4] 2> 337561 T1203 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 337562 T1203 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 337564 T1203 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 337569 T1203 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 337573 T1203 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 337595 T1203 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 337602 T1203 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 337609 T1203 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 337612 T1203 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 337613 T1203 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 337614 T1203 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 337616 T1203 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 337617 T1203 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 337618 T1203 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 337619 T1203 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1371898234811/onenodecollectioncore/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1371898231009/onenodecollection/
[junit4:junit4] 2> 337620 T1203 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1200120
[junit4:junit4] 2> 337621 T1203 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 337622 T1203 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1371898231009/onenodecollection
[junit4:junit4] 2> 337623 T1203 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1371898231009/onenodecollection/index/
[junit4:junit4] 2> 337624 T1203 oasc.SolrCore.initIndex WARN [onenodecollectioncore] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1371898231009/onenodecollection/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 337625 T1203 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1371898231009/onenodecollection/index
[junit4:junit4] 2> 337628 T1203 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@33063306 lockFactory=org.apache.lucene.store.NativeFSLockFactory@63f063f0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 337629 T1203 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 337632 T1203 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 337633 T1203 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 337634 T1203 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 337635 T1203 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 337636 T1203 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 337637 T1203 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 337638 T1203 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 337639 T1203 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 337640 T1203 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 337642 T1203 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 337655 T1203 oass.SolrIndexSearcher.<init> Opening Searcher@37353735 main
[junit4:junit4] 2> 337656 T1203 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1371898231009/onenodecollection/tlog
[junit4:junit4] 2> 337658 T1203 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 337658 T1203 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 337666 T1266 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@37353735 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 337669 T1203 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4:junit4] 2> 337670 T1203 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:52718/o_ collection:onenodecollection shard:shard1
[junit4:junit4] 2> 337672 T1203 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4:junit4] 2> 337684 T1203 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 337690 T1203 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 337691 T1203 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 337691 T1203 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52718/o_/onenodecollectioncore/
[junit4:junit4] 2> 337692 T1203 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 337693 T1203 oasc.SyncStrategy.syncToMe http://127.0.0.1:52718/o_/onenodecollectioncore/ has no replicas
[junit4:junit4] 2> 337694 T1203 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52718/o_/onenodecollectioncore/
[junit4:junit4] 2> 337695 T1203 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4:junit4] 2> 338122 T1191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 338135 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 338135 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 338135 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 338135 T1197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 338135 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 338135 T1259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 338162 T1203 oasc.ZkController.register We are http://127.0.0.1:52718/o_/onenodecollectioncore/ and leader is http://127.0.0.1:52718/o_/onenodecollectioncore/
[junit4:junit4] 2> 338164 T1203 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:52718/o_
[junit4:junit4] 2> 338165 T1203 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 338166 T1203 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4:junit4] 2> 338169 T1203 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 338172 T1203 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1371898231009/onenodecollection&action=CREATE&version=2&numShards=1&name=onenodecollectioncore&wt=javabin&roles=none&collection=onenodecollection} status=0 QTime=2171
[junit4:junit4] 2> 338173 T1168 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 338175 T1168 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C429 name=collection1 org.apache.solr.core.SolrCore@15bb15bb url=http://127.0.0.1:59604/o_/collection1 node=127.0.0.1:59604_o_ C429_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:59604/o_, state=recovering, node_name=127.0.0.1:59604_o_, collection=collection1, shard=shard2, core=collection1}
[junit4:junit4] 2> 338988 T1262 C429 P59604 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:57565/o_/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 338989 T1262 C429 P59604 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:59604/o_ START replicas=[http://127.0.0.1:57565/o_/collection1/] nUpdates=100
[junit4:junit4] 2> ASYNC NEW_CORE C430 name=collection1 org.apache.solr.core.SolrCore@15bb15bb url=http://127.0.0.1:59604/o_/collection1 node=127.0.0.1:59604_o_ C430_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:59604/o_, state=recovering, node_name=127.0.0.1:59604_o_, collection=collection1, shard=shard2, core=collection1}
[junit4:junit4] 2> 339021 T1262 C430 P59604 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 339021 T1262 C430 P59604 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 339022 T1262 C430 P59604 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 339022 T1262 C430 P59604 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 339023 T1262 C430 P59604 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 339023 T1262 C430 P59604 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:57565/o_/collection1/. core=collection1
[junit4:junit4] 2> 339024 T1262 C430 P59604 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C431 name=collection1 org.apache.solr.core.SolrCore@72f072f0 url=http://127.0.0.1:57565/o_/collection1 node=127.0.0.1:57565_o_ C431_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:57565/o_, state=active, node_name=127.0.0.1:57565_o_, collection=collection1, shard=shard2, core=collection1, leader=true}
[junit4:junit4] 2> 339028 T1222 C431 P57565 oasc.SolrCore.execute [collection1] webapp=/o_ path=/get params={getVersions=100&qt=/get&distrib=false&version=2&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 339030 T1220 C431 P57565 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 339032 T1220 C431 P57565 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261c261c lockFactory=org.apache.lucene.store.NativeFSLockFactory@19d819d8),segFN=segments_1,generation=1}
[junit4:junit4] 2> 339033 T1220 C431 P57565 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 339035 T1220 C431 P57565 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261c261c lockFactory=org.apache.lucene.store.NativeFSLockFactory@19d819d8),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261c261c lockFactory=org.apache.lucene.store.NativeFSLockFactory@19d819d8),segFN=segments_2,generation=2}
[junit4:junit4] 2> 339035 T1220 C431 P57565 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 339036 T1220 C431 P57565 oass.SolrIndexSearcher.<init> Opening Searcher@1f4a1f4a realtime
[junit4:junit4] 2> 339037 T1220 C431 P57565 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 339038 T1220 C431 P57565 oasup.LogUpdateProcessor.finish [collection1] webapp=/o_ path=/update params={softCommit=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin&openSearcher=false} {commit=} 0 8
[junit4:junit4] 2> 339039 T1262 C430 P59604 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 339040 T1262 C430 P59604 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 339041 T1221 C431 P57565 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 339042 T1221 C431 P57565 oasc.SolrCore.execute [collection1] webapp=/o_ path=/replication params={qt=/replication&version=2&command=indexversion&wt=javabin} status=0 QTime=1
[junit4:junit4] 2> 339043 T1262 C430 P59604 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 339044 T1262 C430 P59604 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 339044 T1262 C430 P59604 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 339046 T1222 C431 P57565 oasc.SolrCore.execute [collection1] webapp=/o_ path=/replication params={qt=/replication&version=2&generation=2&command=filelist&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 339047 T1262 C430 P59604 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 339048 T1262 C430 P59604 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty4/index.20130622135053111
[junit4:junit4] 2> 339049 T1262 C430 P59604 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@15801580 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ff64ff6) fullCopy=false
[junit4:junit4] 2> 339051 T1220 C431 P57565 oasc.SolrCore.execute [collection1] webapp=/o_ path=/replication params={checksum=true&qt=/replication&generation=2&command=filecontent&file=segments_2&wt=filestream} status=0 QTime=0
[junit4:junit4] 2> 339052 T1262 C430 P59604 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 339053 T1262 C430 P59604 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 339053 T1262 C430 P59604 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 339055 T1262 C430 P59604 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@31dc31dc lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fb85fb8),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@31dc31dc lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fb85fb8),segFN=segments_2,generation=2}
[junit4:junit4] 2> 339055 T1262 C430 P59604 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 339056 T1262 C430 P59604 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 339056 T1262 C430 P59604 oass.SolrIndexSearcher.<init> Opening Searcher@5e1a5e1a main
[junit4:junit4] 2> 339058 T1261 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5e1a5e1a main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 339059 T1262 C430 P59604 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty4/index.20130622135053111 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty4/index.20130622135053111;done=true>>]
[junit4:junit4] 2> 339059 T1262 C430 P59604 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty4/index.20130622135053111
[junit4:junit4] 2> 339060 T1262 C430 P59604 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty4/index.20130622135053111
[junit4:junit4] 2> 339060 T1262 C430 P59604 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 339061 T1262 C430 P59604 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 339061 T1262 C430 P59604 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 339062 T1262 C430 P59604 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 339064 T1262 C430 P59604 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 339178 T1168 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 339650 T1191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 339651 T1191 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52718/o_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:52718_o__onenodecollectioncore",
[junit4:junit4] 2> "node_name":"127.0.0.1:52718_o_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 339657 T1191 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:59604/o_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "node_name":"127.0.0.1:59604_o_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 339664 T1259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 339664 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 339665 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 339665 T1197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 339664 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 339666 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 340182 T1168 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 340183 T1168 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4:junit4] 2> 340184 T1168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 340210 T1168 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 340212 T1270 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@79687968 name:ZooKeeperConnection Watcher:127.0.0.1:39763 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 340213 T1168 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 340307 T1168 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 33322
[junit4:junit4] 2> 340308 T1168 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1552112771
[junit4:junit4] 2> 341200 T1191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 341202 T1191 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:33322/o_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:33322_o_",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 341209 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 341209 T1259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 341209 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 341210 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 341209 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 342311 T1168 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 342312 T1168 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 342314 T1168 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@52035203
[junit4:junit4] 2> 342318 T1168 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 342319 T1168 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 342319 T1168 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 342320 T1168 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 342321 T1168 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 342322 T1168 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 342322 T1168 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/control/data/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/control/data/index;done=false>>]
[junit4:junit4] 2> 342323 T1168 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/control/data/index
[junit4:junit4] 2> 342323 T1168 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/control/data [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/control/data;done=false>>]
[junit4:junit4] 2> 342324 T1168 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/control/data
[junit4:junit4] 2> 342326 T1191 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89908722467733507-127.0.0.1:33322_o_-n_0000000000) am no longer a leader.
[junit4:junit4] 2> 342326 T1259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 342327 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 342328 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 342330 T1210 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 342331 T1226 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 342332 T1259 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 342333 T1242 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 342336 T1210 oasc.Overseer.start Overseer (id=89908722467733510-127.0.0.1:52718_o_-n_0000000001) starting
[junit4:junit4] 2> 342346 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 342346 T1272 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 342348 T1168 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/o_,null}
[junit4:junit4] 2> 342351 T1210 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 342353 T1271 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 342357 T1271 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 342358 T1271 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:33322/o_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:33322_o_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 342366 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 342366 T1259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 342366 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 342366 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 342401 T1168 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 52718
[junit4:junit4] 2> 342401 T1168 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1057898254
[junit4:junit4] 2> 343627 T1190 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 343628 T1190 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 343628 T1190 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 343890 T1271 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 343892 T1271 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52718/o_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:52718_o__onenodecollectioncore",
[junit4:junit4] 2> "node_name":"127.0.0.1:52718_o_",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 343901 T1271 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52718/o_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:52718_o_",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 343909 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 343909 T1259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 343909 T1210 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 343909 T1226 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 345405 T1168 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 345406 T1168 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 345408 T1168 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 345410 T1168 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@46944694
[junit4:junit4] 2> 345414 T1168 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 345415 T1168 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 345416 T1168 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 345416 T1168 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 345417 T1168 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 345418 T1168 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 345419 T1168 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty1/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty1/index;done=false>>]
[junit4:junit4] 2> 345419 T1168 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty1/index
[junit4:junit4] 2> 345420 T1168 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty1 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty1;done=false>>]
[junit4:junit4] 2> 345421 T1168 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1371898231009/jetty1
[junit4:junit4] 2> 345421 T1168 oasc.SolrCore.close [onenodecollectioncore] CLOSING SolrCore org.apache.solr.core.SolrCore@12711271
[junit4:junit4] 2> 345423 T1271 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 345424 T1271 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52718/o_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:52718_o_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 345429 T1168 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 345429 T1168 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 345430 T1271 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52718/o_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:52718_o__onenodecollectioncore",
[junit4:junit4] 2> "node_name":"127.0.0.1:52718_o_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4]
[...truncated too long message...]
lection1"}
[junit4:junit4] 2> 874544 T1819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 874567 T1728 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 53575
[junit4:junit4] 2> 874568 T1728 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1439126983
[junit4:junit4] 2> 875571 T1728 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 875573 T1728 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 875576 T1728 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@68d068d0
[junit4:junit4] 2> 875589 T1728 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=655,adds=655,deletesById=331,deletesByQuery=0,errors=0,cumulative_adds=697,cumulative_deletesById=331,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4] 2> 875590 T1728 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 875591 T1728 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 875591 T1728 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> ASYNC NEW_CORE C1929 name=collection1 org.apache.solr.core.SolrCore@68d068d0 url=http://127.0.0.1:53575/ikgk/sv/collection1 node=127.0.0.1:53575_ikgk%2Fsv C1929_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:53575/ikgk/sv, state=active, node_name=127.0.0.1:53575_ikgk%2Fsv, collection=collection1, shard=shard2, core=collection1, leader=true}
[junit4:junit4] 2> 875599 T1728 C1929 P53575 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50ee50ee lockFactory=org.apache.lucene.store.NativeFSLockFactory@2f402f40),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50ee50ee lockFactory=org.apache.lucene.store.NativeFSLockFactory@2f402f40),segFN=segments_5,generation=5}
[junit4:junit4] 2> 875600 T1728 C1929 P53575 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 875602 T1728 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 875603 T1728 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 875603 T1728 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1371898590419/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1371898590419/jetty4;done=false>>]
[junit4:junit4] 2> 875604 T1728 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1371898590419/jetty4
[junit4:junit4] 2> 875605 T1728 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1371898590419/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1371898590419/jetty4/index;done=false>>]
[junit4:junit4] 2> 875605 T1728 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1371898590419/jetty4/index
[junit4:junit4] 2> 875606 T1854 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89908746021961740-127.0.0.1:53575_ikgk%2Fsv-n_0000000004) am no longer a leader.
[junit4:junit4] 2> 875628 T1728 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/ikgk/sv,null}
[junit4:junit4] 2> 875687 T1728 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 875688 T1728 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51062 51062
[junit4:junit4] 2> 875690 T1728 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 49612
[junit4:junit4] 2> !!!! WARNING: best effort to remove /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ShardSplitTest-1371898590419 FAILED !!!!!
[junit4:junit4] 2> 875691 T1728 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 875692 T1728 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51062 51062
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=6AD2A1B1ADE97BA9 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ml_IN -Dtests.timezone=Hongkong -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR 199s J1 | ShardSplitTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:43941/ikgk/sv returned non ok status:500, message:Server Error
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([6AD2A1B1ADE97BA9:EB342FA9DAB61B95]:0)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
[junit4:junit4] > at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:738)
[junit4:junit4] 2> 875710 T1728 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 199388 T1727 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4] 2> 876908 T1819 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 876908 T1819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 876909 T1819 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {n_ti=Pulsing41(freqCutoff=11 minBlockSize=52 maxBlockSize=113), _version_=Pulsing41(freqCutoff=11 minBlockSize=52 maxBlockSize=113), id=PostingsFormat(name=NestedPulsing)}, docValues:{}, sim=DefaultSimilarity, locale=ml_IN, timezone=Hongkong
[junit4:junit4] 2> NOTE: Linux 3.2.0-48-generic x86/IBM Corporation 1.6.0 (32-bit)/cpus=8,threads=3,free=56148568,total=102611456
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestJmxIntegration, TestReload, SolrIndexConfigTest, TestAtomicUpdateErrorCases, TestFieldCollectionResource, SolrCmdDistributorTest, NoCacheHeaderTest, TestSolrXml, BadIndexSchemaTest, FieldAnalysisRequestHandlerTest, TestMergePolicyConfig, TestAddFieldRealTimeGet, TestSolrQueryParserResource, TestCollationField, SortByFunctionTest, RequestHandlersTest, TestFieldTypeResource, StandardRequestHandlerTest, IndexBasedSpellCheckerTest, CollectionsAPIDistributedZkTest, CurrencyFieldOpenExchangeTest, TestFoldingMultitermQuery, TestSchemaSimilarityResource, DisMaxRequestHandlerTest, TestSolrJ, SampleTest, ZkSolrClientTest, TestIndexingPerformance, DocumentAnalysisRequestHandlerTest, TestAnalyzedSuggestions, CachingDirectoryFactoryTest, TestIndexSearcher, TestMultiCoreConfBootstrap, SolrCoreCheckLockOnStartupTest, TestQueryTypes, TimeZoneUtilsTest, TestPartialUpdateDeduplication, CurrencyFieldXmlFileTest, AliasIntegrationTest, NotRequiredUniqueKeyTest, SimplePostToolTest, UpdateParamsTest, PathHierarchyTokenizerFactoryTest, TestWordDelimiterFilterFactory, CopyFieldTest, TestCoreDiscovery, DOMUtilTest, TestSchemaNameResource, TestDefaultSimilarityFactory, TestStressLucene, XmlUpdateRequestHandlerTest, FullSolrCloudDistribCmdsTest, BasicDistributedZk2Test, DebugComponentTest, TestQuerySenderNoQuery, TestShardHandlerFactory, TermVectorComponentDistributedTest, DateMathParserTest, TestRangeQuery, TestRandomFaceting, TestUpdate, SpellCheckCollatorTest, SolrPluginUtilsTest, ConvertedLegacyTest, TestBinaryResponseWriter, BasicZkTest, DirectSolrSpellCheckerTest, TestSolrXMLSerializer, SpellPossibilityIteratorTest, TestFastOutputStream, ShardSplitTest]
[junit4:junit4] Completed on J1 in 200.67s, 1 test, 1 error <<< FAILURES!
[...truncated 475 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:392: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:385: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1246: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:890: There were test failures: 298 suites, 1256 tests, 2 errors, 1 failure, 14 ignored (8 assumptions)
Total time: 46 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/ibm-j9-jdk6 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure