You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/05/19 13:19:37 UTC

[JENKINS] Lucene-Solr-4.x-MacOSX (64bit/jdk1.7.0) - Build # 468 - Still Failing!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-MacOSX/468/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseParallelGC

2 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
There are still nodes recoverying - waited for 230 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 230 seconds
	at __randomizedtesting.SeedInfo.seed([C1BF6594B1D20D53:4059EB8CC68D6D6F]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:173)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:512)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)


REGRESSION:  org.apache.solr.cloud.ShardSplitTest.testDistribSearch

Error Message:
Server at http://127.0.0.1:54134/xs_z/s returned non ok status:500, message:Server Error

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:54134/xs_z/s returned non ok status:500, message:Server Error
	at __randomizedtesting.SeedInfo.seed([C1BF6594B1D20D53:4059EB8CC68D6D6F]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
	at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:208)
	at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:133)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 9699 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4:junit4]   2> 2095791 T4472 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /fztlc/
[junit4:junit4]   2> 2095804 T4472 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./solrtest-CollectionsAPIDistributedZkTest-1368960623459
[junit4:junit4]   2> 2095806 T4472 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 2095808 T4473 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 2095911 T4472 oasc.ZkTestServer.run start zk server on port:53337
[junit4:junit4]   2> 2095914 T4472 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2095925 T4479 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4df75a83 name:ZooKeeperConnection Watcher:127.0.0.1:53337 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2095926 T4472 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2095927 T4472 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 2095940 T4472 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2095963 T4481 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2cc2021d name:ZooKeeperConnection Watcher:127.0.0.1:53337/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2095964 T4472 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2095969 T4472 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 2095995 T4472 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 2096019 T4472 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 2096029 T4472 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 2096052 T4472 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2096054 T4472 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2096083 T4472 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 2096085 T4472 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 2096101 T4472 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2096102 T4472 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2096113 T4472 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 2096115 T4472 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 2096125 T4472 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 2096126 T4472 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 2096137 T4472 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2096139 T4472 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2096150 T4472 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2096152 T4472 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2096169 T4472 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2096171 T4472 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2096182 T4472 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2096184 T4472 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2097053 T4472 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 2097072 T4472 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53340
[junit4:junit4]   2> 2097074 T4472 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2097075 T4472 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2097077 T4472 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854
[junit4:junit4]   2> 2097078 T4472 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/solr.xml
[junit4:junit4]   2> 2097079 T4472 oasc.CoreContainer.<init> New CoreContainer 33454125
[junit4:junit4]   2> 2097081 T4472 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/'
[junit4:junit4]   2> 2097082 T4472 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/'
[junit4:junit4]   2> 2097351 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854
[junit4:junit4]   2> 2097353 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/collection1
[junit4:junit4]   2> 2097355 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/collection1/conf
[junit4:junit4]   2> 2097536 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/collection1/conf/xslt
[junit4:junit4]   2> 2097541 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/collection1/lib
[junit4:junit4]   2> 2097544 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/collection1/lib/classes
[junit4:junit4]   2> 2097547 T4472 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/conf/core.properties, adding to cores
[junit4:junit4]   2> 2097569 T4472 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2097570 T4472 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2097570 T4472 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2097571 T4472 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2097571 T4472 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2097572 T4472 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2097573 T4472 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2097573 T4472 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2097574 T4472 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2097575 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2097606 T4472 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 2097608 T4472 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53337/solr
[junit4:junit4]   2> 2097608 T4472 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 2097613 T4472 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2097618 T4492 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@76850ab4 name:ZooKeeperConnection Watcher:127.0.0.1:53337 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2097620 T4472 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2097625 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 2097662 T4472 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2097667 T4494 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@49a46317 name:ZooKeeperConnection Watcher:127.0.0.1:53337/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2097668 T4472 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2097674 T4472 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4]   2> 2097683 T4472 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4]   2> 2097698 T4472 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 2097704 T4472 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53340_fztlc
[junit4:junit4]   2> 2097709 T4472 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53340_fztlc
[junit4:junit4]   2> 2097718 T4472 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 2097737 T4472 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 2097744 T4472 oasc.Overseer.start Overseer (id=89716203419992067-127.0.0.1:53340_fztlc-n_0000000000) starting
[junit4:junit4]   2> 2097754 T4472 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4]   2> 2097772 T4496 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 2097773 T4472 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 2097783 T4472 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 2097789 T4472 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 2097801 T4472 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4]   2> 2097802 T4472 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2097802 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 2097811 T4495 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 2097827 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 2097833 T4472 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2097842 T4499 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@621cc396 name:ZooKeeperConnection Watcher:127.0.0.1:53337/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2097843 T4472 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2097848 T4472 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 2097857 T4472 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 2098520 T4472 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 2098529 T4472 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53344
[junit4:junit4]   2> 2098531 T4472 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2098532 T4472 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2098533 T4472 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511
[junit4:junit4]   2> 2098533 T4472 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/solr.xml
[junit4:junit4]   2> 2098534 T4472 oasc.CoreContainer.<init> New CoreContainer 2114466296
[junit4:junit4]   2> 2098535 T4472 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/'
[junit4:junit4]   2> 2098535 T4472 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/'
[junit4:junit4]   2> 2098742 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511
[junit4:junit4]   2> 2098743 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/collection1
[junit4:junit4]   2> 2098745 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/collection1/conf
[junit4:junit4]   2> 2098938 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/collection1/conf/xslt
[junit4:junit4]   2> 2098945 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/collection1/lib
[junit4:junit4]   2> 2098949 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/collection1/lib/classes
[junit4:junit4]   2> 2098955 T4472 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/conf/core.properties, adding to cores
[junit4:junit4]   2> 2098977 T4472 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2098977 T4472 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2098978 T4472 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2098979 T4472 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2098979 T4472 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2098980 T4472 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2098980 T4472 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2098981 T4472 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2098981 T4472 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2098982 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2099032 T4472 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 2099032 T4472 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53337/solr
[junit4:junit4]   2> 2099033 T4472 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 2099038 T4472 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2099043 T4510 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55ce6ab8 name:ZooKeeperConnection Watcher:127.0.0.1:53337 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2099044 T4472 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2099051 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 2099080 T4472 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2099098 T4512 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e1d0b56 name:ZooKeeperConnection Watcher:127.0.0.1:53337/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2099099 T4472 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2099112 T4472 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 2100128 T4472 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53344_fztlc
[junit4:junit4]   2> 2100131 T4472 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53344_fztlc
[junit4:junit4]   2> 2100143 T4494 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 2100145 T4512 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 2100146 T4499 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 2100157 T4472 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4]   2> 2100158 T4472 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2100160 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 2100852 T4472 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 2100866 T4472 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53347
[junit4:junit4]   2> 2100867 T4472 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2100867 T4472 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2100868 T4472 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842
[junit4:junit4]   2> 2100869 T4472 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/solr.xml
[junit4:junit4]   2> 2100869 T4472 oasc.CoreContainer.<init> New CoreContainer 221413636
[junit4:junit4]   2> 2100870 T4472 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/'
[junit4:junit4]   2> 2100871 T4472 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/'
[junit4:junit4]   2> 2101101 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842
[junit4:junit4]   2> 2101103 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/collection1
[junit4:junit4]   2> 2101106 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/collection1/conf
[junit4:junit4]   2> 2101395 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/collection1/conf/xslt
[junit4:junit4]   2> 2101406 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/collection1/lib
[junit4:junit4]   2> 2101409 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/collection1/lib/classes
[junit4:junit4]   2> 2101415 T4472 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/conf/core.properties, adding to cores
[junit4:junit4]   2> 2101439 T4472 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2101441 T4472 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2101442 T4472 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2101442 T4472 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2101443 T4472 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2101444 T4472 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2101445 T4472 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2101446 T4472 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2101447 T4472 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2101448 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2101486 T4472 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 2101487 T4472 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53337/solr
[junit4:junit4]   2> 2101488 T4472 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 2101493 T4472 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2101503 T4524 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71f5e897 name:ZooKeeperConnection Watcher:127.0.0.1:53337 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2101505 T4472 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2101522 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 2101554 T4472 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2101561 T4526 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78b51cd2 name:ZooKeeperConnection Watcher:127.0.0.1:53337/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2101561 T4472 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2101573 T4472 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 2102583 T4472 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53347_fztlc
[junit4:junit4]   2> 2102586 T4472 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53347_fztlc
[junit4:junit4]   2> 2102595 T4494 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 2102596 T4512 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 2102605 T4526 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 2102611 T4499 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 2102611 T4472 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4]   2> 2102612 T4472 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2102613 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 2103445 T4472 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 2103456 T4472 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53350
[junit4:junit4]   2> 2103460 T4472 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2103462 T4472 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2103463 T4472 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292
[junit4:junit4]   2> 2103464 T4472 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/solr.xml
[junit4:junit4]   2> 2103465 T4472 oasc.CoreContainer.<init> New CoreContainer 1576959993
[junit4:junit4]   2> 2103466 T4472 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/'
[junit4:junit4]   2> 2103467 T4472 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/'
[junit4:junit4]   2> 2103688 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292
[junit4:junit4]   2> 2103692 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/collection1
[junit4:junit4]   2> 2103694 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/collection1/conf
[junit4:junit4]   2> 2104022 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/collection1/conf/xslt
[junit4:junit4]   2> 2104030 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/collection1/lib
[junit4:junit4]   2> 2104033 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/collection1/lib/classes
[junit4:junit4]   2> 2104039 T4472 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/conf/core.properties, adding to cores
[junit4:junit4]   2> 2104065 T4472 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2104066 T4472 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2104067 T4472 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2104068 T4472 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2104069 T4472 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2104070 T4472 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2104071 T4472 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2104072 T4472 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2104073 T4472 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2104075 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2104131 T4472 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 2104133 T4472 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53337/solr
[junit4:junit4]   2> 2104134 T4472 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 2104140 T4472 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2104148 T4538 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b4153a5 name:ZooKeeperConnection Watcher:127.0.0.1:53337 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2104150 T4472 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2104158 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 2104198 T4472 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2104208 T4540 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@50c73737 name:ZooKeeperConnection Watcher:127.0.0.1:53337/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2104211 T4472 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2104238 T4472 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 2105253 T4472 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53350_fztlc
[junit4:junit4]   2> 2105257 T4472 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53350_fztlc
[junit4:junit4]   2> 2105281 T4494 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 2105281 T4540 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 2105282 T4512 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 2105283 T4499 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 2105284 T4526 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 2105302 T4472 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4]   2> 2105304 T4472 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2105305 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 2106123 T4472 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 2106132 T4472 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53353
[junit4:junit4]   2> 2106136 T4472 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2106137 T4472 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2106138 T4472 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997
[junit4:junit4]   2> 2106139 T4472 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/solr.xml
[junit4:junit4]   2> 2106140 T4472 oasc.CoreContainer.<init> New CoreContainer 118681180
[junit4:junit4]   2> 2106141 T4472 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/'
[junit4:junit4]   2> 2106142 T4472 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/'
[junit4:junit4]   2> 2106369 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997
[junit4:junit4]   2> 2106371 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/collection1
[junit4:junit4]   2> 2106375 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/collection1/conf
[junit4:junit4]   2> 2106667 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/collection1/conf/xslt
[junit4:junit4]   2> 2106677 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/collection1/lib
[junit4:junit4]   2> 2106680 T4472 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/collection1/lib/classes
[junit4:junit4]   2> 2106685 T4472 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/conf/core.properties, adding to cores
[junit4:junit4]   2> 2106707 T4472 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2106708 T4472 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2106708 T4472 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2106709 T4472 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2106709 T4472 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2106710 T4472 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2106710 T4472 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2106711 T4472 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2106711 T4472 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2106711 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2106746 T4472 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 2106747 T4472 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53337/solr
[junit4:junit4]   2> 2106747 T4472 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 2106749 T4472 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2106756 T4552 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@dff90a1 name:ZooKeeperConnection Watcher:127.0.0.1:53337 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2106758 T4472 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2106765 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 2106799 T4472 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2106804 T4554 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4bf4d55 name:ZooKeeperConnection Watcher:127.0.0.1:53337/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2106805 T4472 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2106826 T4472 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 2107844 T4472 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53353_fztlc
[junit4:junit4]   2> 2108200 T4472 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53353_fztlc
[junit4:junit4]   2> 2108220 T4499 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 2108221 T4494 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 2108225 T4526 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 2108226 T4540 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 2108230 T4554 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 2108247 T4512 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 2108248 T4472 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4]   2> 2108249 T4472 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2108250 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 2108287 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 2108323 T4504 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2
[junit4:junit4]   2> 2108328 T4494 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 2108334 T4494 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 2108336 T4496 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4:junit4]   2> 	  "operation":"createcollection",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "maxShardsPerNode":null,
[junit4:junit4]   2> 	  "createNodeSet":null,
[junit4:junit4]   2> 	  "name":"nodes_used_collection",
[junit4:junit4]   2> 	  "replicationFactor":"2"}
[junit4:junit4]   2> 2108337 T4496 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica1 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:53344_fztlc
[junit4:junit4]   2> 2108338 T4496 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica2 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:53340_fztlc
[junit4:junit4]   2> 2108340 T4496 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica1 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:53353_fztlc
[junit4:junit4]   2> 2108344 T4496 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica2 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:53347_fztlc
[junit4:junit4]   2> 2108353 T4503 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/nodes_used_collection_shard1_replica1
[junit4:junit4]   2> 2108369 T4550 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 2108370 T4550 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 2108369 T4489 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 2108372 T4489 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 2108369 T4517 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 2108374 T4517 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 2108370 T4503 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 2108375 T4489 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 2108382 T4489 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 2108374 T4550 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 2108385 T4550 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 2108386 T4517 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 2108387 T4517 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 2108390 T4503 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 2108391 T4503 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 2108397 T4550 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 2108399 T4503 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 2108400 T4503 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 2108398 T4489 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 2108402 T4489 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 2108399 T4517 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 2108399 T4550 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 2108404 T4517 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 2108417 T4550 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 2108420 T4489 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 2108422 T4503 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 2108430 T4550 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/nodes_used_collection_shard2_replica1/'
[junit4:junit4]   2> 2108440 T4517 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 2108442 T4489 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/nodes_used_collection_shard1_replica2/'
[junit4:junit4]   2> 2108444 T4503 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/nodes_used_collection_shard1_replica1/'
[junit4:junit4]   2> 2108456 T4517 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/nodes_used_collection_shard2_replica2/'
[junit4:junit4]   2> 2108813 T4517 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 2108833 T4503 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 2108897 T4489 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 2108939 T4550 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 2109257 T4503 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2109291 T4517 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2109296 T4503 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 2109309 T4503 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica1] Schema name=test
[junit4:junit4]   2> 2109328 T4517 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 2109340 T4517 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica2] Schema name=test
[junit4:junit4]   2> 2109344 T4489 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2109363 T4550 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2109365 T4489 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 2109382 T4489 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica2] Schema name=test
[junit4:junit4]   2> 2109390 T4550 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 2109407 T4550 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica1] Schema name=test
[junit4:junit4]   2> 2113528 T4550 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2113564 T4550 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2113588 T4517 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2113603 T4503 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2113607 T4550 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2113620 T4517 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2113631 T4503 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2113641 T4503 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2113644 T4517 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2113820 T4550 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2113828 T4517 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2113841 T4550 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2113846 T4503 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2113857 T4517 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2113871 T4550 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2113865 T4503 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2113887 T4517 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2113891 T4550 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2113899 T4550 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2113903 T4550 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2113910 T4517 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2113946 T4517 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2113948 T4517 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2113944 T4489 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2113916 T4503 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2113911 T4550 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2113969 T4550 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2113970 T4550 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2113971 T4550 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/nodes_used_collection_shard2_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/nodes_used_collection_shard2_replica1/data/
[junit4:junit4]   2> 2113972 T4550 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b973e81
[junit4:junit4]   2> 2113974 T4550 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2113968 T4503 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2113977 T4550 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/nodes_used_collection_shard2_replica1/data
[junit4:junit4]   2> 2113977 T4503 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2113979 T4503 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2113966 T4517 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2113983 T4517 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2113993 T4517 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2113981 T4550 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/nodes_used_collection_shard2_replica1/data/index/
[junit4:junit4]   2> 2113995 T4550 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/nodes_used_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2113969 T4489 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2113996 T4550 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/nodes_used_collection_shard2_replica1/data/index
[junit4:junit4]   2> 2113994 T4517 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/nodes_used_collection_shard2_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/nodes_used_collection_shard2_replica2/data/
[junit4:junit4]   2> 2114007 T4517 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b973e81
[junit4:junit4]   2> 2114008 T4517 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2113990 T4503 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2114011 T4503 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2114012 T4503 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2114013 T4517 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/nodes_used_collection_shard2_replica2/data
[junit4:junit4]   2> 2114007 T4489 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2114013 T4503 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/nodes_used_collection_shard1_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/nodes_used_collection_shard1_replica1/data/
[junit4:junit4]   2> 2114024 T4517 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/nodes_used_collection_shard2_replica2/data/index/
[junit4:junit4]   2> 2114025 T4517 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/nodes_used_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2114027 T4503 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b973e81
[junit4:junit4]   2> 2114029 T4503 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2114033 T4517 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/nodes_used_collection_shard2_replica2/data/index
[junit4:junit4]   2> 2114043 T4503 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/nodes_used_collection_shard1_replica1/data
[junit4:junit4]   2> 2114051 T4550 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@127a4bd5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@77d6a038),segFN=segments_1,generation=1}
[junit4:junit4]   2> 2114052 T4550 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 2114054 T4503 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/nodes_used_collection_shard1_replica1/data/index/
[junit4:junit4]   2> 2114055 T4503 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/nodes_used_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2114061 T4503 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/nodes_used_collection_shard1_replica1/data/index
[junit4:junit4]   2> 2114065 T4517 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7b261801 lockFactory=org.apache.lucene.store.NativeFSLockFactory@d44f0e3),segFN=segments_1,generation=1}
[junit4:junit4]   2> 2114066 T4517 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 2114075 T4550 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2114076 T4550 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2114078 T4550 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2114080 T4517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2114081 T4550 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2114082 T4503 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@73b95fa8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@13c14009),segFN=segments_1,generation=1}
[junit4:junit4]   2> 2114084 T4503 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 2114082 T4517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2114083 T4550 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2114089 T4550 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2114089 T4517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2114090 T4550 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2114093 T4517 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2114093 T4550 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2114097 T4517 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2114098 T4517 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2114099 T4517 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2114102 T4517 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2114102 T4503 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2114103 T4550 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2114103 T4517 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2114104 T4503 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2114121 T4503 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2114124 T4503 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2114125 T4503 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2114126 T4503 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2114127 T4503 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2114136 T4503 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2114138 T4503 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2114153 T4550 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2114155 T4503 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2114153 T4517 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2114180 T4517 oass.SolrIndexSearcher.<init> Opening Searcher@20ac7dc3 main
[junit4:junit4]   2> 2114181 T4517 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/nodes_used_collection_shard2_replica2/data/tlog
[junit4:junit4]   2> 2114184 T4517 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2114186 T4517 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2114197 T4561 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@20ac7dc3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2114198 T4550 oass.SolrIndexSearcher.<init> Opening Searcher@82c1d18 main
[junit4:junit4]   2> 2114200 T4550 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/nodes_used_collection_shard2_replica1/data/tlog
[junit4:junit4]   2> 2114201 T4517 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=down
[junit4:junit4]   2> 2114203 T4550 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2114204 T4550 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2114209 T4503 oass.SolrIndexSearcher.<init> Opening Searcher@78bc9c0f main
[junit4:junit4]   2> 2114211 T4562 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@82c1d18 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2114212 T4503 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/nodes_used_collection_shard1_replica1/data/tlog
[junit4:junit4]   2> 2114217 T4550 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=down
[junit4:junit4]   2> 2114222 T4503 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2114223 T4503 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2114230 T4560 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@78bc9c0f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2114234 T4503 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=down
[junit4:junit4]   2> 2114245 T4489 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2114288 T4489 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2114302 T4489 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2114307 T4489 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2114308 T4489 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2114309 T4489 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2114314 T4489 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2114315 T4489 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2114316 T4489 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2114317 T4489 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/nodes_used_collection_shard1_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/nodes_used_collection_shard1_replica2/data/
[junit4:junit4]   2> 2114318 T4489 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b973e81
[junit4:junit4]   2> 2114319 T4489 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2114321 T4489 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/nodes_used_collection_shard1_replica2/data
[junit4:junit4]   2> 2114322 T4489 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/nodes_used_collection_shard1_replica2/data/index/
[junit4:junit4]   2> 2114324 T4489 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/nodes_used_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2114332 T4489 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/nodes_used_collection_shard1_replica2/data/index
[junit4:junit4]   2> 2114354 T4489 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6832daa lockFactory=org.apache.lucene.store.NativeFSLockFactory@1151a7b0),segFN=segments_1,generation=1}
[junit4:junit4]   2> 2114355 T4489 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 2114362 T4489 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2114363 T4489 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2114365 T4489 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2114367 T4489 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2114369 T4489 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2114370 T4489 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2114371 T4489 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2114373 T4489 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2114375 T4489 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2114409 T4489 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2114426 T4489 oass.SolrIndexSearcher.<init> Opening Searcher@2fcbe5e4 main
[junit4:junit4]   2> 2114428 T4489 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/nodes_used_collection_shard1_replica2/data/tlog
[junit4:junit4]   2> 2114430 T4489 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2114431 T4489 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2114438 T4563 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@2fcbe5e4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2114442 T4489 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=down
[junit4:junit4]   2> 2114495 T4495 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2114498 T4495 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53347_fztlc",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53347/fztlc"}
[junit4:junit4]   2> 2114498 T4495 oasc.Overseer$ClusterStateUpdater.createCollection Create collection nodes_used_collection with numShards 2
[junit4:junit4]   2> 2114512 T4495 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53353_fztlc",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53353/fztlc"}
[junit4:junit4]   2> 2114524 T4495 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53344_fztlc",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53344/fztlc"}
[junit4:junit4]   2> 2114547 T4495 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53340_fztlc",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53340/fztlc"}
[junit4:junit4]   2> 2114564 T4494 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2114565 T4554 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2114565 T4499 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2114565 T4526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2114564 T4540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2114565 T4512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2115231 T4517 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 2115233 T4517 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica2 address:http://127.0.0.1:53347/fztlc collection:nodes_used_collection shard:shard2
[junit4:junit4]   2> 2115234 T4550 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 2115236 T4550 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica1 address:http://127.0.0.1:53353/fztlc collection:nodes_used_collection shard:shard2
[junit4:junit4]   2> 2115239 T4517 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard2/election
[junit4:junit4]   2> 2115242 T4503 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica1
[junit4:junit4]   2> 2115243 T4503 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica1 address:http://127.0.0.1:53344/fztlc collection:nodes_used_collection shard:shard1
[junit4:junit4]   2> 2115253 T4503 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2115243 T4550 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard2/election
[junit4:junit4]   2> 2115295 T4503 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2115310 T4503 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=179997
[junit4:junit4]   2> 2115311 T4517 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2115322 T4517 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2115324 T4517 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2115325 T4517 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:53347/fztlc/nodes_used_collection_shard2_replica2/
[junit4:junit4]   2> 2115326 T4517 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:53347/fztlc START replicas=[http://127.0.0.1:53353/fztlc/nodes_used_collection_shard2_replica1/] nUpdates=100
[junit4:junit4]   2> 2115327 T4517 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:53347/fztlc DONE.  We have no versions.  sync failed.
[junit4:junit4]   2> 2115329 T4517 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4]   2> 2115330 T4517 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4]   2> 2115331 T4517 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:53347/fztlc/nodes_used_collection_shard2_replica2/
[junit4:junit4]   2> 2115334 T4517 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard2
[junit4:junit4]   2> ASYNC  NEW_CORE C290 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@61f9cbbb url=http://127.0.0.1:53353/fztlc/nodes_used_collection_shard2_replica1 node=127.0.0.1:53353_fztlc C290_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=down, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:53353_fztlc, base_url=http://127.0.0.1:53353/fztlc}
[junit4:junit4]   2> 2115353 T4545 C290 P53353 oasc.SolrCore.execute [nodes_used_collection_shard2_replica1] webapp=/fztlc path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 2115447 T4489 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 2115448 T4489 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica2 address:http://127.0.0.1:53340/fztlc collection:nodes_used_collection shard:shard1
[junit4:junit4]   2> 2115815 T4503 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2115816 T4503 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2115817 T4503 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:53344/fztlc/nodes_used_collection_shard1_replica1/
[junit4:junit4]   2> 2115817 T4503 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:53344/fztlc START replicas=[http://127.0.0.1:53340/fztlc/nodes_used_collection_shard1_replica2/] nUpdates=100
[junit4:junit4]   2> 2115819 T4503 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:53344/fztlc DONE.  We have no versions.  sync failed.
[junit4:junit4]   2> 2115820 T4503 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4]   2> 2115821 T4503 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4]   2> 2115822 T4503 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:53344/fztlc/nodes_used_collection_shard1_replica1/
[junit4:junit4]   2> 2115825 T4503 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard1
[junit4:junit4]   2> ASYNC  NEW_CORE C291 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@62012198 url=http://127.0.0.1:53340/fztlc/nodes_used_collection_shard1_replica2 node=127.0.0.1:53340_fztlc C291_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=down, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:53340_fztlc, base_url=http://127.0.0.1:53340/fztlc}
[junit4:junit4]   2> 2115842 T4487 C291 P53340 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp=/fztlc path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 2116110 T4495 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2116165 T4554 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2116173 T4540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2116175 T4526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2116176 T4499 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2116184 T4512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2116176 T4494 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2116211 T4517 oasc.ZkController.register We are http://127.0.0.1:53347/fztlc/nodes_used_collection_shard2_replica2/ and leader is http://127.0.0.1:53347/fztlc/nodes_used_collection_shard2_replica2/
[junit4:junit4]   2> 2116211 T4517 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica2 baseURL=http://127.0.0.1:53347/fztlc
[junit4:junit4]   2> 2116212 T4517 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 2116212 T4517 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=active
[junit4:junit4]   2> 2116217 T4517 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2116220 T4517 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=7851 
[junit4:junit4]   2> 2116228 T4503 oasc.ZkController.register We are http://127.0.0.1:53344/fztlc/nodes_used_collection_shard1_replica1/ and leader is http://127.0.0.1:53344/fztlc/nodes_used_collection_shard1_replica1/
[junit4:junit4]   2> 2116229 T4503 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica1 baseURL=http://127.0.0.1:53344/fztlc
[junit4:junit4]   2> 2116229 T4503 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 2116230 T4503 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=active
[junit4:junit4]   2> 2116234 T4503 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2116236 T4503 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=7883 
[junit4:junit4]   2> 2116315 T4550 oasc.ZkController.register We are http://127.0.0.1:53353/fztlc/nodes_used_collection_shard2_replica1/ and leader is http://127.0.0.1:53347/fztlc/nodes_used_collection_shard2_replica2/
[junit4:junit4]   2> 2116316 T4550 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica1 baseURL=http://127.0.0.1:53353/fztlc
[junit4:junit4]   2> 2116316 T4550 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 2116316 T4550 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> 2116318 T4566 C290 P53353 oasc.RecoveryStrategy.run Starting recovery process.  core=nodes_used_collection_shard2_replica1 recoveringAfterStartup=true
[junit4:junit4]   2> 2116318 T4566 C290 P53353 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 2116320 T4550 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2116322 T4550 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=7953 
[junit4:junit4]   2> 2116324 T4566 C290 P53353 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=recovering
[junit4:junit4]   2> 2116327 T4566 C290 P53353 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 2116368 T4519 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:53353_fztlc_nodes_used_collection_shard2_replica1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 2116460 T4489 oasc.ZkController.register We are http://127.0.0.1:53340/fztlc/nodes_used_collection_shard1_replica2/ and leader is http://127.0.0.1:53344/fztlc/nodes_used_collection_shard1_replica1/
[junit4:junit4]   2> 2116461 T4489 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica2 baseURL=http://127.0.0.1:53340/fztlc
[junit4:junit4]   2> 2116461 T4489 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 2116462 T4489 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> 2116463 T4567 C291 P53340 oasc.RecoveryStrategy.run Starting recovery process.  core=nodes_used_collection_shard1_replica2 recoveringAfterStartup=true
[junit4:junit4]   2> 2116463 T4567 C291 P53340 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 2116465 T4567 C291 P53340 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=recovering
[junit4:junit4]   2> 2116467 T4489 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2116473 T4567 C291 P53340 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 2116475 T4489 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=8106 
[junit4:junit4]   2> 2116477 T4496 oasc.OverseerCollectionProcessor.createCollection Finished create command on all shards for collection: nodes_used_collection
[junit4:junit4]   2> 2116510 T4508 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:53340_fztlc_nodes_used_collection_shard1_replica2, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 2116511 T4496 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{success={null={responseHeader={status=0,QTime=7851},core=nodes_used_collection_shard2_replica2},null={responseHeader={status=0,QTime=7883},core=nodes_used_collection_shard1_replica1},null={responseHeader={status=0,QTime=7953},core=nodes_used_collection_shard2_replica1},null={responseHeader={status=0,QTime=8106},core=nodes_used_collection_shard1_replica2}}}
[junit4:junit4]   2> 2116513 T4512 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type NodeDataChanged
[junit4:junit4]   2> 2116517 T4494 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 2116519 T4504 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2} status=0 QTime=8196 
[junit4:junit4]   2> 2116521 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 2116523 T4472 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2116529 T4569 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13e0350e name:ZooKeeperConnection Watcher:127.0.0.1:53337/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2116530 T4472 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2116535 T4472 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 2116549 T4472 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 2116552 T4472 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2116558 T4571 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5f51f251 name:ZooKeeperConnection Watcher:127.0.0.1:53337/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2116558 T4472 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2116564 T4472 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 2116595 T4521 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=3&maxShardsPerNode=2&name=awholynewcollection_0&replicationFactor=2&action=CREATE&wt=javabin&version=2
[junit4:junit4]   2> 2116598 T4494 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 2116603 T4496 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000002 message:{
[junit4:junit4]   2> 	  "operation":"createcollection",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "maxShardsPerNode":"2",
[junit4:junit4]   2> 	  "createNodeSet":null,
[junit4:junit4]   2> 	  "name":"awholynewcollection_0",
[junit4:junit4]   2> 	  "replicationFactor":"2"}
[junit4:junit4]   2> 2116603 T4496 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica1 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:53347_fztlc
[junit4:junit4]   2> 2116604 T4496 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica2 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:53350_fztlc
[junit4:junit4]   2> 2116604 T4496 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica1 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:53353_fztlc
[junit4:junit4]   2> 2116605 T4496 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica2 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:53344_fztlc
[junit4:junit4]   2> 2116605 T4496 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica1 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:53340_fztlc
[junit4:junit4]   2> 2116607 T4496 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica2 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:53347_fztlc
[junit4:junit4]   2> 2116612 T4506 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/awholynewcollection_0_shard2_replica2
[junit4:junit4]   2> 2116614 T4506 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 2116611 T4490 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/awholynewcollection_0_shard3_replica1
[junit4:junit4]   2> 2116615 T4490 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 2116611 T4548 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/awholynewcollection_0_shard2_replica1
[junit4:junit4]   2> 2116618 T4506 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 2116618 T4506 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 2116608 T4522 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard1_replica1
[junit4:junit4]   2> 2116620 T4522 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 2116618 T4518 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard3_replica2
[junit4:junit4]   2> 2116621 T4518 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 2116618 T4548 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 2116614 T4533 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/awholynewcollection_0_shard1_replica2
[junit4:junit4]   2> 2116622 T4533 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 2116622 T4490 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 2116623 T4490 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 2116625 T4506 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 2116625 T4506 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 2116626 T4518 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 2116627 T4518 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 2116627 T4533 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 2116628 T4533 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 2116625 T4522 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 2116627 T4548 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 2116629 T4548 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 2116629 T4522 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 2116634 T4490 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 2116637 T4533 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 2116638 T4533 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 2116637 T4490 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 2116639 T4548 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 2116640 T4518 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 2116642 T4522 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 2116643 T4548 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 2116642 T4518 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 2116643 T4522 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 2116647 T4533 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 2116648 T4506 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 2116657 T4533 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/awholynewcollection_0_shard1_replica2/'
[junit4:junit4]   2> 2116668 T4548 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 2116691 T4506 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/awholynewcollection_0_shard2_replica2/'
[junit4:junit4]   2> 2116702 T4490 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 2116716 T4518 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 2116718 T4522 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 2116721 T4548 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/awholynewcollection_0_shard2_replica1/'
[junit4:junit4]   2> 2116724 T4490 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/awholynewcollection_0_shard3_replica1/'
[junit4:junit4]   2> 2116728 T4518 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard3_replica2/'
[junit4:junit4]   2> 2116732 T4522 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard1_replica1/'
[junit4:junit4]   2> 2116921 T4518 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 2116937 T4548 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 2116981 T4506 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 2117040 T4533 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 2117052 T4490 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 2117156 T4522 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 2117381 T4548 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2117388 T4548 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 2117394 T4548 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica1] Schema name=test
[junit4:junit4]   2> 2117487 T4490 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2117503 T4506 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2117510 T4518 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2117512 T4490 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 2117534 T4490 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica1] Schema name=test
[junit4:junit4]   2> 2117543 T4506 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 2117550 T4506 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica2] Schema name=test
[junit4:junit4]   2> 2117556 T4533 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2117560 T4518 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 2117561 T4533 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 2117592 T4533 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica2] Schema name=test
[junit4:junit4]   2> 2117607 T4522 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2117610 T4518 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica2] Schema name=test
[junit4:junit4]   2> 2117633 T4522 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 2117691 T4522 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica1] Schema name=test
[junit4:junit4]   2> 2117725 T4495 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2117736 T4495 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:53347_fztlc_nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53347_fztlc",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53347/fztlc"}
[junit4:junit4]   2> 2117770 T4495 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:53344_fztlc_nodes_used_collection_shard1_replica1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53344_fztlc",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53344/fztlc"}
[junit4:junit4]   2> 2117834 T4495 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:53353_fztlc_nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53353_fztlc",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53353/fztlc"}
[junit4:junit4]   2> 2117896 T4495 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:53340_fztlc_nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53340_fztlc",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53340/fztlc"}
[junit4:junit4]   2> 2118003 T4526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2118004 T4540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2118004 T4569 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2118009 T4499 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2118009 T4554 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2118010 T4571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2118010 T4512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2118017 T4494 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2118370 T4519 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:53353_fztlc_nodes_used_collection_shard2_replica1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4]   2> 2118371 T4519 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:53353_fztlc_nodes_used_collection_shard2_replica1&state=recovering&nodeName=127.0.0.1:53353_fztlc&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard2_replica2&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003 
[junit4:junit4]   2> 2118513 T4508 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:53340_fztlc_nodes_used_collection_shard1_replica2, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4]   2> 2118514 T4508 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:53340_fztlc_nodes_used_collection_shard1_replica2&state=recovering&nodeName=127.0.0.1:53340_fztlc&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard1_replica1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2004 
[junit4:junit4]   2>  C290_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:53353_fztlc, base_url=http://127.0.0.1:53353/fztlc}
[junit4:junit4]   2> 2120378 T4566 C290 P53353 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:53347/fztlc/nodes_used_collection_shard2_replica2/ core=nodes_used_collection_shard2_replica1 - recoveringAfterStartup=true
[junit4:junit4]   2> 2120379 T4566 C290 P53353 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:53353/fztlc START replicas=[http://127.0.0.1:53347/fztlc/nodes_used_collection_shard2_replica2/] nUpdates=100
[junit4:junit4]   2> 2120380 T4566 C290 P53353 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 2120381 T4566 C290 P53353 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 2120381 T4566 C290 P53353 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 2120382 T4566 C290 P53353 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 2120382 T4566 C290 P53353 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 2120384 T4566 C290 P53353 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:53347/fztlc/nodes_used_collection_shard2_replica2/. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 2120385 T4566 C290 P53353 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C292 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@6feceac1 url=http://127.0.0.1:53347/fztlc/nodes_used_collection_shard2_replica2 node=127.0.0.1:53347_fztlc C292_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:53347_fztlc, base_url=http://127.0.0.1:53347/fztlc, leader=true}
[junit4:junit4]   2> 2120432 T4520 C292 P53347 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 2120472 T4519 C292 P53347 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/fztlc path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=11 
[junit4:junit4]   2> 2120512 T4520 C292 P53347 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7b261801 lockFactory=org.apache.lucene.store.NativeFSLockFactory@d44f0e3),segFN=segments_1,generation=1}
[junit4:junit4]   2> 2120514 T4520 C292 P53347 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 2120515 T4520 C292 P53347 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7b261801 lockFactory=org.apache.lucene.store.NativeFSLockFactory@d44f0e3),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7b261801 lockFactory=org.apache.lucene.store.NativeFSLockFactory@d44f0e3),segFN=segments_2,generation=2}
[junit4:junit4]   2> 2120516 T4520 C292 P53347 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 2120517 T4520 C292 P53347 oass.SolrIndexSearcher.<init> Opening Searcher@7049be5a realtime
[junit4:junit4]   2> 2120518 T4520 C292 P53347 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 2120518 T4520 C292 P53347 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard2_replica2] webapp=/fztlc path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 87
[junit4:junit4]   2>  C291_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:53340_fztlc, base_url=http://127.0.0.1:53340/fztlc}
[junit4:junit4]   2> 2120522 T4567 C291 P53340 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:53344/fztlc/nodes_used_collection_shard1_replica1/ core=nodes_used_collection_shard1_replica2 - recoveringAfterStartup=true
[junit4:junit4]   2> 2120523 T4567 C291 P53340 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:53340/fztlc START replicas=[http://127.0.0.1:53344/fztlc/nodes_used_collection_shard1_replica1/] nUpdates=100
[junit4:junit4]   2> 2120523 T4566 C290 P53353 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 2120524 T4567 C291 P53340 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 2120582 T4567 C291 P53340 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 2120582 T4567 C291 P53340 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 2120583 T4567 C291 P53340 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 2120583 T4567 C291 P53340 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 2120583 T4567 C291 P53340 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:53344/fztlc/nodes_used_collection_shard1_replica1/. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 2120584 T4567 C291 P53340 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 2120584 T4566 C290 P53353 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> ASYNC  NEW_CORE C293 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@6607928b url=http://127.0.0.1:53344/fztlc/nodes_used_collection_shard1_replica1 node=127.0.0.1:53344_fztlc C293_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:53344_fztlc, base_url=http://127.0.0.1:53344/fztlc, leader=true}
[junit4:junit4]   2> 2120605 T4507 C293 P53344 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp=/fztlc path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=2 
[junit4:junit4]   2> 2120619 T4519 C292 P53347 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 2120621 T4519 C292 P53347 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/fztlc path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=11 
[junit4:junit4]   2> 2120625 T4566 C290 P53353 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 2120625 T4566 C290 P53353 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 2120626 T4566 C290 P53353 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 2120634 T4520 C292 P53347 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/fztlc path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 2120636 T4566 C290 P53353 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 2120638 T4566 C290 P53353 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/nodes_used_collection_shard2_replica1/data/index.20130519055048289
[junit4:junit4]   2> 2120639 T4566 C290 P53353 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@1550b53a lockFactory=org.apache.lucene.store.NativeFSLockFactory@27bdb441) fullCopy=false
[junit4:junit4]   2> 2120646 T4517 C292 P53347 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/fztlc path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1 
[junit4:junit4]   2> 2120648 T4503 C293 P53344 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 2120678 T4566 C290 P53353 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 2120678 T4503 C293 P53344 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@73b95fa8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@13c14009),segFN=segments_1,generation=1}
[junit4:junit4]   2> 2120680 T4503 C293 P53344 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 2120682 T4503 C293 P53344 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@73b95fa8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@13c14009),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@73b95fa8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@13c14009),segFN=segments_2,generation=2}
[junit4:junit4]   2> 2120683 T4503 C293 P53344 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 2120685 T4503 C293 P53344 oass.SolrIndexSearcher.<init> Opening Searcher@6f787658 realtime
[junit4:junit4]   2> 2120686 T4503 C293 P53344 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 2120687 T4503 C293 P53344 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard1_replica1] webapp=/fztlc path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 39
[junit4:junit4]   2> 2120689 T4566 C290 P53353 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 2120690 T4566 C290 P53353 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 2120696 T4566 C290 P53353 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@127a4bd5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@77d6a038),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@127a4bd5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@77d6a038),segFN=segments_2,generation=2}
[junit4:junit4]   2> 2120691 T4567 C291 P53340 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 2120708 T4567 C291 P53340 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 2120699 T4566 C290 P53353 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 2120749 T4566 C290 P53353 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 2120750 T4504 C293 P53344 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 2120750 T4504 C293 P53344 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp=/fztlc path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 2120750 T4566 C290 P53353 oass.SolrIndexSearcher.<init> Opening Searcher@2d193aa6 main
[junit4:junit4]   2> 2120751 T4567 C291 P53340 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 2120756 T4567 C291 P53340 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 2120759 T4567 C291 P53340 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 2120752 T4562 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@2d193aa6 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 2120762 T4566 C290 P53353 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/nodes_used_collection_shard2_replica1/data/index.20130519055048289 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/nodes_used_collection_shard2_replica1/data/index.20130519055048289;done=true>>]
[junit4:junit4]   2> 2120772 T4566 C290 P53353 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/nodes_used_collection_shard2_replica1/data/index.20130519055048289
[junit4:junit4]   2> 2120778 T4566 C290 P53353 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/nodes_used_collection_shard2_replica1/data/index.20130519055048289
[junit4:junit4]   2> 2120778 T4566 C290 P53353 oasc.RecoveryStrategy.replay No replay needed. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 2120779 T4566 C290 P53353 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 2120779 T4566 C290 P53353 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=active
[junit4:junit4]   2> 2120796 T4508 C293 P53344 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp=/fztlc path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=23 
[junit4:junit4]   2> 2120798 T4566 C290 P53353 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 2120804 T4567 C291 P53340 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 2120807 T4567 C291 P53340 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/nodes_used_collection_shard1_replica2/data/index.20130519055048457
[junit4:junit4]   2> 2120808 T4567 C291 P53340 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@172eb420 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c007a4b) fullCopy=false
[junit4:junit4]   2> 2120811 T4507 C293 P53344 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp=/fztlc path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 2120829 T4567 C291 P53340 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 2120831 T4567 C291 P53340 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 2120831 T4567 C291 P53340 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 2120883 T4567 C291 P53340 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6832daa lockFactory=org.apache.lucene.store.NativeFSLockFactory@1151a7b0),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6832daa lockFactory=org.apache.lucene.store.NativeFSLockFactory@1151a7b0),segFN=segments_2,generation=2}
[junit4:junit4]   2> 2120886 T4567 C291 P53340 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 2120887 T4567 C291 P53340 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 2120888 T4567 C291 P53340 oass.SolrIndexSearcher.<init> Opening Searcher@67dc617a main
[junit4:junit4]   2> 2120891 T4563 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@67dc617a main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 2120894 T4567 C291 P53340 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/nodes_used_collection_shard1_replica2/data/index.20130519055048457 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/nodes_used_collection_shard1_replica2/data/index.20130519055048457;done=true>>]
[junit4:junit4]   2> 2120895 T4567 C291 P53340 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/nodes_used_collection_shard1_replica2/data/index.20130519055048457
[junit4:junit4]   2> 2120895 T4567 C291 P53340 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/nodes_used_collection_shard1_replica2/data/index.20130519055048457
[junit4:junit4]   2> 2120896 T4567 C291 P53340 oasc.RecoveryStrategy.replay No replay needed. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 2120896 T4567 C291 P53340 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 2120897 T4567 C291 P53340 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=active
[junit4:junit4]   2> 2120911 T4567 C291 P53340 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 2121217 T4495 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2121221 T4495 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:53353_fztlc_nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53353_fztlc",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53353/fztlc"}
[junit4:junit4]   2> 2121244 T4495 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:53340_fztlc_nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53340_fztlc",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53340/fztlc"}
[junit4:junit4]   2> 2121284 T4526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2121285 T4494 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2121286 T4554 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2121287 T4540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2121287 T4571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2121288 T4499 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2121286 T4569 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2121288 T4512 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 2122551 T4490 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2122612 T4490 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2122716 T4490 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2122868 T4548 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2122887 T4490 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2122896 T4490 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2122899 T4548 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2122906 T4490 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2122909 T4490 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2122910 T4490 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2122910 T4490 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2122914 T4490 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2122914 T4490 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2122915 T4490 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2122916 T4490 oasc.SolrCore.<init> [awholynewcollection_0_shard3_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/awholynewcollection_0_shard3_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/awholynewcollection_0_shard3_replica1/data/
[junit4:junit4]   2> 2122917 T4490 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b973e81
[junit4:junit4]   2> 2122918 T4490 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2122919 T4490 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/awholynewcollection_0_shard3_replica1/data
[junit4:junit4]   2> 2122919 T4490 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/awholynewcollection_0_shard3_replica1/data/index/
[junit4:junit4]   2> 2122920 T4490 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard3_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/awholynewcollection_0_shard3_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2122921 T4490 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/awholynewcollection_0_shard3_replica1/data/index
[junit4:junit4]   2> 2122928 T4490 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@effe89d lockFactory=org.apache.lucene.store.NativeFSLockFactory@5ac61daa),segFN=segments_1,generation=1}
[junit4:junit4]   2> 2122928 T4490 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 2122933 T4490 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2122934 T4490 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2122937 T4490 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2122938 T4490 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2122940 T4490 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2122940 T4490 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2122941 T4490 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2122942 T4490 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2122943 T4490 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2122962 T4548 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2123055 T4490 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2123078 T4522 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2123092 T4506 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2123106 T4490 oass.SolrIndexSearcher.<init> Opening Searcher@8bb0679 main
[junit4:junit4]   2> 2123108 T4490 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1368960623854/awholynewcollection_0_shard3_replica1/data/tlog
[junit4:junit4]   2> 2123110 T4490 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2123110 T4490 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2123115 T4578 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard3_replica1] Registered new searcher Searcher@8bb0679 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2123153 T4490 oasc.ZkController.publish publishing core=awholynewcollection_0_shard3_replica1 state=down
[junit4:junit4]   2> 2123150 T4506 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2123161 T4522 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2123172 T4522 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2123176 T4506 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2123174 T4533 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2123208 T4533 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2123249 T4533 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2123257 T4548 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2123330 T4518 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2123338 T4522 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2123339 T4548 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2123367 T4518 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2123379 T4522 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2123400 T4548 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2123402 T4518 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2123404 T4522 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2123413 T4506 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2123418 T4548 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2123419 T4548 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2123422 T4548 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2123421 T4522 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2123425 T4522 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2123427 T4522 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2123437 T4548 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2123438 T4548 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2123443 T4548 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2123445 T4548 oasc.SolrCore.<init> [awholynewcollection_0_shard2_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/awholynewcollection_0_shard2_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/awholynewcollection_0_shard2_replica1/data/
[junit4:junit4]   2> 2123446 T4548 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b973e81
[junit4:junit4]   2> 2123447 T4548 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2123442 T4522 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2123449 T4522 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2123451 T4522 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2123454 T4522 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard1_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard1_replica1/data/
[junit4:junit4]   2> 2123459 T4522 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b973e81
[junit4:junit4]   2> 2123449 T4506 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2123465 T4522 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2123465 T4548 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/awholynewcollection_0_shard2_replica1/data
[junit4:junit4]   2> 2123475 T4522 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard1_replica1/data
[junit4:junit4]   2> 2123478 T4548 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/awholynewcollection_0_shard2_replica1/data/index/
[junit4:junit4]   2> 2123479 T4548 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard2_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/awholynewcollection_0_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2123478 T4533 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2123483 T4522 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard1_replica1/data/index/
[junit4:junit4]   2> 2123486 T4522 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard1_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2123488 T4548 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/awholynewcollection_0_shard2_replica1/data/index
[junit4:junit4]   2> 2123497 T4506 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2123498 T4533 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2123509 T4522 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard1_replica1/data/index
[junit4:junit4]   2> 2123519 T4533 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2123514 T4506 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2123535 T4506 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2123536 T4506 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2123535 T4522 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@135938cd lockFactory=org.apache.lucene.store.NativeFSLockFactory@78754d2b),segFN=segments_1,generation=1}
[junit4:junit4]   2> 2123545 T4522 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 2123534 T4548 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1aa25fee lockFactory=org.apache.lucene.store.NativeFSLockFactory@61f4900e),segFN=segments_1,generation=1}
[junit4:junit4]   2> 2123552 T4548 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 2123556 T4522 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2123557 T4522 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2123557 T4548 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2123558 T4522 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2123559 T4522 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2123560 T4522 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2123564 T4522 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2123565 T4522 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2123550 T4506 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2123567 T4506 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2123568 T4506 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2123569 T4506 oasc.SolrCore.<init> [awholynewcollection_0_shard2_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/awholynewcollection_0_shard2_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/awholynewcollection_0_shard2_replica2/data/
[junit4:junit4]   2> 2123570 T4506 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b973e81
[junit4:junit4]   2> 2123571 T4518 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2123571 T4506 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2123536 T4533 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2123573 T4533 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2123574 T4506 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/awholynewcollection_0_shard2_replica2/data
[junit4:junit4]   2> 2123567 T4522 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2123577 T4522 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2123558 T4548 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2123594 T4548 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2123596 T4548 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2123597 T4548 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2123598 T4548 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2123600 T4548 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2123575 T4506 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/awholynewcollection_0_shard2_replica2/data/index/
[junit4:junit4]   2> 2123602 T4506 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard2_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/awholynewcollection_0_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2123574 T4533 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2123613 T4548 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2123615 T4548 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2123623 T4506 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/awholynewcollection_0_shard2_replica2/data/index
[junit4:junit4]   2> 2123638 T4518 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2123640 T4548 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2123648 T4533 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2123650 T4533 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2123650 T4533 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2123652 T4533 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/awholynewcollection_0_shard1_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/awholynewcollection_0_shard1_replica2/data/
[junit4:junit4]   2> 2123653 T4533 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b973e81
[junit4:junit4]   2> 2123654 T4533 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2123658 T4506 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f6935d4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@513ec130),segFN=segments_1,generation=1}
[junit4:junit4]   2> 2123659 T4506 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 2123660 T4548 oass.SolrIndexSearcher.<init> Opening Searcher@29da119a main
[junit4:junit4]   2> 2123661 T4548 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1368960632997/awholynewcollection_0_shard2_replica1/data/tlog
[junit4:junit4]   2> 2123663 T4548 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2123664 T4548 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2123669 T4579 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard2_replica1] Registered new searcher Searcher@29da119a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2123670 T4533 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/awholynewcollection_0_shard1_replica2/data
[junit4:junit4]   2> 2123673 T4548 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica1 state=down
[junit4:junit4]   2> 2123684 T4533 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/awholynewcollection_0_shard1_replica2/data/index/
[junit4:junit4]   2> 2123685 T4533 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard1_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/awholynewcollection_0_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2123689 T4506 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2123690 T4506 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2123700 T4506 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2123701 T4506 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2123703 T4506 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2123703 T4506 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2123704 T4506 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2123705 T4506 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2123706 T4506 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2123659 T4522 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2123697 T4533 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/awholynewcollection_0_shard1_replica2/data/index
[junit4:junit4]   2> 2123736 T4518 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2123738 T4522 oass.SolrIndexSearcher.<init> Opening Searcher@6256fb83 main
[junit4:junit4]   2> 2123740 T4522 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard1_replica1/data/tlog
[junit4:junit4]   2> 2123743 T4522 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2123744 T4522 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2123746 T4506 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2123753 T4580 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard1_replica1] Registered new searcher Searcher@6256fb83 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2123756 T4522 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica1 state=down
[junit4:junit4]   2> 2123753 T4533 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a4107e lockFactory=org.apache.lucene.store.NativeFSLockFactory@437794d),segFN=segments_1,generation=1}
[junit4:junit4]   2> 2123758 T4533 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 2123765 T4533 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2123766 T4533 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2123767 T4533 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2123768 T4533 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2123770 T4533 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2123772 T4533 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2123773 T4533 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2123775 T4533 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2123776 T4533 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2123792 T4506 oass.SolrIndexSearcher.<init> Opening Searcher@5b473264 main
[junit4:junit4]   2> 2123794 T4506 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1368960625511/awholynewcollection_0_shard2_replica2/data/tlog
[junit4:junit4]   2> 2123796 T4506 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2123797 T4506 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2123803 T4518 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2123806 T4518 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2123812 T4518 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2123812 T4533 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2123827 T4581 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard2_replica2] Registered new searcher Searcher@5b473264 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2123835 T4506 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica2 state=down
[junit4:junit4]   2> 2123842 T4518 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 2123844 T4518 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 2123844 T4518 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2123846 T4518 oasc.SolrCore.<init> [awholynewcollection_0_shard3_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard3_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard3_replica2/data/
[junit4:junit4]   2> 2123846 T4518 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b973e81
[junit4:junit4]   2> 2123847 T4518 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 2123843 T4533 oass.SolrIndexSearcher.<init> Opening Searcher@6ea7eb5f main
[junit4:junit4]   2> 2123849 T4533 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1368960630292/awholynewcollection_0_shard1_replica2/data/tlog
[junit4:junit4]   2> 2123850 T4518 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard3_replica2/data
[junit4:junit4]   2> 2123852 T4533 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2123853 T4533 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2123858 T4582 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard1_replica2] Registered new searcher Searcher@6ea7eb5f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2123859 T4518 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard3_replica2/data/index/
[junit4:junit4]   2> 2123860 T4533 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica2 state=down
[junit4:junit4]   2> 2123860 T4518 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard3_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard3_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2123868 T4518 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard3_replica2/data/index
[junit4:junit4]   2> 2123876 T4518 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@227f69fe lockFactory=org.apache.lucene.store.NativeFSLockFactory@77da908b),segFN=segments_1,generation=1}
[junit4:junit4]   2> 2123877 T4518 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 2123882 T4518 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2123883 T4518 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2123886 T4518 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2123888 T4518 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2123890 T4518 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2123891 T4518 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2123891 T4518 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2123893 T4518 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2123894 T4518 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2123921 T4518 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2123944 T4518 oass.SolrIndexSearcher.<init> Opening Searcher@643a8514 main
[junit4:junit4]   2> 2123945 T4518 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1368960627842/awholynewcollection_0_shard3_replica2/data/tlog
[junit4:junit4]   2> 2123947 T4518 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2123948 T4518 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2123956 T4583 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard3_replica2] Registered new searcher Searcher@643a8514 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2123960 T4518 oasc.ZkController.publish publishing core=awholynewcollection_0_shard3_replica2 state=down
[junit4:junit4]   2> 2124372 T4495 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2124374 T4495 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard3",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"awholynewcollection_0_shard3_replica1",
[junit4:junit4]   2> 	  "collection":"awholynewcollection_0",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:53340_fztlc",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:53340/fztlc"}
[junit4:junit4]   2> 2124375 T4495 oasc.Overseer$ClusterStateUpdater.createCollection Create collection awholynewcollection_0 with numShards 3
[junit4:junit4]   2> 2124385 T4495 oasc.Overseer$ClusterStateUpdater.upda

[...truncated too long message...]

itTest-1368962197378 FAILED !!!!!
[junit4:junit4]   2> 3718835 T7117 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4]   2> 3718835 T7117 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:54127 54127
[junit4:junit4]   1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=C1BF6594B1D20D53 -Dtests.slow=true -Dtests.locale=sl -Dtests.timezone=Israel -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   49.1s | ShardSplitTest.testDistribSearch <<<
[junit4:junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:54134/xs_z/s returned non ok status:500, message:Server Error
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([C1BF6594B1D20D53:4059EB8CC68D6D6F]:0)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4]    > 	at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:208)
[junit4:junit4]    > 	at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:133)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 3718847 T7117 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 49142 T7116 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4]   2> 3719699 T7208 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 3719700 T7208 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=433), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=433)), sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=sl, timezone=Israel
[junit4:junit4]   2> NOTE: Mac OS X 10.8.3 x86_64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=2,threads=5,free=138729224,total=371720192
[junit4:junit4]   2> NOTE: All tests run in this JVM: [StandardRequestHandlerTest, SignatureUpdateProcessorFactoryTest, TestMergePolicyConfig, TestPseudoReturnFields, TestDefaultSimilarityFactory, TestCSVResponseWriter, HardAutoCommitTest, TestLMDirichletSimilarityFactory, ShardRoutingCustomTest, DistributedQueryElevationComponentTest, TestSolrQueryParser, TestManagedSchema, ShowFileRequestHandlerTest, CircularListTest, TestDistributedGrouping, TestSort, TestDocSet, LoggingHandlerTest, SolrCoreCheckLockOnStartupTest, JSONWriterTest, TestStressLucene, TestMultiCoreConfBootstrap, ResourceLoaderTest, TestRecovery, PreAnalyzedUpdateProcessorTest, DebugComponentTest, LegacyHTMLStripCharFilterTest, TestCoreContainer, DirectUpdateHandlerTest, ClusterStateTest, TestManagedSchemaFieldResource, DisMaxRequestHandlerTest, RecoveryZkTest, SolrRequestParserTest, TestBinaryField, QueryEqualityTest, IndexSchemaTest, AnalysisAfterCoreReloadTest, TestSolrDeletionPolicy2, TestStressReorder, TestLuceneMatchVersion, DirectSolrSpellCheckerTest, TestLazyCores, TestRealTimeGet, SolrPluginUtilsTest, SyncSliceTest, TestIBSimilarityFactory, DOMUtilTest, TermsComponentTest, TestPartialUpdateDeduplication, CSVRequestHandlerTest, FullSolrCloudDistribCmdsTest, ResponseLogComponentTest, DirectUpdateHandlerOptimizeTest, TestWriterPerf, TestReload, BasicDistributedZkTest, FieldAnalysisRequestHandlerTest, BadComponentTest, StatelessScriptUpdateProcessorFactoryTest, DistanceFunctionTest, TestFieldTypeResource, TestReversedWildcardFilterFactory, ShardRoutingTest, CoreContainerCoreInitFailuresTest, XmlUpdateRequestHandlerTest, AliasIntegrationTest, SchemaVersionSpecificBehaviorTest, SolrTestCaseJ4Test, TestBM25SimilarityFactory, CacheHeaderTest, FieldMutatingUpdateProcessorTest, TestDistributedSearch, TestExtendedDismaxParser, TestSolrQueryParserResource, SolrIndexSplitterTest, TestQuerySenderNoQuery, RAMDirectoryFactoryTest, ZkCLITest, CurrencyFieldOpenExchangeTest, TestAddFieldRealTimeGet, DocumentBuilderTest, TestCharFilters, UUIDFieldTest, JsonLoaderTest, TestCopyFieldCollectionResource, SuggesterFSTTest, IndexSchemaRuntimeFieldTest, DocumentAnalysisRequestHandlerTest, ChaosMonkeySafeLeaderTest, TestSolr4Spatial, URLClassifyProcessorTest, AutoCommitTest, TestSchemaVersionResource, TestRandomFaceting, TestNumberUtils, TestDynamicFieldCollectionResource, XsltUpdateRequestHandlerTest, PolyFieldTest, PingRequestHandlerTest, TestDocumentBuilder, TestElisionMultitermQuery, OverseerCollectionProcessorTest, LukeRequestHandlerTest, TestSolrXml, TestLFUCache, TestHashPartitioner, TestPhraseSuggestions, DateFieldTest, TestUtils, TestZkChroot, SolrIndexConfigTest, TestDFRSimilarityFactory, TermVectorComponentDistributedTest, OpenExchangeRatesOrgProviderTest, BadIndexSchemaTest, TestShardHandlerFactory, TestFunctionQuery, SpellingQueryConverterTest, TestSolrDeletionPolicy1, TestSearchPerf, WordBreakSolrSpellCheckerTest, DocValuesTest, TestPluginEnable, TestFieldResource, TestReplicationHandler, TestDefaultSearchFieldResource, TestCloudManagedSchemaAddField, TestFieldTypeCollectionResource, ExternalFileFieldSortTest, DateMathParserTest, TestRTGBase, FileBasedSpellCheckerTest, SpellCheckComponentTest, OutputWriterTest, PreAnalyzedFieldTest, QueryParsingTest, IndexReaderFactoryTest, TestFaceting, MultiTermTest, TestMaxScoreQueryParser, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, OpenCloseCoreStressTest, OverseerTest, LeaderElectionIntegrationTest, BasicZkTest, ClusterStateUpdateTest, LeaderElectionTest, ZkSolrClientTest, TestRandomDVFaceting, SliceStateUpdateTest, DistributedSpellCheckComponentTest, ZkControllerTest, TestJoin, TestStressVersions, DistributedTermsComponentTest, TestRangeQuery, SimpleFacetsTest, SolrCoreTest, StatsComponentTest, TestGroupingSearch, SolrCmdDistributorTest, QueryElevationComponentTest, PeerSyncTest, ConvertedLegacyTest, TestFiltering, BasicFunctionalityTest, TestBadConfig, TestIndexSearcher, HighlighterTest, SoftAutoCommitTest, CurrencyFieldXmlFileTest, SimplePostToolTest, TestCoreDiscovery, SpellCheckCollatorTest, CoreAdminHandlerTest, TestFoldingMultitermQuery, SuggesterTSTTest, SuggesterTest, TestTrie, SpatialFilterTest, SuggesterWFSTTest, TestCSVLoader, NoCacheHeaderTest, TestUpdate, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, DefaultValueUpdateProcessorTest, SortByFunctionTest, TestRemoteStreaming, DocValuesMultiTest, SolrInfoMBeanTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, TestQueryUtils, TestQueryTypes, PrimitiveFieldTypeTest, TestOmitPositions, RequestHandlersTest, TestValueSourceCache, PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, TestIndexingPerformance, MoreLikeThisHandlerTest, RequiredFieldsTest, TestArbitraryIndexDir, FastVectorHighlighterTest, RegexBoostProcessorTest, TestJmxIntegration, TestCollationField, ReturnFieldsTest, UpdateRequestProcessorFactoryTest, MBeansHandlerTest, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, BinaryUpdateRequestHandlerTest, TestPHPSerializedResponseWriter, TestComponentsName, SearchHandlerTest, TestBinaryResponseWriter, HighlighterConfigTest, SOLR749Test, TestPropInject, TestQuerySenderListener, TestPropInjectDefaults, UpdateParamsTest, AlternateDirectoryTest, TestSolrIndexConfig, CopyFieldTest, TestStressRecovery, SampleTest, NumericFieldsTest, MinimalSchemaTest, TestConfig, TestFuzzyAnalyzedSuggestions, TestSolrCoreProperties, TestPostingsSolrHighlighter, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, SpellPossibilityIteratorTest, TestCodecSupport, SynonymTokenizerTest, TestXIncludeConfig, EchoParamsTest, TestSweetSpotSimilarityFactory, TestPerFieldSimilarity, TestLMJelinekMercerSimilarityFactory, TestFastWriter, TestJmxMonitoredMap, TimeZoneUtilsTest, TestFastOutputStream, ScriptEngineTest, PluginInfoTest, TestFastLRUCache, ChaosMonkeyNothingIsSafeTest, TestSolrXMLSerializer, TestSystemIdResolver, PrimUtilsTest, TestSuggestSpellingConverter, TestSolrJ, TestLRUCache, ZkNodePropsTest, SliceStateTest, SystemInfoHandlerTest, FileUtilsTest, TestCollationKeyRangeQueries, ChaosMonkeyShardSplitTest, ShardSplitTest]
[junit4:junit4] Completed in 50.20s, 1 test, 1 error <<< FAILURES!

[...truncated 49 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/build.xml:383: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/build.xml:363: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/common-build.xml:887: There were test failures: 296 suites, 1234 tests, 1 error, 1 failure, 18 ignored (6 assumptions)

Total time: 118 minutes 30 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-4.x-MacOSX (64bit/jdk1.7.0) - Build # 470 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-MacOSX/470/
Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
There are still nodes recoverying - waited for 230 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 230 seconds
	at __randomizedtesting.SeedInfo.seed([9DAA94E6B71ECFAB:1C4C1AFEC041AF97]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:173)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:512)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 9613 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4:junit4]   2> 1963809 T3713 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_/
[junit4:junit4]   2> 1963815 T3713 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./solrtest-CollectionsAPIDistributedZkTest-1369017378308
[junit4:junit4]   2> 1963818 T3713 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 1963819 T3714 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 1963921 T3713 oasc.ZkTestServer.run start zk server on port:52952
[junit4:junit4]   2> 1963923 T3713 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1963932 T3720 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63f80dad name:ZooKeeperConnection Watcher:127.0.0.1:52952 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1963932 T3713 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1963933 T3713 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 1963944 T3713 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1963950 T3722 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@988c0a6 name:ZooKeeperConnection Watcher:127.0.0.1:52952/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1963951 T3713 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1963951 T3713 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 1963960 T3713 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 1963969 T3713 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 1963977 T3713 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 1963986 T3713 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 1963987 T3713 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 1964001 T3713 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 1964002 T3713 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 1964010 T3713 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 1964012 T3713 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 1964020 T3713 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 1964021 T3713 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 1964028 T3713 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 1964029 T3713 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 1964037 T3713 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 1964038 T3713 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 1964049 T3713 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 1964050 T3713 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 1964059 T3713 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 1964059 T3713 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 1964067 T3713 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 1964069 T3713 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 1964586 T3713 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 1964608 T3713 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52955
[junit4:junit4]   2> 1964609 T3713 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1964610 T3713 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1964611 T3713 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369017378570
[junit4:junit4]   2> 1964611 T3713 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369017378570/solr.xml
[junit4:junit4]   2> 1964612 T3713 oasc.CoreContainer.<init> New CoreContainer 1238403298
[junit4:junit4]   2> 1964613 T3713 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369017378570/'
[junit4:junit4]   2> 1964613 T3713 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369017378570/'
[junit4:junit4]   2> 1964770 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369017378570
[junit4:junit4]   2> 1964772 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369017378570/collection1
[junit4:junit4]   2> 1964774 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369017378570/collection1/conf
[junit4:junit4]   2> 1964947 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369017378570/collection1/conf/xslt
[junit4:junit4]   2> 1964952 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369017378570/collection1/lib
[junit4:junit4]   2> 1964954 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369017378570/collection1/lib/classes
[junit4:junit4]   2> 1964957 T3713 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369017378570/conf/core.properties, adding to cores
[junit4:junit4]   2> 1964978 T3713 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1964979 T3713 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1964980 T3713 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1964981 T3713 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1964982 T3713 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1964983 T3713 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1964984 T3713 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1964985 T3713 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1964986 T3713 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1964988 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1965019 T3713 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 1965020 T3713 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52952/solr
[junit4:junit4]   2> 1965020 T3713 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1965022 T3713 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1965028 T3733 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@534368f1 name:ZooKeeperConnection Watcher:127.0.0.1:52952 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1965029 T3713 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1965034 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 1965063 T3713 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1965068 T3735 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13f5e6ae name:ZooKeeperConnection Watcher:127.0.0.1:52952/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1965069 T3713 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1965072 T3713 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4]   2> 1965082 T3713 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4]   2> 1965092 T3713 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 1965098 T3713 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52955__
[junit4:junit4]   2> 1965100 T3713 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52955__
[junit4:junit4]   2> 1965110 T3713 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 1965125 T3713 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 1965134 T3713 oasc.Overseer.start Overseer (id=89719922905448451-127.0.0.1:52955__-n_0000000000) starting
[junit4:junit4]   2> 1965147 T3713 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4]   2> 1965160 T3736 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 1965164 T3737 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 1965167 T3713 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 1965834 T3713 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4]   2> 1965873 T3713 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1965885 T3713 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4]   2> 1965885 T3713 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1965886 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1965907 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 1965910 T3713 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1965915 T3740 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1150ddbe name:ZooKeeperConnection Watcher:127.0.0.1:52952/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1965916 T3713 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1965918 T3713 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1965925 T3713 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 1966412 T3713 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 1966418 T3713 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52959
[junit4:junit4]   2> 1966419 T3713 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1966420 T3713 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1966420 T3713 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416
[junit4:junit4]   2> 1966421 T3713 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/solr.xml
[junit4:junit4]   2> 1966422 T3713 oasc.CoreContainer.<init> New CoreContainer 1283045594
[junit4:junit4]   2> 1966422 T3713 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/'
[junit4:junit4]   2> 1966423 T3713 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/'
[junit4:junit4]   2> 1966581 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416
[junit4:junit4]   2> 1966584 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/collection1
[junit4:junit4]   2> 1966586 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/collection1/conf
[junit4:junit4]   2> 1966751 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/collection1/conf/xslt
[junit4:junit4]   2> 1966756 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/collection1/lib
[junit4:junit4]   2> 1966757 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/collection1/lib/classes
[junit4:junit4]   2> 1966761 T3713 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/conf/core.properties, adding to cores
[junit4:junit4]   2> 1966782 T3713 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1966783 T3713 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1966783 T3713 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1966784 T3713 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1966784 T3713 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1966785 T3713 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1966785 T3713 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1966785 T3713 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1966786 T3713 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1966786 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1966812 T3713 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 1966813 T3713 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52952/solr
[junit4:junit4]   2> 1966813 T3713 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1966815 T3713 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1966822 T3751 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@756cdff name:ZooKeeperConnection Watcher:127.0.0.1:52952 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1966823 T3713 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1966828 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 1966855 T3713 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1966861 T3753 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@516e0b6f name:ZooKeeperConnection Watcher:127.0.0.1:52952/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1966862 T3713 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1966876 T3713 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1967887 T3713 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52959__
[junit4:junit4]   2> 1967891 T3713 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52959__
[junit4:junit4]   2> 1967899 T3753 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 1967899 T3740 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 1967900 T3735 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 1967912 T3713 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4]   2> 1967913 T3713 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1967913 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1968444 T3713 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 1968449 T3713 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52962
[junit4:junit4]   2> 1968451 T3713 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1968452 T3713 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1968452 T3713 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426
[junit4:junit4]   2> 1968453 T3713 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/solr.xml
[junit4:junit4]   2> 1968453 T3713 oasc.CoreContainer.<init> New CoreContainer 442396659
[junit4:junit4]   2> 1968454 T3713 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/'
[junit4:junit4]   2> 1968455 T3713 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/'
[junit4:junit4]   2> 1968609 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426
[junit4:junit4]   2> 1968611 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/collection1
[junit4:junit4]   2> 1968613 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/collection1/conf
[junit4:junit4]   2> 1968778 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/collection1/conf/xslt
[junit4:junit4]   2> 1968784 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/collection1/lib
[junit4:junit4]   2> 1968787 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/collection1/lib/classes
[junit4:junit4]   2> 1968790 T3713 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/conf/core.properties, adding to cores
[junit4:junit4]   2> 1968808 T3713 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1968809 T3713 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1968809 T3713 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1968810 T3713 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1968810 T3713 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1968811 T3713 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1968811 T3713 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1968812 T3713 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1968813 T3713 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1968813 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1968843 T3713 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 1968844 T3713 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52952/solr
[junit4:junit4]   2> 1968846 T3713 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1968847 T3713 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1968853 T3765 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56f1fb2b name:ZooKeeperConnection Watcher:127.0.0.1:52952 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1968855 T3713 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1968860 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 1968884 T3713 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1968890 T3767 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@133a230 name:ZooKeeperConnection Watcher:127.0.0.1:52952/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1968891 T3713 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1968904 T3713 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1969916 T3713 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52962__
[junit4:junit4]   2> 1969919 T3713 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52962__
[junit4:junit4]   2> 1969927 T3753 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 1969927 T3735 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 1969928 T3767 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 1969928 T3740 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 1969939 T3713 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4]   2> 1969940 T3713 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1969941 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1970486 T3713 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 1970493 T3713 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52965
[junit4:junit4]   2> 1970495 T3713 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1970497 T3713 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1970497 T3713 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452
[junit4:junit4]   2> 1970498 T3713 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/solr.xml
[junit4:junit4]   2> 1970499 T3713 oasc.CoreContainer.<init> New CoreContainer 129537282
[junit4:junit4]   2> 1970501 T3713 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/'
[junit4:junit4]   2> 1970502 T3713 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/'
[junit4:junit4]   2> 1970670 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452
[junit4:junit4]   2> 1970672 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/collection1
[junit4:junit4]   2> 1970674 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/collection1/conf
[junit4:junit4]   2> 1970840 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/collection1/conf/xslt
[junit4:junit4]   2> 1970845 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/collection1/lib
[junit4:junit4]   2> 1970846 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/collection1/lib/classes
[junit4:junit4]   2> 1970849 T3713 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/conf/core.properties, adding to cores
[junit4:junit4]   2> 1970868 T3713 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1970868 T3713 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1970869 T3713 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1970870 T3713 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1970870 T3713 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1970871 T3713 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1970871 T3713 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1970872 T3713 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1970872 T3713 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1970873 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1970915 T3713 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 1970917 T3713 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52952/solr
[junit4:junit4]   2> 1970919 T3713 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1970921 T3713 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1970926 T3779 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@eb4215f name:ZooKeeperConnection Watcher:127.0.0.1:52952 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1970928 T3713 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1970936 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 1970963 T3713 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1970968 T3781 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@49487f2e name:ZooKeeperConnection Watcher:127.0.0.1:52952/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1970968 T3713 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1970980 T3713 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1971989 T3713 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52965__
[junit4:junit4]   2> 1971992 T3713 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52965__
[junit4:junit4]   2> 1971999 T3767 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 1972000 T3735 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 1972000 T3753 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 1972001 T3781 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 1972001 T3740 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 1972013 T3713 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4]   2> 1972014 T3713 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1972015 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1972504 T3713 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4]   2> 1972510 T3713 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52968
[junit4:junit4]   2> 1972512 T3713 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1972513 T3713 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1972514 T3713 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525
[junit4:junit4]   2> 1972514 T3713 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/solr.xml
[junit4:junit4]   2> 1972515 T3713 oasc.CoreContainer.<init> New CoreContainer 182280022
[junit4:junit4]   2> 1972516 T3713 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/'
[junit4:junit4]   2> 1972516 T3713 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/'
[junit4:junit4]   2> 1972672 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525
[junit4:junit4]   2> 1972674 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/collection1
[junit4:junit4]   2> 1972675 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/collection1/conf
[junit4:junit4]   2> 1972846 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/collection1/conf/xslt
[junit4:junit4]   2> 1972850 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/collection1/lib
[junit4:junit4]   2> 1972852 T3713 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/collection1/lib/classes
[junit4:junit4]   2> 1972855 T3713 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/conf/core.properties, adding to cores
[junit4:junit4]   2> 1972872 T3713 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1972873 T3713 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1972874 T3713 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1972874 T3713 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1972875 T3713 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1972876 T3713 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1972876 T3713 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1972877 T3713 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1972877 T3713 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1972878 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1972903 T3713 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4]   2> 1972904 T3713 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52952/solr
[junit4:junit4]   2> 1972905 T3713 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1972907 T3713 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1972915 T3793 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d42729e name:ZooKeeperConnection Watcher:127.0.0.1:52952 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1972916 T3713 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1972921 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 1972946 T3713 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1972952 T3795 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ae5dde3 name:ZooKeeperConnection Watcher:127.0.0.1:52952/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1972953 T3713 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1972966 T3713 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1973978 T3713 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52968__
[junit4:junit4]   2> 1973981 T3713 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52968__
[junit4:junit4]   2> 1973991 T3735 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1973992 T3753 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1973993 T3740 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1973994 T3781 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1973994 T3767 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1973994 T3795 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 1974007 T3713 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4]   2> 1974008 T3713 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1974009 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1974028 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1974062 T3745 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2
[junit4:junit4]   2> 1974067 T3735 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 1974073 T3737 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4:junit4]   2> 	  "operation":"createcollection",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "maxShardsPerNode":null,
[junit4:junit4]   2> 	  "createNodeSet":null,
[junit4:junit4]   2> 	  "name":"nodes_used_collection",
[junit4:junit4]   2> 	  "replicationFactor":"2"}
[junit4:junit4]   2> 1974073 T3737 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica1 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:52965__
[junit4:junit4]   2> 1974075 T3737 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica2 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:52959__
[junit4:junit4]   2> 1974076 T3737 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica1 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:52962__
[junit4:junit4]   2> 1974078 T3737 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica2 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:52968__
[junit4:junit4]   2> 1974089 T3773 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/nodes_used_collection_shard1_replica1
[junit4:junit4]   2> 1974095 T3773 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 1974093 T3759 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 1974098 T3759 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 1974099 T3773 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 1974099 T3773 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1974100 T3759 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 1974101 T3759 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1974093 T3746 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1974103 T3746 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 1974105 T3773 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1974107 T3773 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 1974108 T3746 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 1974109 T3746 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1974096 T3789 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1974111 T3789 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4]   2> 1974112 T3759 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1974114 T3759 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 1974118 T3789 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4]   2> 1974120 T3789 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1974122 T3746 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1974124 T3746 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 1974130 T3789 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1974131 T3789 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4]   2> 1974131 T3773 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 1974135 T3759 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 1974143 T3746 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 1974146 T3773 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/nodes_used_collection_shard1_replica1/'
[junit4:junit4]   2> 1974147 T3789 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4]   2> 1974156 T3746 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/nodes_used_collection_shard1_replica2/'
[junit4:junit4]   2> 1974161 T3789 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/nodes_used_collection_shard2_replica2/'
[junit4:junit4]   2> 1974166 T3759 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/nodes_used_collection_shard2_replica1/'
[junit4:junit4]   2> 1974423 T3789 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 1974425 T3759 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 1974435 T3746 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 1974437 T3773 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 1974717 T3789 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1974731 T3773 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1974733 T3789 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1974748 T3773 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1974755 T3773 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica1] Schema name=test
[junit4:junit4]   2> 1974762 T3746 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1974774 T3759 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1974780 T3746 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1974781 T3759 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1974781 T3789 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica2] Schema name=test
[junit4:junit4]   2> 1974791 T3759 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica1] Schema name=test
[junit4:junit4]   2> 1974801 T3746 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica2] Schema name=test
[junit4:junit4]   2> 1976992 T3789 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1977029 T3789 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1977038 T3789 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1977106 T3759 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1977128 T3759 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1977132 T3773 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1977147 T3773 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1977152 T3759 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1977153 T3773 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1977174 T3789 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1977186 T3746 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1977196 T3789 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1977211 T3746 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1977222 T3746 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1977223 T3789 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1977235 T3789 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1977237 T3789 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1977237 T3789 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1977245 T3789 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1977246 T3789 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1977245 T3759 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1977247 T3789 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1977249 T3789 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/nodes_used_collection_shard2_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/nodes_used_collection_shard2_replica2/data/
[junit4:junit4]   2> 1977250 T3789 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2c17277b
[junit4:junit4]   2> 1977251 T3789 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1977255 T3789 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/nodes_used_collection_shard2_replica2/data
[junit4:junit4]   2> 1977256 T3789 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/nodes_used_collection_shard2_replica2/data/index/
[junit4:junit4]   2> 1977257 T3789 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/nodes_used_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1977258 T3789 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/nodes_used_collection_shard2_replica2/data/index
[junit4:junit4]   2> 1977263 T3759 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1977269 T3789 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7feb4c87 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3b079af9),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1977270 T3789 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1977272 T3773 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1977278 T3789 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1977280 T3789 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1977283 T3789 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1977285 T3789 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1977286 T3789 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1977286 T3789 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1977287 T3789 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1977289 T3789 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1977290 T3789 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1977307 T3759 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1977308 T3773 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1977314 T3759 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1977323 T3759 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1977324 T3759 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1977331 T3789 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1977334 T3773 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1977336 T3789 oass.SolrIndexSearcher.<init> Opening Searcher@6bbb5a83 main
[junit4:junit4]   2> 1977337 T3789 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/nodes_used_collection_shard2_replica2/data/tlog
[junit4:junit4]   2> 1977338 T3759 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1977339 T3759 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1977339 T3759 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1977340 T3759 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/nodes_used_collection_shard2_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/nodes_used_collection_shard2_replica1/data/
[junit4:junit4]   2> 1977340 T3789 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1977341 T3789 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1977342 T3746 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1977341 T3759 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2c17277b
[junit4:junit4]   2> 1977343 T3759 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1977341 T3773 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1977346 T3773 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1977347 T3773 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1977346 T3759 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/nodes_used_collection_shard2_replica1/data
[junit4:junit4]   2> 1977348 T3759 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/nodes_used_collection_shard2_replica1/data/index/
[junit4:junit4]   2> 1977348 T3759 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/nodes_used_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1977350 T3759 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/nodes_used_collection_shard2_replica1/data/index
[junit4:junit4]   2> 1977357 T3759 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@20d1729f lockFactory=org.apache.lucene.store.NativeFSLockFactory@33022a81),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1977357 T3759 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1977357 T3746 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1977359 T3801 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@6bbb5a83 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1977362 T3789 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=down
[junit4:junit4]   2> 1977360 T3773 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1977363 T3773 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1977363 T3773 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1977364 T3773 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/nodes_used_collection_shard1_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/nodes_used_collection_shard1_replica1/data/
[junit4:junit4]   2> 1977365 T3773 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2c17277b
[junit4:junit4]   2> 1977368 T3773 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1977369 T3773 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/nodes_used_collection_shard1_replica1/data
[junit4:junit4]   2> 1977369 T3773 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/nodes_used_collection_shard1_replica1/data/index/
[junit4:junit4]   2> 1977370 T3773 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/nodes_used_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1977370 T3773 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/nodes_used_collection_shard1_replica1/data/index
[junit4:junit4]   2> 1977377 T3773 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@41a22cce lockFactory=org.apache.lucene.store.NativeFSLockFactory@116c32b6),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1977377 T3773 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1977368 T3759 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1977379 T3759 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1977380 T3759 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1977381 T3759 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1977382 T3773 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1977383 T3773 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1977385 T3759 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1977386 T3759 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1977386 T3759 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1977387 T3759 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1977388 T3759 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1977392 T3773 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1977393 T3773 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1977394 T3773 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1977394 T3773 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1977395 T3773 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1977396 T3773 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1977397 T3773 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1977399 T3746 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1977417 T3759 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1977417 T3773 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1977419 T3746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1977420 T3746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1977421 T3746 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1977424 T3773 oass.SolrIndexSearcher.<init> Opening Searcher@1c010cfb main
[junit4:junit4]   2> 1977429 T3773 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/nodes_used_collection_shard1_replica1/data/tlog
[junit4:junit4]   2> 1977432 T3773 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1977433 T3773 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1977440 T3746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1977441 T3746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1977441 T3746 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1977442 T3746 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/nodes_used_collection_shard1_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/nodes_used_collection_shard1_replica2/data/
[junit4:junit4]   2> 1977443 T3746 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2c17277b
[junit4:junit4]   2> 1977443 T3746 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1977445 T3746 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/nodes_used_collection_shard1_replica2/data
[junit4:junit4]   2> 1977445 T3746 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/nodes_used_collection_shard1_replica2/data/index/
[junit4:junit4]   2> 1977446 T3746 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/nodes_used_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1977447 T3802 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@1c010cfb main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1977448 T3759 oass.SolrIndexSearcher.<init> Opening Searcher@2cf387d1 main
[junit4:junit4]   2> 1977449 T3759 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/nodes_used_collection_shard2_replica1/data/tlog
[junit4:junit4]   2> 1977451 T3746 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/nodes_used_collection_shard1_replica2/data/index
[junit4:junit4]   2> 1977451 T3773 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=down
[junit4:junit4]   2> 1977454 T3759 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1977455 T3759 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1977459 T3746 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6d3f8e90 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2859c165),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1977459 T3746 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1977463 T3803 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@2cf387d1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1977465 T3746 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1977466 T3746 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1977466 T3759 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=down
[junit4:junit4]   2> 1977467 T3746 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1977469 T3746 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1977470 T3746 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1977470 T3746 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1977471 T3746 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1977472 T3746 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1977472 T3746 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1977492 T3746 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1977502 T3746 oass.SolrIndexSearcher.<init> Opening Searcher@3fc5cc6c main
[junit4:junit4]   2> 1977503 T3746 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/nodes_used_collection_shard1_replica2/data/tlog
[junit4:junit4]   2> 1977505 T3746 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1977505 T3746 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1977510 T3804 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@3fc5cc6c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1977513 T3746 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=down
[junit4:junit4]   2> 1978765 T3736 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1978767 T3736 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52968__",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52968/_"}
[junit4:junit4]   2> 1978768 T3736 oasc.Overseer$ClusterStateUpdater.createCollection Create collection nodes_used_collection with numShards 2
[junit4:junit4]   2> 1978777 T3736 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52965__",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52965/_"}
[junit4:junit4]   2> 1978786 T3736 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52962__",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52962/_"}
[junit4:junit4]   2> 1978796 T3736 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":null,
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52959__",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52959/_"}
[junit4:junit4]   2> 1978807 T3740 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1978808 T3767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1978808 T3735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1978807 T3753 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1978808 T3781 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1978808 T3795 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1979378 T3789 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica2
[junit4:junit4]   2> 1979379 T3789 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica2 address:http://127.0.0.1:52968/_ collection:nodes_used_collection shard:shard2
[junit4:junit4]   2> 1979381 T3789 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard2/election
[junit4:junit4]   2> 1979403 T3789 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 1979412 T3789 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=179998
[junit4:junit4]   2> 1979459 T3773 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica1
[junit4:junit4]   2> 1979460 T3773 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica1 address:http://127.0.0.1:52965/_ collection:nodes_used_collection shard:shard1
[junit4:junit4]   2> 1979462 T3773 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard1/election
[junit4:junit4]   2> 1979472 T3759 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 1979473 T3759 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica1 address:http://127.0.0.1:52962/_ collection:nodes_used_collection shard:shard2
[junit4:junit4]   2> 1979480 T3773 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 1979487 T3773 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=179999
[junit4:junit4]   2> 1979520 T3746 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1979521 T3746 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica2 address:http://127.0.0.1:52959/_ collection:nodes_used_collection shard:shard1
[junit4:junit4]   2> 1979922 T3789 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 1979922 T3789 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 1979923 T3789 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52968/_/nodes_used_collection_shard2_replica2/
[junit4:junit4]   2> 1979923 T3789 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:52968/_ START replicas=[http://127.0.0.1:52962/_/nodes_used_collection_shard2_replica1/] nUpdates=100
[junit4:junit4]   2> 1979924 T3789 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:52968/_ DONE.  We have no versions.  sync failed.
[junit4:junit4]   2> 1979925 T3789 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4]   2> 1979926 T3789 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4]   2> 1979926 T3789 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52968/_/nodes_used_collection_shard2_replica2/
[junit4:junit4]   2> 1979928 T3789 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard2
[junit4:junit4]   2> ASYNC  NEW_CORE C228 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@593385b6 url=http://127.0.0.1:52962/_/nodes_used_collection_shard2_replica1 node=127.0.0.1:52962__ C228_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=down, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:52962__, base_url=http://127.0.0.1:52962/_}
[junit4:junit4]   2> 1979941 T3762 C228 P52962 oasc.SolrCore.execute [nodes_used_collection_shard2_replica1] webapp=/_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 1979991 T3773 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 1979992 T3773 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 1979993 T3773 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52965/_/nodes_used_collection_shard1_replica1/
[junit4:junit4]   2> 1979993 T3773 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:52965/_ START replicas=[http://127.0.0.1:52959/_/nodes_used_collection_shard1_replica2/] nUpdates=100
[junit4:junit4]   2> 1979994 T3773 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:52965/_ DONE.  We have no versions.  sync failed.
[junit4:junit4]   2> 1979995 T3773 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4]   2> 1979995 T3773 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4]   2> 1979997 T3773 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52965/_/nodes_used_collection_shard1_replica1/
[junit4:junit4]   2> 1979998 T3773 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard1
[junit4:junit4]   2> ASYNC  NEW_CORE C229 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@87dafda url=http://127.0.0.1:52959/_/nodes_used_collection_shard1_replica2 node=127.0.0.1:52959__ C229_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=down, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:52959__, base_url=http://127.0.0.1:52959/_}
[junit4:junit4]   2> 1980011 T3748 C229 P52959 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp=/_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 1980346 T3736 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1980443 T3753 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1980444 T3735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1980445 T3795 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1980444 T3740 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1980445 T3767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1980447 T3781 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1980460 T3789 oasc.ZkController.register We are http://127.0.0.1:52968/_/nodes_used_collection_shard2_replica2/ and leader is http://127.0.0.1:52968/_/nodes_used_collection_shard2_replica2/
[junit4:junit4]   2> 1980460 T3789 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica2 baseURL=http://127.0.0.1:52968/_
[junit4:junit4]   2> 1980461 T3789 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 1980461 T3789 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=active
[junit4:junit4]   2> 1980465 T3789 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1980467 T3789 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=6371 
[junit4:junit4]   2> 1980475 T3773 oasc.ZkController.register We are http://127.0.0.1:52965/_/nodes_used_collection_shard1_replica1/ and leader is http://127.0.0.1:52965/_/nodes_used_collection_shard1_replica1/
[junit4:junit4]   2> 1980476 T3773 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica1 baseURL=http://127.0.0.1:52965/_
[junit4:junit4]   2> 1980476 T3773 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 1980477 T3773 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=active
[junit4:junit4]   2> 1980480 T3773 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1980482 T3773 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=6394 
[junit4:junit4]   2> 1980484 T3759 oasc.ZkController.register We are http://127.0.0.1:52962/_/nodes_used_collection_shard2_replica1/ and leader is http://127.0.0.1:52968/_/nodes_used_collection_shard2_replica2/
[junit4:junit4]   2> 1980484 T3759 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica1 baseURL=http://127.0.0.1:52962/_
[junit4:junit4]   2> 1980485 T3759 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 1980485 T3759 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> 1980486 T3807 C228 P52962 oasc.RecoveryStrategy.run Starting recovery process.  core=nodes_used_collection_shard2_replica1 recoveringAfterStartup=true
[junit4:junit4]   2> 1980487 T3807 C228 P52962 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 1980487 T3759 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1980488 T3807 C228 P52962 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=recovering
[junit4:junit4]   2> 1980490 T3759 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=6397 
[junit4:junit4]   2> 1980492 T3807 C228 P52962 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1980521 T3790 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:52962___nodes_used_collection_shard2_replica1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 1980532 T3746 oasc.ZkController.register We are http://127.0.0.1:52959/_/nodes_used_collection_shard1_replica2/ and leader is http://127.0.0.1:52965/_/nodes_used_collection_shard1_replica1/
[junit4:junit4]   2> 1980532 T3746 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica2 baseURL=http://127.0.0.1:52959/_
[junit4:junit4]   2> 1980533 T3746 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1980533 T3746 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> 1980534 T3808 C229 P52959 oasc.RecoveryStrategy.run Starting recovery process.  core=nodes_used_collection_shard1_replica2 recoveringAfterStartup=true
[junit4:junit4]   2> 1980535 T3808 C229 P52959 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 1980535 T3746 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1980536 T3808 C229 P52959 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=recovering
[junit4:junit4]   2> 1980538 T3746 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=6445 
[junit4:junit4]   2> 1980539 T3808 C229 P52959 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 1980540 T3737 oasc.OverseerCollectionProcessor.createCollection Finished create command on all shards for collection: nodes_used_collection
[junit4:junit4]   2> 1980547 T3753 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type NodeDataChanged
[junit4:junit4]   2> 1980549 T3735 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 1980551 T3737 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{success={null={responseHeader={status=0,QTime=6371},core=nodes_used_collection_shard2_replica2},null={responseHeader={status=0,QTime=6394},core=nodes_used_collection_shard1_replica1},null={responseHeader={status=0,QTime=6397},core=nodes_used_collection_shard2_replica1},null={responseHeader={status=0,QTime=6445},core=nodes_used_collection_shard1_replica2}}}
[junit4:junit4]   2> 1980555 T3745 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2} status=0 QTime=6493 
[junit4:junit4]   2> 1980556 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 1980565 T3713 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1980571 T3774 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:52959___nodes_used_collection_shard1_replica2, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4]   2> 1980577 T3810 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c09f265 name:ZooKeeperConnection Watcher:127.0.0.1:52952/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1980578 T3713 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1980580 T3713 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1980587 T3713 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 1980590 T3713 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1980598 T3812 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@35765854 name:ZooKeeperConnection Watcher:127.0.0.1:52952/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1980603 T3713 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1980606 T3713 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1980655 T3758 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=3&maxShardsPerNode=3&name=awholynewcollection_0&replicationFactor=4&action=CREATE&wt=javabin&version=2
[junit4:junit4]   2> 1980658 T3735 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4]   2> 1980663 T3737 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000002 message:{
[junit4:junit4]   2> 	  "operation":"createcollection",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "maxShardsPerNode":"3",
[junit4:junit4]   2> 	  "createNodeSet":null,
[junit4:junit4]   2> 	  "name":"awholynewcollection_0",
[junit4:junit4]   2> 	  "replicationFactor":"4"}
[junit4:junit4]   2> 1980663 T3737 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica1 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:52962__
[junit4:junit4]   2> 1980663 T3737 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica2 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:52959__
[junit4:junit4]   2> 1980664 T3737 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica3 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:52968__
[junit4:junit4]   2> 1980664 T3737 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica4 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:52955__
[junit4:junit4]   2> 1980665 T3737 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica1 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:52965__
[junit4:junit4]   2> 1980666 T3737 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica2 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:52962__
[junit4:junit4]   2> 1980667 T3760 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/awholynewcollection_0_shard1_replica1
[junit4:junit4]   2> 1980667 T3760 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1980671 T3729 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica4' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369017378570/awholynewcollection_0_shard1_replica4
[junit4:junit4]   2> 1980672 T3729 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1980669 T3785 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica3' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard1_replica3
[junit4:junit4]   2> 1980675 T3785 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1980675 T3761 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/awholynewcollection_0_shard2_replica2
[junit4:junit4]   2> 1980676 T3761 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1980668 T3737 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica3 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:52959__
[junit4:junit4]   2> 1980677 T3785 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1980677 T3785 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1980668 T3744 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/awholynewcollection_0_shard1_replica2
[junit4:junit4]   2> 1980678 T3744 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1980679 T3737 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica4 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:52968__
[junit4:junit4]   2> 1980673 T3776 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/awholynewcollection_0_shard2_replica1
[junit4:junit4]   2> 1980680 T3776 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1980673 T3760 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1980681 T3760 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1980682 T3737 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica1 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:52955__
[junit4:junit4]   2> 1980687 T3729 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1980691 T3729 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1980692 T3787 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica4' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard2_replica4
[junit4:junit4]   2> 1980694 T3787 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1980687 T3776 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1980696 T3776 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1980686 T3747 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica3' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/awholynewcollection_0_shard2_replica3
[junit4:junit4]   2> 1980697 T3747 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1980685 T3744 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1980683 T3761 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1980699 T3761 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1980698 T3744 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1980700 T3760 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1980694 T3730 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369017378570/awholynewcollection_0_shard3_replica1
[junit4:junit4]   2> 1980701 T3730 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1980690 T3785 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1980688 T3737 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica2 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:52965__
[junit4:junit4]   2> 1980702 T3785 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1980701 T3760 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1980704 T3737 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica3 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:52962__
[junit4:junit4]   2> 1980706 T3737 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica4 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:52959__
[junit4:junit4]   2> 1980704 T3787 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1980706 T3787 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1980712 T3762 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica3' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/awholynewcollection_0_shard3_replica3
[junit4:junit4]   2> 1980713 T3762 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1980714 T3747 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1980714 T3747 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1980721 T3748 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica4' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/awholynewcollection_0_shard3_replica4
[junit4:junit4]   2> 1980723 T3748 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1980720 T3762 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4]   2> 1980724 T3762 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4]   2> 1980715 T3776 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1980719 T3777 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/awholynewcollection_0_shard3_replica2
[junit4:junit4]   2> 1980727 T3777 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4]   2> 1980715 T3761 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1980728 T3761 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1980727 T3787 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1980729 T3787 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1980726 T3776 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1980726 T3744 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1980728 T3729 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4]   2> 1980732 T3729 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1980728 T3730 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1980733 T3730 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1980731 T3744 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1980736 T3777 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1980737 T3777 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1980738 T3748 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1980739 T3748 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1980740 T3785 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1980743 T3747 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1980745 T3762 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4]   2> 1980746 T3760 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1980760 T3787 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1980763 T3777 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/awholynewcollection_0_shard3_replica2/'
[junit4:junit4]   2> 1980764 T3730 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369017378570/awholynewcollection_0_shard3_replica1/'
[junit4:junit4]   2> 1980766 T3729 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1980768 T3776 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1980770 T3748 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/awholynewcollection_0_shard3_replica4/'
[junit4:junit4]   2> 1980772 T3744 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1980773 T3785 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard1_replica3/'
[junit4:junit4]   2> 1980774 T3761 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1980824 T3760 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/awholynewcollection_0_shard1_replica1/'
[junit4:junit4]   2> 1980831 T3761 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/awholynewcollection_0_shard2_replica2/'
[junit4:junit4]   2> 1980834 T3747 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1980837 T3787 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard2_replica4/'
[junit4:junit4]   2> 1980837 T3729 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369017378570/awholynewcollection_0_shard1_replica4/'
[junit4:junit4]   2> 1980840 T3776 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/awholynewcollection_0_shard2_replica1/'
[junit4:junit4]   2> 1980849 T3762 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4]   2> 1980853 T3744 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/awholynewcollection_0_shard1_replica2/'
[junit4:junit4]   2> 1980861 T3747 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/awholynewcollection_0_shard2_replica3/'
[junit4:junit4]   2> 1980865 T3762 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/awholynewcollection_0_shard3_replica3/'
[junit4:junit4]   2> 1981174 T3747 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 1981179 T3744 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 1981241 T3787 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 1981266 T3777 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 1981323 T3785 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 1981343 T3762 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 1981361 T3730 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 1981369 T3760 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 1981409 T3761 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 1981415 T3748 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 1981525 T3776 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 1981616 T3744 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1981655 T3744 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1981668 T3729 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4]   2> 1981760 T3744 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica2] Schema name=test
[junit4:junit4]   2> 1981769 T3787 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1981804 T3787 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1981812 T3787 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica4] Schema name=test
[junit4:junit4]   2> 1981870 T3785 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1981907 T3730 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1981909 T3785 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1981929 T3785 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica3] Schema name=test
[junit4:junit4]   2> 1981954 T3730 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1981962 T3730 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica1] Schema name=test
[junit4:junit4]   2> 1981968 T3762 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1981974 T3777 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1981983 T3777 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1981991 T3777 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica2] Schema name=test
[junit4:junit4]   2> 1981993 T3762 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1982002 T3762 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica3] Schema name=test
[junit4:junit4]   2> 1982016 T3736 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1982020 T3736 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:52968___nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52968__",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52968/_"}
[junit4:junit4]   2> 1982045 T3736 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:52965___nodes_used_collection_shard1_replica1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52965__",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52965/_"}
[junit4:junit4]   2> 1982070 T3736 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:52962___nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52962__",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52962/_"}
[junit4:junit4]   2> 1982116 T3736 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:52959___nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52959__",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52959/_"}
[junit4:junit4]   2> 1982121 T3761 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1982130 T3761 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1982138 T3760 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1982138 T3761 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica2] Schema name=test
[junit4:junit4]   2> 1982144 T3760 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1982149 T3735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1982150 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1982150 T3781 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1982151 T3812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1982151 T3795 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1982152 T3767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1982151 T3753 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1982152 T3740 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1982177 T3760 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica1] Schema name=test
[junit4:junit4]   2> 1982300 T3729 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1982325 T3748 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1982326 T3729 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1982340 T3729 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica4] Schema name=test
[junit4:junit4]   2> 1982350 T3748 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1982363 T3776 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1982370 T3748 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica4] Schema name=test
[junit4:junit4]   2> 1982376 T3776 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1982384 T3776 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica1] Schema name=test
[junit4:junit4]   2> 1982408 T3747 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1982421 T3747 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4]   2> 1982435 T3747 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica3] Schema name=test
[junit4:junit4]   2> 1982524 T3790 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:52962___nodes_used_collection_shard2_replica1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4]   2> 1982525 T3790 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:52962___nodes_used_collection_shard2_replica1&state=recovering&nodeName=127.0.0.1:52962__&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard2_replica2&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2004 
[junit4:junit4]   2> 1982574 T3774 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:52959___nodes_used_collection_shard1_replica2, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4]   2> 1982575 T3774 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:52959___nodes_used_collection_shard1_replica2&state=recovering&nodeName=127.0.0.1:52959__&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard1_replica1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2004 
[junit4:junit4]   2>  C228_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:52962__, base_url=http://127.0.0.1:52962/_}
[junit4:junit4]   2> 1984530 T3807 C228 P52962 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:52968/_/nodes_used_collection_shard2_replica2/ core=nodes_used_collection_shard2_replica1 - recoveringAfterStartup=true
[junit4:junit4]   2> 1984531 T3807 C228 P52962 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:52962/_ START replicas=[http://127.0.0.1:52968/_/nodes_used_collection_shard2_replica2/] nUpdates=100
[junit4:junit4]   2> 1984532 T3807 C228 P52962 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 1984532 T3807 C228 P52962 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 1984533 T3807 C228 P52962 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 1984533 T3807 C228 P52962 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 1984534 T3807 C228 P52962 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 1984534 T3807 C228 P52962 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:52968/_/nodes_used_collection_shard2_replica2/. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 1984535 T3807 C228 P52962 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2>  C229_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:52959__, base_url=http://127.0.0.1:52959/_}
[junit4:junit4]   2> 1984578 T3808 C229 P52959 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:52965/_/nodes_used_collection_shard1_replica1/ core=nodes_used_collection_shard1_replica2 - recoveringAfterStartup=true
[junit4:junit4]   2> 1984587 T3808 C229 P52959 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:52959/_ START replicas=[http://127.0.0.1:52965/_/nodes_used_collection_shard1_replica1/] nUpdates=100
[junit4:junit4]   2> 1984589 T3808 C229 P52959 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 1984590 T3808 C229 P52959 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1984590 T3808 C229 P52959 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1984590 T3808 C229 P52959 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1984591 T3808 C229 P52959 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 1984611 T3808 C229 P52959 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:52965/_/nodes_used_collection_shard1_replica1/. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1984612 T3808 C229 P52959 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C230 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@56f32655 url=http://127.0.0.1:52965/_/nodes_used_collection_shard1_replica1 node=127.0.0.1:52965__ C230_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:52965__, base_url=http://127.0.0.1:52965/_, leader=true}
[junit4:junit4]   2> 1984619 T3773 C230 P52965 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp=/_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 1984659 T3775 C230 P52965 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> ASYNC  NEW_CORE C231 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@55cce9ad url=http://127.0.0.1:52968/_/nodes_used_collection_shard2_replica2 node=127.0.0.1:52968__ C231_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:52968__, base_url=http://127.0.0.1:52968/_, leader=true}
[junit4:junit4]   2> 1984676 T3786 C231 P52968 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 1984704 T3775 C230 P52965 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@41a22cce lockFactory=org.apache.lucene.store.NativeFSLockFactory@116c32b6),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1984707 T3775 C230 P52965 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1984709 T3775 C230 P52965 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@41a22cce lockFactory=org.apache.lucene.store.NativeFSLockFactory@116c32b6),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@41a22cce lockFactory=org.apache.lucene.store.NativeFSLockFactory@116c32b6),segFN=segments_2,generation=2}
[junit4:junit4]   2> 1984710 T3775 C230 P52965 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 1984710 T3775 C230 P52965 oass.SolrIndexSearcher.<init> Opening Searcher@41ea0cca realtime
[junit4:junit4]   2> 1984711 T3775 C230 P52965 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 1984712 T3775 C230 P52965 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard1_replica1] webapp=/_ path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 52
[junit4:junit4]   2> 1984717 T3808 C229 P52959 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 1984718 T3808 C229 P52959 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 1984724 T3772 C230 P52965 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 1984724 T3772 C230 P52965 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp=/_ path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 1984727 T3808 C229 P52959 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 1984727 T3808 C229 P52959 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 1984727 T3808 C229 P52959 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 1984750 T3773 C230 P52965 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp=/_ path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 1984751 T3808 C229 P52959 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 1984754 T3808 C229 P52959 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/nodes_used_collection_shard1_replica2/data/index.20130520093639243
[junit4:junit4]   2> 1984764 T3808 C229 P52959 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@25ca3f78 lockFactory=org.apache.lucene.store.NativeFSLockFactory@666a5680) fullCopy=false
[junit4:junit4]   2> 1984778 T3775 C230 P52965 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp=/_ path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1 
[junit4:junit4]   2> 1984781 T3808 C229 P52959 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 1984790 T3808 C229 P52959 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 1984791 T3808 C229 P52959 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1984794 T3808 C229 P52959 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6d3f8e90 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2859c165),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6d3f8e90 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2859c165),segFN=segments_2,generation=2}
[junit4:junit4]   2> 1984796 T3808 C229 P52959 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 1984797 T3808 C229 P52959 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 1984798 T3808 C229 P52959 oass.SolrIndexSearcher.<init> Opening Searcher@56f6285a main
[junit4:junit4]   2> 1984799 T3804 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@56f6285a main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 1984809 T3808 C229 P52959 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/nodes_used_collection_shard1_replica2/data/index.20130520093639243 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/nodes_used_collection_shard1_replica2/data/index.20130520093639243;done=true>>]
[junit4:junit4]   2> 1984810 T3808 C229 P52959 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/nodes_used_collection_shard1_replica2/data/index.20130520093639243
[junit4:junit4]   2> 1984820 T3808 C229 P52959 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/nodes_used_collection_shard1_replica2/data/index.20130520093639243
[junit4:junit4]   2> 1984820 T3808 C229 P52959 oasc.RecoveryStrategy.replay No replay needed. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1984820 T3808 C229 P52959 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1984821 T3808 C229 P52959 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=active
[junit4:junit4]   2> 1984832 T3788 C231 P52968 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4]   2> 1984841 T3808 C229 P52959 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=nodes_used_collection_shard1_replica2
[junit4:junit4]   2> 1984860 T3788 C231 P52968 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7feb4c87 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3b079af9),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1984861 T3788 C231 P52968 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1984862 T3788 C231 P52968 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7feb4c87 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3b079af9),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7feb4c87 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3b079af9),segFN=segments_2,generation=2}
[junit4:junit4]   2> 1984863 T3788 C231 P52968 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 1984864 T3788 C231 P52968 oass.SolrIndexSearcher.<init> Opening Searcher@19bd2e6 realtime
[junit4:junit4]   2> 1984864 T3788 C231 P52968 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 1984865 T3788 C231 P52968 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard2_replica2] webapp=/_ path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 33
[junit4:junit4]   2> 1984867 T3807 C228 P52962 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 1984868 T3807 C228 P52962 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 1984888 T3789 C231 P52968 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 1984889 T3789 C231 P52968 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/_ path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 1984890 T3807 C228 P52962 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 1984890 T3807 C228 P52962 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 1984891 T3807 C228 P52962 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 1984898 T3786 C231 P52968 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/_ path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 1984900 T3807 C228 P52962 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 1984903 T3807 C228 P52962 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/nodes_used_collection_shard2_replica1/data/index.20130520093639392
[junit4:junit4]   2> 1984905 T3807 C228 P52962 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@7fdae8cd lockFactory=org.apache.lucene.store.NativeFSLockFactory@201fb7ff) fullCopy=false
[junit4:junit4]   2> 1984915 T3788 C231 P52968 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/_ path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1 
[junit4:junit4]   2> 1984929 T3807 C228 P52962 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 1984940 T3807 C228 P52962 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 1984940 T3807 C228 P52962 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 1984977 T3807 C228 P52962 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@20d1729f lockFactory=org.apache.lucene.store.NativeFSLockFactory@33022a81),segFN=segments_1,generation=1}
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@20d1729f lockFactory=org.apache.lucene.store.NativeFSLockFactory@33022a81),segFN=segments_2,generation=2}
[junit4:junit4]   2> 1984979 T3807 C228 P52962 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4]   2> 1984980 T3807 C228 P52962 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 1984981 T3807 C228 P52962 oass.SolrIndexSearcher.<init> Opening Searcher@3d6d7a95 main
[junit4:junit4]   2> 1984982 T3803 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@3d6d7a95 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 1984983 T3807 C228 P52962 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/nodes_used_collection_shard2_replica1/data/index.20130520093639392 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/nodes_used_collection_shard2_replica1/data/index.20130520093639392;done=true>>]
[junit4:junit4]   2> 1984983 T3807 C228 P52962 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/nodes_used_collection_shard2_replica1/data/index.20130520093639392
[junit4:junit4]   2> 1984984 T3807 C228 P52962 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/nodes_used_collection_shard2_replica1/data/index.20130520093639392
[junit4:junit4]   2> 1984984 T3807 C228 P52962 oasc.RecoveryStrategy.replay No replay needed. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 1984984 T3807 C228 P52962 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 1984985 T3807 C228 P52962 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=active
[junit4:junit4]   2> 1984993 T3807 C228 P52962 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=nodes_used_collection_shard2_replica1
[junit4:junit4]   2> 1985330 T3736 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 1985333 T3736 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:52959___nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52959__",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52959/_"}
[junit4:junit4]   2> 1985359 T3736 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "core_node_name":"127.0.0.1:52962___nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "shard_state":"active",
[junit4:junit4]   2> 	  "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4]   2> 	  "collection":"nodes_used_collection",
[junit4:junit4]   2> 	  "shard_range":null,
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52962__",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52962/_"}
[junit4:junit4]   2> 1985392 T3812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1985393 T3810 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1985394 T3781 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1985394 T3795 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1985395 T3740 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1985395 T3767 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1985395 T3753 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1985412 T3735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 1989165 T3785 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1989186 T3785 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1989194 T3785 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1989370 T3744 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1989386 T3777 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1989412 T3787 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1989407 T3760 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1989424 T3777 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1989463 T3785 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1989468 T3787 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1989484 T3760 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1989496 T3760 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1989535 T3787 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1989539 T3762 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1989558 T3777 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1989584 T3785 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1989588 T3744 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1989600 T3762 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1989608 T3744 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1989622 T3785 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1989629 T3785 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1989631 T3785 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1989631 T3785 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1989639 T3762 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1989654 T3785 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1989655 T3785 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1989655 T3785 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1989656 T3785 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica3] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard1_replica3/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard1_replica3/data/
[junit4:junit4]   2> 1989656 T3785 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2c17277b
[junit4:junit4]   2> 1989662 T3785 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1989677 T3785 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard1_replica3/data
[junit4:junit4]   2> 1989678 T3785 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard1_replica3/data/index/
[junit4:junit4]   2> 1989679 T3785 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard1_replica3] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard1_replica3/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1989691 T3785 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard1_replica3/data/index
[junit4:junit4]   2> 1989704 T3760 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1989735 T3760 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1989751 T3744 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1989762 T3785 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@65a4b22c lockFactory=org.apache.lucene.store.NativeFSLockFactory@519b081),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1989763 T3785 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1989768 T3785 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1989769 T3785 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1989771 T3785 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1989772 T3785 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1989773 T3785 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1989773 T3785 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1989774 T3785 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1989790 T3787 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1989794 T3777 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1989850 T3747 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1989854 T3787 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1989854 T3785 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1989874 T3785 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1989871 T3777 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1989871 T3760 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1989890 T3747 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1989897 T3760 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1989898 T3760 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1989898 T3760 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1989864 T3744 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1989906 T3747 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1989911 T3777 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1989925 T3760 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1989947 T3760 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1989949 T3760 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1989950 T3760 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/awholynewcollection_0_shard1_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/awholynewcollection_0_shard1_replica1/data/
[junit4:junit4]   2> 1989950 T3760 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2c17277b
[junit4:junit4]   2> 1989955 T3760 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1989934 T3777 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1989958 T3777 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1989959 T3777 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1989964 T3748 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1989967 T3777 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1989968 T3777 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1989969 T3777 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1989970 T3777 oasc.SolrCore.<init> [awholynewcollection_0_shard3_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/awholynewcollection_0_shard3_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/awholynewcollection_0_shard3_replica2/data/
[junit4:junit4]   2> 1989971 T3777 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2c17277b
[junit4:junit4]   2> 1989972 T3777 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1989970 T3760 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/awholynewcollection_0_shard1_replica1/data
[junit4:junit4]   2> 1989976 T3760 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/awholynewcollection_0_shard1_replica1/data/index/
[junit4:junit4]   2> 1989981 T3760 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard1_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/awholynewcollection_0_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1989988 T3777 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/awholynewcollection_0_shard3_replica2/data
[junit4:junit4]   2> 1989990 T3777 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/awholynewcollection_0_shard3_replica2/data/index/
[junit4:junit4]   2> 1989991 T3777 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard3_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/awholynewcollection_0_shard3_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1989993 T3760 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/awholynewcollection_0_shard1_replica1/data/index
[junit4:junit4]   2> 1989996 T3744 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1990006 T3777 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369017384452/awholynewcollection_0_shard3_replica2/data/index
[junit4:junit4]   2> 1990022 T3744 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1990023 T3744 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1990024 T3744 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1990025 T3787 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1990030 T3785 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1990043 T3744 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1990045 T3744 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1990045 T3744 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1990046 T3744 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/awholynewcollection_0_shard1_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/awholynewcollection_0_shard1_replica2/data/
[junit4:junit4]   2> 1990047 T3744 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2c17277b
[junit4:junit4]   2> 1990048 T3744 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1990052 T3777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@40f679c1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2365978a),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1990053 T3777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1990062 T3730 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1990069 T3787 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1990070 T3787 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1990072 T3787 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1990081 T3785 oass.SolrIndexSearcher.<init> Opening Searcher@4ec271e7 main
[junit4:junit4]   2> 1990082 T3785 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard1_replica3/data/tlog
[junit4:junit4]   2> 1990083 T3777 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1990084 T3777 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1990086 T3777 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1990088 T3744 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/awholynewcollection_0_shard1_replica2/data
[junit4:junit4]   2> 1990089 T3744 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/awholynewcollection_0_shard1_replica2/data/index/
[junit4:junit4]   2> 1990090 T3744 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard1_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/awholynewcollection_0_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1990103 T3748 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1990115 T3748 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1990083 T3785 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1990119 T3785 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1990117 T3762 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1990117 T3760 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@142d59f8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4d8fe605),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1990124 T3760 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1990116 T3744 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/awholynewcollection_0_shard1_replica2/data/index
[junit4:junit4]   2> 1990105 T3787 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1990132 T3787 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1990133 T3787 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1990134 T3787 oasc.SolrCore.<init> [awholynewcollection_0_shard2_replica4] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard2_replica4/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard2_replica4/data/
[junit4:junit4]   2> 1990135 T3787 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2c17277b
[junit4:junit4]   2> 1990136 T3787 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4]   2> 1990137 T3729 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1990119 T3777 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1990151 T3787 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard2_replica4/data
[junit4:junit4]   2> 1990152 T3787 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard2_replica4/data/index/
[junit4:junit4]   2> 1990152 T3787 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard2_replica4] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard2_replica4/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1990164 T3787 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard2_replica4/data/index
[junit4:junit4]   2> 1990187 T3730 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1990164 T3761 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1990202 T3762 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1990150 T3729 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1990132 T3825 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard1_replica3] Registered new searcher Searcher@4ec271e7 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1990151 T3777 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1990212 T3777 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1990216 T3777 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1990217 T3777 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1990218 T3777 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1990215 T3785 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica3 state=down
[junit4:junit4]   2> 1990220 T3744 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@bb005f4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3d927f35),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1990221 T3744 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1990213 T3760 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1990231 T3760 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1990235 T3760 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1990236 T3760 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1990238 T3760 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1990239 T3760 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1990242 T3787 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6a04107f lockFactory=org.apache.lucene.store.NativeFSLockFactory@536b7cff),segFN=segments_1,generation=1}
[junit4:junit4]   2> 1990243 T3787 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4]   2> 1990250 T3744 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1990252 T3744 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1990254 T3744 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1990262 T3744 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1990263 T3744 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1990264 T3744 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1990265 T3744 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1990267 T3744 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1990268 T3744 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1990271 T3787 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1990272 T3787 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1990273 T3787 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1990274 T3787 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1990275 T3787 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1990276 T3787 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1990276 T3787 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1990277 T3787 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1990297 T3787 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1990300 T3777 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1990302 T3787 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1990243 T3760 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1990308 T3760 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1990308 T3760 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1990296 T3729 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1990311 T3747 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1990314 T3760 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1990294 T3761 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1990260 T3730 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1990330 T3787 oass.SolrIndexSearcher.<init> Opening Searcher@68add5b4 main
[junit4:junit4]   2> 1990331 T3787 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369017386525/awholynewcollection_0_shard2_replica4/data/tlog
[junit4:junit4]   2> 1990335 T3787 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1990336 T3787 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1990345 T3744 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1990352 T3747 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1990353 T3762 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1990357 T3761 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1990355 T3744 oass.SolrIndexSearcher.<init> Opening Searcher@97db472 main
[junit4:junit4]   2> 1990374 T3744 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369017380416/awholynewcollection_0_shard1_replica2/data/tlog
[junit4:junit4]   2> 1990354 T3760 oass.SolrIndexSearcher.<init> Opening Searcher@5e4a24cf main
[junit4:junit4]   2> 1990376 T3760 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369017382426/awholynewcollection_0_shard1_replica1/data/tlog
[junit4:junit4]   2> 1990378 T3760 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1990379 T3760 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1990368 T3747 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1990388 T3747 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1990389 T3747 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1990394 T3747 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1990394 T3760 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica1 state=down
[junit4:junit4]   2> 1990391 T3828 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard1_replica1] Registered new searcher Searcher@5e4a24cf main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1990390 T3744 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1990398 T3744 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1990390 T3777 oass.SolrIndexSearcher.<init> Opening Searcher@1d369faf main
[junit4:junit4]   2> 1990404 T3827 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard2_replica4] Registered new searcher Searcher@68add5b4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1990402 T3762 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4]   2> 1990411 T3762 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4]   2> 1990411 T3762 oass.OpenExchangeR

[...truncated too long message...]

o.seed([9DAA94E6B71ECFAB:1C4C1AFEC041AF97]:0)
[junit4:junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:173)
[junit4:junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
[junit4:junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
[junit4:junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:512)
[junit4:junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 2251449 T3713 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 287647 T3712 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
[junit4:junit4]   2> 2252551 T3795 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/awholynewcollection_0/leader_elect/shard1/election
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4]   2> 
[junit4:junit4]   2> 2252551 T3795 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/awholynewcollection_0/leader_elect/shard2/election
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4]   2> 
[junit4:junit4]   2> 2252552 T3795 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2252553 T3795 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 2252553 T3795 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {timestamp=Lucene41(blocksize=128), range_facet_l=Lucene41(blocksize=128), id=PostingsFormat(name=Memory doPackFST= true), range_facet_sl=PostingsFormat(name=Memory doPackFST= true), a_t=Lucene41(blocksize=128), text=PostingsFormat(name=MockRandom), range_facet_si=Lucene41(blocksize=128), _version_=PostingsFormat(name=Memory doPackFST= false), other_tl1=PostingsFormat(name=Memory doPackFST= false), multiDefault=Lucene41(blocksize=128), a_si=PostingsFormat(name=Memory doPackFST= false), intDefault=PostingsFormat(name=Memory doPackFST= false)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=ro_RO, timezone=Asia/Ho_Chi_Minh
[junit4:junit4]   2> NOTE: Mac OS X 10.8.3 x86_64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=2,threads=1,free=89034816,total=389349376
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestCharFilters, UpdateRequestProcessorFactoryTest, HighlighterConfigTest, StatelessScriptUpdateProcessorFactoryTest, TestUniqueKeyFieldResource, ResourceLoaderTest, TestSolrJ, URLClassifyProcessorTest, TestSolrIndexConfig, TestPartialUpdateDeduplication, TestSchemaSimilarityResource, SoftAutoCommitTest, FileBasedSpellCheckerTest, SuggesterTSTTest, TestMultiCoreConfBootstrap, DirectUpdateHandlerTest, TestCollationKeyRangeQueries, TestDocumentBuilder, PreAnalyzedUpdateProcessorTest, ShowFileRequestHandlerTest, TestWriterPerf, TestStressLucene, TestExtendedDismaxParser, SystemInfoHandlerTest, TestComponentsName, ConvertedLegacyTest, DocValuesTest, RegexBoostProcessorTest, DistributedSpellCheckComponentTest, SolrIndexSplitterTest, TestCollationField, JSONWriterTest, TestLuceneMatchVersion, SpellingQueryConverterTest, DateMathParserTest, ChaosMonkeyNothingIsSafeTest, TestRTGBase, SynonymTokenizerTest, TestJmxIntegration, RequiredFieldsTest, TestJoin, ExternalFileFieldSortTest, TestRandomDVFaceting, TestFiltering, TestDocSet, ChaosMonkeyShardSplitTest, SortByFunctionTest, SimplePostToolTest, DocumentBuilderTest, SearchHandlerTest, TestCoreContainer, TestRangeQuery, TestPropInject, TestCloudManagedSchemaAddField, PingRequestHandlerTest, TestXIncludeConfig, SpatialFilterTest, LegacyHTMLStripCharFilterTest, TestPluginEnable, ClusterStateUpdateTest, TestSolrQueryParserDefaultOperatorResource, FileUtilsTest, TestBM25SimilarityFactory, TestQuerySenderListener, TestAnalyzedSuggestions, TestMaxScoreQueryParser, DistanceFunctionTest, TestCoreDiscovery, AutoCommitTest, IndexReaderFactoryTest, TestSolrXMLSerializer, TestRandomFaceting, PrimitiveFieldTypeTest, CacheHeaderTest, JsonLoaderTest, TestSearchPerf, StatsComponentTest, CoreContainerCoreInitFailuresTest, TestCSVLoader, TestPropInjectDefaults, TestFastWriter, SliceStateUpdateTest, WordBreakSolrSpellCheckerTest, TermsComponentTest, MBeansHandlerTest, TestReload, TestDynamicFieldResource, TestShardHandlerFactory, TestQuerySenderNoQuery, ScriptEngineTest, ClusterStateTest, MultiTermTest, SampleTest, IndexSchemaTest, PolyFieldTest, OpenExchangeRatesOrgProviderTest, ReturnFieldsTest, TestStressRecovery, TestDistributedGrouping, SuggesterWFSTTest, DisMaxRequestHandlerTest, TestQueryUtils, TestFuzzyAnalyzedSuggestions, TestSolrQueryParser, DirectSolrConnectionTest, DirectUpdateHandlerOptimizeTest, TestQueryTypes, OutputWriterTest, SchemaVersionSpecificBehaviorTest, PluginInfoTest, NotRequiredUniqueKeyTest, TestManagedSchema, TestFieldTypeResource, AnalysisAfterCoreReloadTest, SuggesterTest, TestLMDirichletSimilarityFactory, TestSolrCoreProperties, EchoParamsTest, XmlUpdateRequestHandlerTest, SolrCoreTest, TestSolrDeletionPolicy2, UUIDFieldTest, CoreAdminHandlerTest, TestGroupingSearch, TestZkChroot, BadIndexSchemaTest, TestSort, LeaderElectionTest, BasicDistributedZk2Test, TestFastLRUCache, ShardRoutingCustomTest, TestSchemaNameResource, TestFieldResource, TestRemoteStreaming, OverseerTest, DistributedQueryElevationComponentTest, TestUtils, BasicDistributedZkTest, TestRealTimeGet, TestSolr4Spatial, QueryParsingTest, TestRecovery, ShardSplitTest, BasicFunctionalityTest, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest]
[junit4:junit4] Completed in 290.49s, 1 test, 1 failure <<< FAILURES!

[...truncated 453 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/build.xml:383: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/build.xml:363: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/common-build.xml:887: There were test failures: 296 suites, 1234 tests, 1 failure, 18 ignored (6 assumptions)

Total time: 95 minutes 50 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Re: [JENKINS] Lucene-Solr-4.x-MacOSX (64bit/jdk1.6.0) - Build # 469 - Still Failing!

Posted by Shai Erera <se...@gmail.com>.
Well, startHttpServer returns only after the server has been started, but
I've seen in the past that it may not start all the servlets immediately,
i.e. lazy loading.

Maybe we can make HttpReplicatorTest.startServer() loop while
HttpReplicator.checkForUpdate throws ConnectException? That way, when the
test actually starts, the server is already guaranteed to be up.

But looking at the exception, and given the Jetty instance has been
started, I'm not sure it will solve it.

Shai


On Sun, May 19, 2013 at 11:38 PM, Uwe Schindler <uw...@thetaphi.de> wrote:

> Could it be that the jetty server did not yet started up completely? Osx
> is slow...
>
>
>
> Policeman Jenkins Server <je...@thetaphi.de> schrieb:
>>
>>  Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-MacOSX/469/
>> Java: 64bit/jdk1.6.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
>>
>> 1 tests failed.
>> REGRESSION:  org.apache.lucene.replicator.http.HttpReplicatorTest.testBasic
>>
>> Error Message:
>> Connection to http://localhost:52124 refused
>>
>> Stack Trace:
>> org.apache.http.conn.HttpHostConnectException: Connection to http://localhost:52124 refused
>>  at __randomizedtesting.SeedInfo.seed([E75C887E5C9618BD:4CA6956B834A9E93]:0)
>>
>>  at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
>>  at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
>>
>>  at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
>>  at
>> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
>>  at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
>>  at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
>>
>>  at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
>>  at org.apache.lucene.replicator.http.HttpClientBase.executeGET(HttpClientBase.java:178)
>>  at org.apache.lucene.replicator.http.HttpReplicator.checkForUpdate(HttpReplicator.java:51)
>>
>>  at org.apache.lucene.replicator.ReplicationClient.doUpdate(ReplicationClient.java:196)
>>  at org.apache.lucene.replicator.ReplicationClient.updateNow(ReplicationClient.java:402)
>>  at org.apache.lucene.replicator.http.HttpReplicatorTest.testBasic(HttpReplicatorTest.java:110)
>>
>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>  at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>  at java.lang.reflect.Method.invoke(Method.java:597)
>>
>>  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
>>  at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
>>  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
>>
>>  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
>>  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
>>  at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>>
>>  at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
>>  at
>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>  at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>>
>>  at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>>  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>>  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>
>>  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
>>  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
>>
>>  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
>>  at
>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
>>  at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
>>  at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
>>
>>  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
>>  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>>  at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>>
>>  at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>>  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>
>>  at
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>>  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>
>>  at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
>>  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>>
>>  at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>>  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
>>
>>  at java.lang.Thread.run(Thread.java:680)
>> Caused by: java.net.ConnectException: Connection
>> refused
>>  at java.net.PlainSocketImpl.socketConnect(Native Method)
>>  at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
>>
>>  at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
>>  at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
>>
>>  at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:432)
>>  at java.net.Socket.connect(Socket.java:529)
>>  at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
>>
>>  at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
>>  ... 48 more
>>
>>
>>
>>
>> Build Log:
>> [...truncated 7459 lines...]
>> [junit4:junit4] Suite: org.apache.lucene.replicator.http.HttpReplicatorTest
>>
>> [junit4:junit4]   2> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
>> [junit4:junit4]   2> SLF4J: Defaulting to no-operation (NOP) logger implementation
>> [junit4:junit4]   2> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
>>
>> [junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HttpReplicatorTest -Dtests.method=testBasic -Dtests.seed=E75C887E5C9618BD -Dtests.slow=true -Dtests.locale=nl_NL -Dtests.timezone=Europe/Monaco -Dtests.file.encoding=ISO-8859-1
>>
>> [junit4:junit4] ERROR   3.42s | HttpReplicatorTest.testBasic <<<
>> [junit4:junit4]    > Throwable #1: org.apache.http.conn.HttpHostConnectException: Connection to http://localhost:52124 refused
>>
>> [junit4:junit4]    >  at __randomizedtesting.SeedInfo.seed([E75C887E5C9618BD:4CA6956B834A9E93]:0)
>> [junit4:junit4]    >  at
>> org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
>> [junit4:junit4]    >  at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
>>
>> [junit4:junit4]    >  at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
>> [junit4:junit4]    >  at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
>>
>> [junit4:junit4]    >  at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
>> [junit4:junit4]    >  at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
>>
>> [junit4:junit4]    >  at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
>> [junit4:junit4]    >  at org.apache.lucene.replicator.http.HttpClientBase.executeGET(HttpClientBase.java:178)
>>
>> [junit4:junit4]    >
>> at org.apache.lucene.replicator.http.HttpReplicator.checkForUpdate(HttpReplicator.java:51)
>> [junit4:junit4]    >  at org.apache.lucene.replicator.ReplicationClient.doUpdate(ReplicationClient.java:196)
>> [junit4:junit4]    >  at org.apache.lucene.replicator.ReplicationClient.updateNow(ReplicationClient.java:402)
>>
>> [junit4:junit4]    >  at org.apache.lucene.replicator.http.HttpReplicatorTest.testBasic(HttpReplicatorTest.java:110)
>> [junit4:junit4]    >  at java.lang.Thread.run(Thread.java:680)
>> [junit4:junit4]    > Caused by: java.net.ConnectException: Connection refused
>>
>> [junit4:junit4]    >  at java.net.PlainSocketImpl.socketConnect(Native Method)
>> [junit4:junit4]    >  at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
>>
>> [junit4:junit4]    >  at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
>> [junit4:junit4]    >  at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
>>
>> [junit4:junit4]    >  at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:432)
>> [junit4:junit4]    >  at java.net.Socket.connect(Socket.java:529)
>>
>> [junit4:junit4]    >  at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
>> [junit4:junit4]    >  at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
>>
>> [junit4:junit4]    >  ... 48 more
>> [junit4:junit4]   2> NOTE: test params are: codec=Lucene41, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=nl_NL, timezone=Europe/Monaco
>> [junit4:junit4]   2> NOTE: Mac OS X 10.8.3 x86_64/Apple Inc.
>> 1.6.0_45 (64-bit)/cpus=2,threads=1,free=67789600,total=85000192
>> [junit4:junit4]   2> NOTE: All tests run in this JVM: [SessionTokenTest, IndexRevisionTest, IndexAndTaxonomyRevisionTest, IndexAndTaxonomyReplicationClientTest, IndexReplicationClientTest, LocalReplicatorTest, HttpReplicatorTest]
>>
>> [junit4:junit4] Completed in 3.50s, 1 test, 1 error <<< FAILURES!
>>
>> [...truncated 9 lines...]
>> BUILD FAILED
>> /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/build.xml:383: The following error occurred while executing this line:
>>
>> /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/build.xml:363: The following error occurred while executing this line:
>> /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/build.xml:39: The following error occurred while executing this line:
>>
>> /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/build.xml:554: The following error occurred while executing this line:
>> /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/common-build.xml:1827: The following error occurred while executing this line:
>>
>> /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/module-build.xml:61: The following error occurred while executing this line:
>> /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
>>
>> /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/common-build.xml:887: There were test failures: 7 suites, 30 tests, 1 error
>>
>> Total time: 51 minutes 44 seconds
>> Build step 'Invoke Ant' marked build as failure
>>
>> Description set: Java: 64bit/jdk1.6.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
>> Archiving artifacts
>> Recording test results
>> Email was triggered for: Failure
>> Sending email for trigger: Failure
>>
>>
>>
>> ------------------------------
>>
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
> --
> Uwe Schindler
> H.-H.-Meier-Allee 63, 28213 Bremen
> http://www.thetaphi.de
>

Re: [JENKINS] Lucene-Solr-4.x-MacOSX (64bit/jdk1.6.0) - Build # 469 - Still Failing!

Posted by Uwe Schindler <uw...@thetaphi.de>.
Could it be that the jetty server did not yet started up completely? Osx is slow...



Policeman Jenkins Server <je...@thetaphi.de> schrieb:
>Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-MacOSX/469/
>Java: 64bit/jdk1.6.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
>
>1 tests failed.
>REGRESSION: 
>org.apache.lucene.replicator.http.HttpReplicatorTest.testBasic
>
>Error Message:
>Connection to http://localhost:52124 refused
>
>Stack Trace:
>org.apache.http.conn.HttpHostConnectException: Connection to
>http://localhost:52124 refused
>	at
>__randomizedtesting.SeedInfo.seed([E75C887E5C9618BD:4CA6956B834A9E93]:0)
>	at
>org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
>	at
>org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
>	at
>org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
>	at
>org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
>	at
>org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
>	at
>org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
>	at
>org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
>	at
>org.apache.lucene.replicator.http.HttpClientBase.executeGET(HttpClientBase.java:178)
>	at
>org.apache.lucene.replicator.http.HttpReplicator.checkForUpdate(HttpReplicator.java:51)
>	at
>org.apache.lucene.replicator.ReplicationClient.doUpdate(ReplicationClient.java:196)
>	at
>org.apache.lucene.replicator.ReplicationClient.updateNow(ReplicationClient.java:402)
>	at
>org.apache.lucene.replicator.http.HttpReplicatorTest.testBasic(HttpReplicatorTest.java:110)
>	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>	at
>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>	at
>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>	at java.lang.reflect.Method.invoke(Method.java:597)
>	at
>com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
>	at
>com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
>	at
>com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
>	at
>com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
>	at
>com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
>	at
>org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>	at
>org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
>	at
>org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>	at
>com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>	at
>org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>	at
>org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>	at
>org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>	at
>com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>	at
>com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
>	at
>com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
>	at
>com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
>	at
>com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
>	at
>com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
>	at
>com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
>	at
>com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
>	at
>org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>	at
>org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>	at
>com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>	at
>com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>	at
>com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>	at
>com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>	at
>org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
>	at
>org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>	at
>org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
>	at
>org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>	at
>com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>	at
>com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
>	at java.lang.Thread.run(Thread.java:680)
>Caused by: java.net.ConnectException: Connection refused
>	at java.net.PlainSocketImpl.socketConnect(Native Method)
>	at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
>	at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
>	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
>	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:432)
>	at java.net.Socket.connect(Socket.java:529)
>	at
>org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
>	at
>org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
>	... 48 more
>
>
>
>
>Build Log:
>[...truncated 7459 lines...]
>[junit4:junit4] Suite:
>org.apache.lucene.replicator.http.HttpReplicatorTest
>[junit4:junit4]   2> SLF4J: Failed to load class
>"org.slf4j.impl.StaticLoggerBinder".
>[junit4:junit4]   2> SLF4J: Defaulting to no-operation (NOP) logger
>implementation
>[junit4:junit4]   2> SLF4J: See
>http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
>[junit4:junit4]   2> NOTE: reproduce with: ant test 
>-Dtestcase=HttpReplicatorTest -Dtests.method=testBasic
>-Dtests.seed=E75C887E5C9618BD -Dtests.slow=true -Dtests.locale=nl_NL
>-Dtests.timezone=Europe/Monaco -Dtests.file.encoding=ISO-8859-1
>[junit4:junit4] ERROR   3.42s | HttpReplicatorTest.testBasic <<<
>[junit4:junit4]    > Throwable #1:
>org.apache.http.conn.HttpHostConnectException: Connection to
>http://localhost:52124 refused
>[junit4:junit4]    > 	at
>__randomizedtesting.SeedInfo.seed([E75C887E5C9618BD:4CA6956B834A9E93]:0)
>[junit4:junit4]    > 	at
>org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
>[junit4:junit4]    > 	at
>org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
>[junit4:junit4]    > 	at
>org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
>[junit4:junit4]    > 	at
>org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
>[junit4:junit4]    > 	at
>org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
>[junit4:junit4]    > 	at
>org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
>[junit4:junit4]    > 	at
>org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
>[junit4:junit4]    > 	at
>org.apache.lucene.replicator.http.HttpClientBase.executeGET(HttpClientBase.java:178)
>[junit4:junit4]    > 	at
>org.apache.lucene.replicator.http.HttpReplicator.checkForUpdate(HttpReplicator.java:51)
>[junit4:junit4]    > 	at
>org.apache.lucene.replicator.ReplicationClient.doUpdate(ReplicationClient.java:196)
>[junit4:junit4]    > 	at
>org.apache.lucene.replicator.ReplicationClient.updateNow(ReplicationClient.java:402)
>[junit4:junit4]    > 	at
>org.apache.lucene.replicator.http.HttpReplicatorTest.testBasic(HttpReplicatorTest.java:110)
>[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:680)
>[junit4:junit4]    > Caused by: java.net.ConnectException: Connection
>refused
>[junit4:junit4]    > 	at java.net.PlainSocketImpl.socketConnect(Native
>Method)
>[junit4:junit4]    > 	at
>java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
>[junit4:junit4]    > 	at
>java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
>[junit4:junit4]    > 	at
>java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
>[junit4:junit4]    > 	at
>java.net.SocksSocketImpl.connect(SocksSocketImpl.java:432)
>[junit4:junit4]    > 	at java.net.Socket.connect(Socket.java:529)
>[junit4:junit4]    > 	at
>org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
>[junit4:junit4]    > 	at
>org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
>[junit4:junit4]    > 	... 48 more
>[junit4:junit4]   2> NOTE: test params are: codec=Lucene41,
>sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {},
>locale=nl_NL, timezone=Europe/Monaco
>[junit4:junit4]   2> NOTE: Mac OS X 10.8.3 x86_64/Apple Inc. 1.6.0_45
>(64-bit)/cpus=2,threads=1,free=67789600,total=85000192
>[junit4:junit4]   2> NOTE: All tests run in this JVM:
>[SessionTokenTest, IndexRevisionTest, IndexAndTaxonomyRevisionTest,
>IndexAndTaxonomyReplicationClientTest, IndexReplicationClientTest,
>LocalReplicatorTest, HttpReplicatorTest]
>[junit4:junit4] Completed in 3.50s, 1 test, 1 error <<< FAILURES!
>
>[...truncated 9 lines...]
>BUILD FAILED
>/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/build.xml:383:
>The following error occurred while executing this line:
>/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/build.xml:363:
>The following error occurred while executing this line:
>/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/build.xml:39:
>The following error occurred while executing this line:
>/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/build.xml:554:
>The following error occurred while executing this line:
>/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/common-build.xml:1827:
>The following error occurred while executing this line:
>/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/module-build.xml:61:
>The following error occurred while executing this line:
>/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/common-build.xml:1243:
>The following error occurred while executing this line:
>/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/common-build.xml:887:
>There were test failures: 7 suites, 30 tests, 1 error
>
>Total time: 51 minutes 44 seconds
>Build step 'Invoke Ant' marked build as failure
>Description set: Java: 64bit/jdk1.6.0 -XX:-UseCompressedOops
>-XX:+UseConcMarkSweepGC
>Archiving artifacts
>Recording test results
>Email was triggered for: Failure
>Sending email for trigger: Failure
>
>
>
>
>------------------------------------------------------------------------
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>For additional commands, e-mail: dev-help@lucene.apache.org

--
Uwe Schindler
H.-H.-Meier-Allee 63, 28213 Bremen
http://www.thetaphi.de

[JENKINS] Lucene-Solr-4.x-MacOSX (64bit/jdk1.6.0) - Build # 469 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-MacOSX/469/
Java: 64bit/jdk1.6.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
REGRESSION:  org.apache.lucene.replicator.http.HttpReplicatorTest.testBasic

Error Message:
Connection to http://localhost:52124 refused

Stack Trace:
org.apache.http.conn.HttpHostConnectException: Connection to http://localhost:52124 refused
	at __randomizedtesting.SeedInfo.seed([E75C887E5C9618BD:4CA6956B834A9E93]:0)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
	at org.apache.lucene.replicator.http.HttpClientBase.executeGET(HttpClientBase.java:178)
	at org.apache.lucene.replicator.http.HttpReplicator.checkForUpdate(HttpReplicator.java:51)
	at org.apache.lucene.replicator.ReplicationClient.doUpdate(ReplicationClient.java:196)
	at org.apache.lucene.replicator.ReplicationClient.updateNow(ReplicationClient.java:402)
	at org.apache.lucene.replicator.http.HttpReplicatorTest.testBasic(HttpReplicatorTest.java:110)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:680)
Caused by: java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
	at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:432)
	at java.net.Socket.connect(Socket.java:529)
	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
	... 48 more




Build Log:
[...truncated 7459 lines...]
[junit4:junit4] Suite: org.apache.lucene.replicator.http.HttpReplicatorTest
[junit4:junit4]   2> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[junit4:junit4]   2> SLF4J: Defaulting to no-operation (NOP) logger implementation
[junit4:junit4]   2> SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HttpReplicatorTest -Dtests.method=testBasic -Dtests.seed=E75C887E5C9618BD -Dtests.slow=true -Dtests.locale=nl_NL -Dtests.timezone=Europe/Monaco -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   3.42s | HttpReplicatorTest.testBasic <<<
[junit4:junit4]    > Throwable #1: org.apache.http.conn.HttpHostConnectException: Connection to http://localhost:52124 refused
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([E75C887E5C9618BD:4CA6956B834A9E93]:0)
[junit4:junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
[junit4:junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4]    > 	at org.apache.lucene.replicator.http.HttpClientBase.executeGET(HttpClientBase.java:178)
[junit4:junit4]    > 	at org.apache.lucene.replicator.http.HttpReplicator.checkForUpdate(HttpReplicator.java:51)
[junit4:junit4]    > 	at org.apache.lucene.replicator.ReplicationClient.doUpdate(ReplicationClient.java:196)
[junit4:junit4]    > 	at org.apache.lucene.replicator.ReplicationClient.updateNow(ReplicationClient.java:402)
[junit4:junit4]    > 	at org.apache.lucene.replicator.http.HttpReplicatorTest.testBasic(HttpReplicatorTest.java:110)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:680)
[junit4:junit4]    > Caused by: java.net.ConnectException: Connection refused
[junit4:junit4]    > 	at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4]    > 	at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
[junit4:junit4]    > 	at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
[junit4:junit4]    > 	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
[junit4:junit4]    > 	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:432)
[junit4:junit4]    > 	at java.net.Socket.connect(Socket.java:529)
[junit4:junit4]    > 	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4]    > 	... 48 more
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=nl_NL, timezone=Europe/Monaco
[junit4:junit4]   2> NOTE: Mac OS X 10.8.3 x86_64/Apple Inc. 1.6.0_45 (64-bit)/cpus=2,threads=1,free=67789600,total=85000192
[junit4:junit4]   2> NOTE: All tests run in this JVM: [SessionTokenTest, IndexRevisionTest, IndexAndTaxonomyRevisionTest, IndexAndTaxonomyReplicationClientTest, IndexReplicationClientTest, LocalReplicatorTest, HttpReplicatorTest]
[junit4:junit4] Completed in 3.50s, 1 test, 1 error <<< FAILURES!

[...truncated 9 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/build.xml:383: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/build.xml:363: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/build.xml:554: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/common-build.xml:1827: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/module-build.xml:61: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/common-build.xml:887: There were test failures: 7 suites, 30 tests, 1 error

Total time: 51 minutes 44 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.6.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure