You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/09/14 19:33:46 UTC

[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 1831 - Still Failing!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1831/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseParallelGC

8 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1
	at __randomizedtesting.SeedInfo.seed([3BDC3FDF1484ADAB:BA3AB1C763DBCD97]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:550)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:583)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:205)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
	at sun.reflect.GeneratedMethodAccessor49.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.cloud.DeleteReplicaTest.testDistribSearch

Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:52466/tlpp/id, http://127.0.0.1:52472/tlpp/id, http://127.0.0.1:52475/tlpp/id, http://127.0.0.1:52481/tlpp/id, http://127.0.0.1:52478/tlpp/id]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:52466/tlpp/id, http://127.0.0.1:52472/tlpp/id, http://127.0.0.1:52475/tlpp/id, http://127.0.0.1:52481/tlpp/id, http://127.0.0.1:52478/tlpp/id]
	at __randomizedtesting.SeedInfo.seed([3BDC3FDF1484ADAB:BA3AB1C763DBCD97]:0)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:322)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.sendRequest(CloudSolrServer.java:874)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:658)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601)
	at org.apache.solr.cloud.DeleteReplicaTest.removeAndWaitForReplicaGone(DeleteReplicaTest.java:139)
	at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:123)
	at org.apache.solr.cloud.DeleteReplicaTest.doTest(DeleteReplicaTest.java:87)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
	at sun.reflect.GeneratedMethodAccessor49.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not  remove replica : delLiveColl/shard2/core_node4
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:550)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:343)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:304)
	... 46 more


FAILED:  org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.testDistribSearch

Error Message:
No live SolrServers available to handle this request:[https://127.0.0.1:52445/u, https://127.0.0.1:52450/u, https://127.0.0.1:52453/u]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:52445/u, https://127.0.0.1:52450/u, https://127.0.0.1:52453/u]
	at __randomizedtesting.SeedInfo.seed([3BDC3FDF1484ADAB:BA3AB1C763DBCD97]:0)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:322)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.sendRequest(CloudSolrServer.java:874)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:658)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601)
	at org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.removeAndWaitForLastReplicaGone(DeleteLastCustomShardedReplicaTest.java:116)
	at org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.doTest(DeleteLastCustomShardedReplicaTest.java:106)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
	at sun.reflect.GeneratedMethodAccessor49.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not  remove replica : customcollreplicadeletion/a/core_node2
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:550)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:343)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:304)
	... 45 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestZkChroot

Error Message:
Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=1487, name=coreLoadExecutor-1085-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below.

Stack Trace:
com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=1487, name=coreLoadExecutor-1085-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below.
	at java.lang.Thread.getStackTrace(Thread.java:1589)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:166)
	at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:731)
	at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:1274)
	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1165)
	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1157)
	at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:40)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:351)
	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:276)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:488)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:794)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.AssertionError: Directory not closed: MockDirectoryWrapper(RAMDirectory@23a61826 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@30c8f23d)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:699)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:696)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:183)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:712)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	... 1 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestZkChroot

Error Message:
Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=1516, name=coreLoadExecutor-1095-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below.

Stack Trace:
com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=1516, name=coreLoadExecutor-1095-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below.
	at java.lang.Thread.getStackTrace(Thread.java:1589)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:166)
	at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:731)
	at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:1274)
	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1165)
	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1157)
	at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:40)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:351)
	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:276)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:488)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:794)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.AssertionError: Directory not closed: MockDirectoryWrapper(RAMDirectory@64920f92 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f61c373)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:699)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:696)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:183)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:712)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	... 1 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestZkChroot

Error Message:
Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=1534, name=coreLoadExecutor-1102-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below.

Stack Trace:
com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=1534, name=coreLoadExecutor-1102-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below.
	at java.lang.Thread.getStackTrace(Thread.java:1589)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:166)
	at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:731)
	at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:1274)
	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1165)
	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1157)
	at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:40)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:351)
	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:276)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:488)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:794)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.AssertionError: Directory not closed: MockDirectoryWrapper(RAMDirectory@722432b5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4b66c31d)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:699)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:696)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:183)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:712)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	... 1 more


FAILED:  org.apache.solr.core.CachingDirectoryFactoryTest.stressTest

Error Message:
Captured an uncaught exception in thread: Thread[id=2843, name=Thread-1031, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=2843, name=Thread-1031, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest]
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path9/11" "write")
	at __randomizedtesting.SeedInfo.seed([3BDC3FDF1484ADAB]:0)
	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
	at java.security.AccessController.checkPermission(AccessController.java:559)
	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
	at sun.nio.fs.UnixPath.checkWrite(UnixPath.java:801)
	at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:376)
	at java.nio.file.Files.createDirectory(Files.java:630)
	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
	at java.nio.file.Files.createDirectories(Files.java:683)
	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)


FAILED:  org.apache.solr.core.RAMDirectoryFactoryTest.test

Error Message:
access denied ("java.io.FilePermission" "/fake/path" "write")

Stack Trace:
java.security.AccessControlException: access denied ("java.io.FilePermission" "/fake/path" "write")
	at __randomizedtesting.SeedInfo.seed([3BDC3FDF1484ADAB:B3880005BA78C053]:0)
	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
	at java.security.AccessController.checkPermission(AccessController.java:559)
	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
	at sun.nio.fs.UnixPath.checkWrite(UnixPath.java:801)
	at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:376)
	at java.nio.file.Files.createDirectory(Files.java:630)
	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
	at java.nio.file.Files.createDirectories(Files.java:683)
	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
	at org.apache.solr.core.RAMDirectoryFactoryTest.dotestOpenReturnsTheSameForSamePath(RAMDirectoryFactoryTest.java:46)
	at org.apache.solr.core.RAMDirectoryFactoryTest.test(RAMDirectoryFactoryTest.java:33)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10611 lines...]
   [junit4] Suite: org.apache.solr.core.RAMDirectoryFactoryTest
   [junit4]   2> 328660 T636 oasc.CachingDirectoryFactory.injectLockFactory WARN No lockType configured for RAMDirectory@21747af5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7fb7d028 assuming 'simple'
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=RAMDirectoryFactoryTest -Dtests.method=test -Dtests.seed=3BDC3FDF1484ADAB -Dtests.slow=true -Dtests.locale=sr__#Latn -Dtests.timezone=BET -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.06s | RAMDirectoryFactoryTest.test <<<
   [junit4]    > Throwable #1: java.security.AccessControlException: access denied ("java.io.FilePermission" "/fake/path" "write")
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([3BDC3FDF1484ADAB:B3880005BA78C053]:0)
   [junit4]    > 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]    > 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]    > 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]    > 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]    > 	at sun.nio.fs.UnixPath.checkWrite(UnixPath.java:801)
   [junit4]    > 	at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:376)
   [junit4]    > 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]    > 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]    > 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]    > 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]    > 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]    > 	at org.apache.solr.core.RAMDirectoryFactoryTest.dotestOpenReturnsTheSameForSamePath(RAMDirectoryFactoryTest.java:46)
   [junit4]    > 	at org.apache.solr.core.RAMDirectoryFactoryTest.test(RAMDirectoryFactoryTest.java:33)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Lucene410: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=sr__#Latn, timezone=BET
   [junit4]   2> NOTE: Mac OS X 10.8.5 x86_64/Oracle Corporation 1.7.0_67 (64-bit)/cpus=2,threads=1,free=107907632,total=239599616
   [junit4]   2> NOTE: All tests run in this JVM: [BadIndexSchemaTest, TestIndexingPerformance, FieldFacetTest, FieldFacetExtrasTest, StandardRequestHandlerTest, OverseerStatusTest, IndexSchemaTest, TestManagedSchema, TestCoreDiscovery, TestRTGBase, DistributedFacetPivotSmallTest, CoreAdminCreateDiscoverTest, OpenExchangeRatesOrgProviderTest, TestDocSet, TestSweetSpotSimilarityFactory, MigrateRouteKeyTest, FunctionTest, FastVectorHighlighterTest, TestExceedMaxTermLength, DirectUpdateHandlerTest, RAMDirectoryFactoryTest]
   [junit4] Completed in 0.07s, 1 test, 1 error <<< FAILURES!

[...truncated 44 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestZkChroot
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestZkChroot-3BDC3FDF1484ADAB-001/init-core-data-001
   [junit4]   2> 642014 T1474 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 642018 T1474 oas.SolrTestCaseJ4.setUp ###Starting testChrootBootstrap
   [junit4]   2> 642020 T1474 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 642021 T1475 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 642122 T1474 oasc.ZkTestServer.run start zk server on port:49636
   [junit4]   2> 642123 T1474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/'
   [junit4]   2> 642191 T1474 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/solr.xml
   [junit4]   2> 642276 T1474 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr
   [junit4]   2> 642276 T1474 oasc.CoreContainer.<init> New CoreContainer 1198499309
   [junit4]   2> 642277 T1474 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/]
   [junit4]   2> 642278 T1474 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 642278 T1474 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 642278 T1474 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 642278 T1474 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 642278 T1474 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 642279 T1474 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 642280 T1474 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 642280 T1474 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 642280 T1474 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 642284 T1474 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=0&connTimeout=0&retry=false
   [junit4]   2> 642286 T1474 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 642286 T1474 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 642286 T1474 oasc.CoreContainer.load Host Name: 
   [junit4]   2> 642286 T1474 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49636/foo/bar
   [junit4]   2> 642287 T1474 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 642287 T1474 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 642288 T1474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 642293 T1481 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72babca7 name:ZooKeeperConnection Watcher:127.0.0.1:49636 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 642293 T1474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 642293 T1474 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 642295 T1474 oascc.SolrZkClient.makePath makePath: /foo/bar
   [junit4]   2> 642646 T1474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 642649 T1483 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@73d2e2ae name:ZooKeeperConnection Watcher:127.0.0.1:49636/foo/bar got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 642650 T1474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 642654 T1474 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 642662 T1474 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 642670 T1474 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 642677 T1474 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 642684 T1474 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 642693 T1474 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 642697 T1474 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/192.168.57.17:8983_solr
   [junit4]   2> 642698 T1474 oascc.SolrZkClient.makePath makePath: /live_nodes/192.168.57.17:8983_solr
   [junit4]   2> 642703 T1474 oascc.SolrZkClient.makePath makePath: /collections
   [junit4]   2> 642708 T1474 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 642713 T1474 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 642718 T1474 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 642722 T1474 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 192.168.57.17:8983_solr
   [junit4]   2> 642722 T1474 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 642728 T1474 oasc.Overseer.start Overseer (id=92452207950888961-192.168.57.17:8983_solr-n_0000000000) starting
   [junit4]   2> 642734 T1474 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 642756 T1474 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 642758 T1485 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 642761 T1474 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 642767 T1474 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 642771 T1474 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 642778 T1474 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr
   [junit4]   2> 642779 T1484 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 642793 T1474 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/
   [junit4]   2> 642795 T1474 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 642795 T1474 oasc.ZkController.bootstrapConf bootstrapping config for 1 cores into ZooKeeper using solr.xml from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/
   [junit4]   2> 642795 T1474 oasc.ZkController.bootstrapConf Uploading directory /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/conf with name collection1 for SolrCore collection1
   [junit4]   2> 642797 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/_rest_managed.json
   [junit4]   2> 642809 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/_schema_analysis_stopwords_english.json
   [junit4]   2> 642816 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/_schema_analysis_synonyms_english.json
   [junit4]   2> 642823 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/admin-extra.html
   [junit4]   2> 642830 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/admin-extra.menu-bottom.html
   [junit4]   2> 642837 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/admin-extra.menu-top.html
   [junit4]   2> 642845 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/clustering/carrot2/kmeans-attributes.xml
   [junit4]   2> 642857 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/clustering/carrot2/lingo-attributes.xml
   [junit4]   2> 642866 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/clustering/carrot2/stc-attributes.xml
   [junit4]   2> 642874 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/currency.xml
   [junit4]   2> 642882 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/elevate.xml
   [junit4]   2> 642890 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/contractions_ca.txt
   [junit4]   2> 642899 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/contractions_fr.txt
   [junit4]   2> 642909 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/contractions_ga.txt
   [junit4]   2> 642919 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/contractions_it.txt
   [junit4]   2> 642928 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/hyphenations_ga.txt
   [junit4]   2> 642937 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stemdict_nl.txt
   [junit4]   2> 642945 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stoptags_ja.txt
   [junit4]   2> 642954 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_ar.txt
   [junit4]   2> 642961 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_bg.txt
   [junit4]   2> 642968 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_ca.txt
   [junit4]   2> 642976 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_ckb.txt
   [junit4]   2> 642984 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_cz.txt
   [junit4]   2> 642992 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_da.txt
   [junit4]   2> 642999 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_de.txt
   [junit4]   2> 643010 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_el.txt
   [junit4]   2> 643018 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_en.txt
   [junit4]   2> 643027 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_es.txt
   [junit4]   2> 643036 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_eu.txt
   [junit4]   2> 643045 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_fa.txt
   [junit4]   2> 643053 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_fi.txt
   [junit4]   2> 643061 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_fr.txt
   [junit4]   2> 643069 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_ga.txt
   [junit4]   2> 643077 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_gl.txt
   [junit4]   2> 643084 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_hi.txt
   [junit4]   2> 643092 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_hu.txt
   [junit4]   2> 643100 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_hy.txt
   [junit4]   2> 643108 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_id.txt
   [junit4]   2> 643116 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_it.txt
   [junit4]   2> 643125 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_ja.txt
   [junit4]   2> 643133 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_lv.txt
   [junit4]   2> 643141 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_nl.txt
   [junit4]   2> 643149 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_no.txt
   [junit4]   2> 643158 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_pt.txt
   [junit4]   2> 643165 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_ro.txt
   [junit4]   2> 643173 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_ru.txt
   [junit4]   2> 643181 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_sv.txt
   [junit4]   2> 643189 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_th.txt
   [junit4]   2> 643197 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_tr.txt
   [junit4]   2> 643206 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/userdict_ja.txt
   [junit4]   2> 643215 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/mapping-FoldToASCII.txt
   [junit4]   2> 643223 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 643231 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/protwords.txt
   [junit4]   2> 643240 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/schema.xml
   [junit4]   2> 643248 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/scripts.conf
   [junit4]   2> 643257 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/solrconfig.xml
   [junit4]   2> 643265 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/spellings.txt
   [junit4]   2> 643273 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/stopwords.txt
   [junit4]   2> 643279 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/synonyms.txt
   [junit4]   2> 643288 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/update-script.js
   [junit4]   2> 643296 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/browse.vm
   [junit4]   2> 643305 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/cluster.vm
   [junit4]   2> 643313 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/cluster_results.vm
   [junit4]   2> 643321 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/debug.vm
   [junit4]   2> 643329 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/did_you_mean.vm
   [junit4]   2> 643337 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/error.vm
   [junit4]   2> 643344 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/facet_fields.vm
   [junit4]   2> 643353 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/facet_pivot.vm
   [junit4]   2> 643361 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/facet_queries.vm
   [junit4]   2> 643370 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/facet_ranges.vm
   [junit4]   2> 643378 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/facets.vm
   [junit4]   2> 643386 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/footer.vm
   [junit4]   2> 643394 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/head.vm
   [junit4]   2> 643402 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/header.vm
   [junit4]   2> 643408 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/hit.vm
   [junit4]   2> 643417 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/hit_grouped.vm
   [junit4]   2> 643425 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/hit_plain.vm
   [junit4]   2> 643434 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/join_doc.vm
   [junit4]   2> 643442 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/jquery.autocomplete.css
   [junit4]   2> 643450 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/jquery.autocomplete.js
   [junit4]   2> 643459 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/layout.vm
   [junit4]   2> 643469 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/main.css
   [junit4]   2> 643478 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/mime_type_lists.vm
   [junit4]   2> 643489 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/pagination_bottom.vm
   [junit4]   2> 643498 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/pagination_top.vm
   [junit4]   2> 643508 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/product_doc.vm
   [junit4]   2> 643517 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/query.vm
   [junit4]   2> 643527 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/query_form.vm
   [junit4]   2> 643536 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/query_group.vm
   [junit4]   2> 643545 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/query_spatial.vm
   [junit4]   2> 643554 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/README.txt
   [junit4]   2> 643564 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/results_list.vm
   [junit4]   2> 643574 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/richtext_doc.vm
   [junit4]   2> 643584 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/suggest.vm
   [junit4]   2> 643593 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/tabs.vm
   [junit4]   2> 643604 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/VM_global_library.vm
   [junit4]   2> 643614 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/xslt/example.xsl
   [junit4]   2> 643625 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/xslt/example_atom.xsl
   [junit4]   2> 643636 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/xslt/example_rss.xsl
   [junit4]   2> 643646 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/xslt/luke.xsl
   [junit4]   2> 643656 T1474 oascc.SolrZkClient.makePath makePath: /configs/collection1/xslt/updateXml.xsl
   [junit4]   2> 643669 T1474 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr
   [junit4]   2> 643673 T1474 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/
   [junit4]   2> 643673 T1474 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 643676 T1487 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 643676 T1487 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 643679 T1487 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 643680 T1488 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 643687 T1484 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "node_name":"192.168.57.17:8983_solr",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "base_url":"http://192.168.57.17:8983/solr",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "roles":null}
   [junit4]   2> 643688 T1484 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 643695 T1488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 644682 T1487 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 644682 T1487 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 644684 T1487 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:collection1
   [junit4]   2> 644684 T1487 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 644689 T1487 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 644692 T1487 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=collection1 specified config exists in ZooKeeper
   [junit4]   2> 644692 T1487 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/'
   [junit4]   2> 644759 T1487 oasc.SolrConfig.initLibs Adding specified lib dirs to ClassLoader
   [junit4]   2> 644763 T1487 oasc.SolrResourceLoader.addToClassLoader WARN No files added to classloader from lib: ../../../contrib/extraction/lib (resolved as: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/../../../contrib/extraction/lib).
   [junit4]   2> 644763 T1487 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/../../../dist).
   [junit4]   2> 644765 T1487 oasc.SolrResourceLoader.addToClassLoader WARN No files added to classloader from lib: ../../../contrib/clustering/lib/ (resolved as: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/../../../contrib/clustering/lib).
   [junit4]   2> 644765 T1487 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/../../../dist).
   [junit4]   2> 644766 T1487 oasc.SolrResourceLoader.addToClassLoader WARN No files added to classloader from lib: ../../../contrib/langid/lib/ (resolved as: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/../../../contrib/langid/lib).
   [junit4]   2> 644767 T1487 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/../../../dist).
   [junit4]   2> 644768 T1487 oasc.SolrResourceLoader.addToClassLoader WARN No files added to classloader from lib: ../../../contrib/velocity/lib (resolved as: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/../../../contrib/velocity/lib).
   [junit4]   2> 644768 T1487 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/../../../dist).
   [junit4]   2> 644816 T1487 oasu.SolrIndexConfig.<init> IndexWriter infoStream solr logging is enabled
   [junit4]   2> 644819 T1487 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 644911 T1487 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 644925 T1487 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 644935 T1487 oass.IndexSchema.readSchema [collection1] Schema name=example
   [junit4]   2> 646026 T1487 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 646041 T1487 oasr.RestManager$Registry.registerManagedResource Registered ManagedResource impl org.apache.solr.rest.schema.analysis.ManagedWordSetResource for path /schema/analysis/stopwords/english
   [junit4]   2> 646046 T1487 oasr.RestManager$Registry.registerManagedResource Registered ManagedResource impl org.apache.solr.rest.schema.analysis.ManagedSynonymFilterFactory$SynonymManager for path /schema/analysis/synonyms/english
   [junit4]   2> 646127 T1487 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 646151 T1487 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 646525 T1487 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 646526 T1487 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 646526 T1487 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/data/
   [junit4]   2> 646526 T1487 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a67ecd0
   [junit4]   2> 646582 T1487 oasc.SolrCore.initListeners [collection1] Added SolrEventListener for newSearcher: org.apache.solr.core.QuerySenderListener{queries=[]}
   [junit4]   2> 646582 T1487 oasc.SolrCore.initListeners [collection1] Added SolrEventListener for firstSearcher: org.apache.solr.core.QuerySenderListener{queries=[{q=static firstSearcher warming in solrconfig.xml}]}
   [junit4]   2> 646583 T1487 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@557175b0
   [junit4]   2> 646583 T1487 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 646584 T1487 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 646584 T1487 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 646584 T1487 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 0 directories currently being tracked
   [junit4]   2> 646584 T1487 oasc.CoreContainer.create ERROR Error creating core [collection1]: access denied ("java.io.FilePermission" "/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/data" "write") org.apache.solr.common.SolrException: access denied ("java.io.FilePermission" "/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/data" "write")
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:890)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/collection1/data" "write")
   [junit4]   2> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]   2> 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]   2> 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]   2> 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]   2> 	at sun.nio.fs.UnixPath.checkWrite(UnixPath.java:801)
   [junit4]   2> 	at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:376)
   [junit4]   2> 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]   2> 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]   2> 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]   2> 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]   2> 	at org.apache.lucene.store.NativeFSLockFactory.<init>(NativeFSLockFactory.java:93)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:502)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:276)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:488)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:794)
   [junit4]   2> 	... 8 more
   [junit4]   2> 
   [junit4]   2> 646591 T1474 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 646596 T1474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 646601 T1491 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fdf82d1 name:ZooKeeperConnection Watcher:127.0.0.1:49636 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 646601 T1474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 646601 T1474 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 646611 T1474 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1198499309
   [junit4]   2> 646612 T1474 oasc.Overseer.close Overseer (id=92452207950888961-192.168.57.17:8983_solr-n_0000000000) closing
   [junit4]   2> 646615 T1484 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 192.168.57.17:8983_solr
   [junit4]   2> 648122 T1492 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:347)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:344)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:344)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:346)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:296)
   [junit4]   2> 
   [junit4]   2> 648122 T1488 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 648126 T1474 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1198499309
   [junit4]   2> 648129 T1474 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:49636 49636
   [junit4]   2> 648139 T1475 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:49636 49636
   [junit4]   2> 648368 T1474 oas.SolrTestCaseJ4.tearDown ###Ending testChrootBootstrap
   [junit4]   2> 648387 T1474 oas.SolrTestCaseJ4.setUp ###Starting testNoBootstrapConf
   [junit4]   2> 648388 T1474 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 648389 T1493 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 648490 T1474 oasc.ZkTestServer.run start zk server on port:49640
   [junit4]   2> 648490 T1474 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 648492 T1474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 648497 T1499 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@59530d2d name:ZooKeeperConnection Watcher:127.0.0.1:49640 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 648498 T1474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 648498 T1474 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 648501 T1474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/'
   [junit4]   2> 648546 T1474 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/solr.xml
   [junit4]   2> 648604 T1474 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr
   [junit4]   2> 648605 T1474 oasc.CoreContainer.<init> New CoreContainer 1330775242
   [junit4]   2> 648605 T1474 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/]
   [junit4]   2> 648606 T1474 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 648606 T1474 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 648606 T1474 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 648606 T1474 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 648607 T1474 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 648607 T1474 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 648607 T1474 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 648607 T1474 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 648608 T1474 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 648611 T1474 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=0&connTimeout=0&retry=false
   [junit4]   2> 648614 T1474 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 648614 T1474 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 648614 T1474 oasc.CoreContainer.load Host Name: 
   [junit4]   2> 648615 T1474 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49640/foo/bar2
   [junit4]   2> 648615 T1474 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 648615 T1474 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 648617 T1474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 648621 T1501 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@25ff363d name:ZooKeeperConnection Watcher:127.0.0.1:49640 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 648621 T1474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 648622 T1474 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 648635 T1474 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:49640 49640
   [junit4]   2> 648650 T1493 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:49640 49640
   [junit4]   2> 648891 T1474 oas.SolrTestCaseJ4.tearDown ###Ending testNoBootstrapConf
   [junit4]   2> 648897 T1474 oas.SolrTestCaseJ4.setUp ###Starting testWithUploadDir
   [junit4]   2> 648899 T1474 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 648900 T1502 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 649001 T1474 oasc.ZkTestServer.run start zk server on port:49643
   [junit4]   2> 649001 T1474 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 649003 T1474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 649009 T1508 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5ceb17f3 name:ZooKeeperConnection Watcher:127.0.0.1:49643 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 649009 T1474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 649010 T1474 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 649013 T1474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/'
   [junit4]   2> 649052 T1474 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/solr.xml
   [junit4]   2> 649102 T1474 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr
   [junit4]   2> 649103 T1474 oasc.CoreContainer.<init> New CoreContainer 673088499
   [junit4]   2> 649103 T1474 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/solr/]
   [junit4]   2> 649105 T1474 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 649105 T1474 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 649106 T1474 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 649106 T1474 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 649107 T1474 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 649108 T1474 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 649108 T1474 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 649108 T1474 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 649108 T1474 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 649113 T1474 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=0&connTimeout=0&retry=false
   [junit4]   2> 649116 T1474 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 649116 T1474 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 649116 T1474 oasc.CoreContainer.load Host Name: 
   [junit4]   2> 649117 T1474 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49643/foo/bar3
   [junit4]   2> 649117 T1474 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 649118 T1474 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 649120 T1474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 649123 T1510 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12b481b2 name:ZooKeeperConnection Watcher:127.0.0.1:49643 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 649124 T1474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 649124 T1474 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 649129 T1474 oascc.SolrZkClient.makePath makePath: /foo/bar3
   [junit4]   2> 649151 T1474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 649156 T1512 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e0b51a0 name:ZooKeeperConnection Watcher:127.0.0.1:49643/foo/bar3 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 649156 T1474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 649161 T1474 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 649172 T1474 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 649181 T1474 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 649190 T1474 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 649197 T1474 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 649205 T1474 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 649209 T1474 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/192.168.57.17:8983_solr
   [junit4]   2> 649211 T1474 oascc.SolrZkClient.makePath makePath: /live_nodes/192.168.57.17:8983_solr
   [junit4]   2> 649218 T1474 oascc.SolrZkClient.makePath makePath: /collections
   [junit4]   2> 649225 T1474 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 649232 T1474 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 649240 T1474 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 649247 T1474 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 192.168.57.17:8983_solr
   [junit4]   2> 649248 T1474 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 649255 T1474 oasc.Overseer.start Overseer (id=92452208401776642-192.168.57.17:8983_solr-n_0000000000) starting
   [junit4]   2> 649264 T1474 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 649294 T1474 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 649297 T1514 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 649299 T1474 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 649308 T1474 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 649315 T1474 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 649325 T1513 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 649326 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/_rest_managed.json
   [junit4]   2> 649339 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/_schema_analysis_stopwords_english.json
   [junit4]   2> 649348 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/_schema_analysis_synonyms_english.json
   [junit4]   2> 649357 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/admin-extra.html
   [junit4]   2> 649365 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/admin-extra.menu-bottom.html
   [junit4]   2> 649374 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/admin-extra.menu-top.html
   [junit4]   2> 649382 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/clustering/carrot2/kmeans-attributes.xml
   [junit4]   2> 649398 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/clustering/carrot2/lingo-attributes.xml
   [junit4]   2> 649409 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/clustering/carrot2/stc-attributes.xml
   [junit4]   2> 649421 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/currency.xml
   [junit4]   2> 649428 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/elevate.xml
   [junit4]   2> 649440 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/contractions_ca.txt
   [junit4]   2> 649453 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/contractions_fr.txt
   [junit4]   2> 649463 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/contractions_ga.txt
   [junit4]   2> 649474 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/contractions_it.txt
   [junit4]   2> 649484 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/hyphenations_ga.txt
   [junit4]   2> 649495 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stemdict_nl.txt
   [junit4]   2> 649509 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stoptags_ja.txt
   [junit4]   2> 649521 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_ar.txt
   [junit4]   2> 649530 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_bg.txt
   [junit4]   2> 649542 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_ca.txt
   [junit4]   2> 649552 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_ckb.txt
   [junit4]   2> 649563 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_cz.txt
   [junit4]   2> 649572 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_da.txt
   [junit4]   2> 649580 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_de.txt
   [junit4]   2> 649590 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_el.txt
   [junit4]   2> 649606 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_en.txt
   [junit4]   2> 649622 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_es.txt
   [junit4]   2> 649635 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_eu.txt
   [junit4]   2> 649657 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_fa.txt
   [junit4]   2> 649667 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_fi.txt
   [junit4]   2> 649677 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_fr.txt
   [junit4]   2> 649687 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_ga.txt
   [junit4]   2> 649697 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_gl.txt
   [junit4]   2> 649706 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_hi.txt
   [junit4]   2> 649715 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_hu.txt
   [junit4]   2> 649725 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_hy.txt
   [junit4]   2> 649735 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_id.txt
   [junit4]   2> 649746 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_it.txt
   [junit4]   2> 649756 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_ja.txt
   [junit4]   2> 649765 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_lv.txt
   [junit4]   2> 649775 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_nl.txt
   [junit4]   2> 649784 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_no.txt
   [junit4]   2> 649794 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_pt.txt
   [junit4]   2> 649804 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_ro.txt
   [junit4]   2> 649814 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_ru.txt
   [junit4]   2> 649823 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_sv.txt
   [junit4]   2> 649832 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_th.txt
   [junit4]   2> 649841 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/stopwords_tr.txt
   [junit4]   2> 649850 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/lang/userdict_ja.txt
   [junit4]   2> 649859 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/mapping-FoldToASCII.txt
   [junit4]   2> 649869 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/mapping-ISOLatin1Accent.txt
   [junit4]   2> 649878 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/protwords.txt
   [junit4]   2> 649885 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/schema.xml
   [junit4]   2> 649894 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/scripts.conf
   [junit4]   2> 649902 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/solrconfig.xml
   [junit4]   2> 649908 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/spellings.txt
   [junit4]   2> 649915 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/stopwords.txt
   [junit4]   2> 649921 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/synonyms.txt
   [junit4]   2> 649927 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/update-script.js
   [junit4]   2> 649935 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/browse.vm
   [junit4]   2> 649945 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/cluster.vm
   [junit4]   2> 649954 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/cluster_results.vm
   [junit4]   2> 649959 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/debug.vm
   [junit4]   2> 649966 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/did_you_mean.vm
   [junit4]   2> 649974 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/error.vm
   [junit4]   2> 649982 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/facet_fields.vm
   [junit4]   2> 649990 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/facet_pivot.vm
   [junit4]   2> 649998 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/facet_queries.vm
   [junit4]   2> 650008 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/facet_ranges.vm
   [junit4]   2> 650018 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/facets.vm
   [junit4]   2> 650028 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/footer.vm
   [junit4]   2> 650037 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/head.vm
   [junit4]   2> 650046 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/header.vm
   [junit4]   2> 650062 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/hit.vm
   [junit4]   2> 650077 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/hit_grouped.vm
   [junit4]   2> 650094 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/hit_plain.vm
   [junit4]   2> 650114 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/join_doc.vm
   [junit4]   2> 650125 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/jquery.autocomplete.css
   [junit4]   2> 650137 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/jquery.autocomplete.js
   [junit4]   2> 650153 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/layout.vm
   [junit4]   2> 650171 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/main.css
   [junit4]   2> 650184 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/mime_type_lists.vm
   [junit4]   2> 650195 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/pagination_bottom.vm
   [junit4]   2> 650204 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/pagination_top.vm
   [junit4]   2> 650215 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/product_doc.vm
   [junit4]   2> 650224 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/query.vm
   [junit4]   2> 650234 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/query_form.vm
   [junit4]   2> 650243 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/query_group.vm
   [junit4]   2> 650253 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/query_spatial.vm
   [junit4]   2> 650262 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/README.txt
   [junit4]   2> 650272 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/results_list.vm
   [junit4]   2> 650282 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/richtext_doc.vm
   [junit4]   2> 650292 T1474 oascc.SolrZkClient.makePath makePath: /configs/testWithUploadDir/velocity/suggest.vm
   [junit4]   2> 650300 T1474 oascc.SolrZkClient.

[...truncated too long message...]

439 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene410: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=es_CO, timezone=Portugal
   [junit4]   2> NOTE: Mac OS X 10.8.5 x86_64/Oracle Corporation 1.7.0_67 (64-bit)/cpus=2,threads=1,free=189505088,total=457703424
   [junit4]   2> NOTE: All tests run in this JVM: [BadIndexSchemaTest, TestIndexingPerformance, FieldFacetTest, FieldFacetExtrasTest, StandardRequestHandlerTest, OverseerStatusTest, IndexSchemaTest, TestManagedSchema, TestCoreDiscovery, TestRTGBase, DistributedFacetPivotSmallTest, CoreAdminCreateDiscoverTest, OpenExchangeRatesOrgProviderTest, TestDocSet, TestSweetSpotSimilarityFactory, MigrateRouteKeyTest, FunctionTest, FastVectorHighlighterTest, TestExceedMaxTermLength, DirectUpdateHandlerTest, RAMDirectoryFactoryTest, TermVectorComponentDistributedTest, ReturnFieldsTest, TestCodecSupport, TestDefaultSimilarityFactory, UUIDUpdateProcessorFallbackTest, ResourceLoaderTest, ConnectionManagerTest, CustomCollectionTest, TestPerFieldSimilarity, IndexBasedSpellCheckerTest, TestRandomMergePolicy, TestOrdValues, UnloadDistributedZkTest, TestZkChroot, TestReRankQParserPlugin, HdfsCollectionsAPIDistributedZkTest, ReplicationFactorTest, TestSolrIndexConfig, TriLevelCompositeIdRoutingTest, TestRealTimeGet, TestRemoteStreaming, AbstractAnalyticsStatsTest, SuggesterFSTTest, TestCharFilters, TestJmxIntegration, QueryParsingTest, HighlighterConfigTest, DocumentBuilderTest, TestCoreContainer, BasicFunctionalityTest, TestSolrJ, TestChildDocTransformer, TestSolrXmlPersistor, CoreAdminRequestStatusTest, ZkSolrClientTest, ExpressionTest, AnalysisErrorHandlingTest, TestFieldTypeResource, SoftAutoCommitTest, DocValuesMultiTest, SolrInfoMBeanTest, DateRangeFieldTest, BlockDirectoryTest, TestDistributedSearch, CursorPagingTest, DistributedFacetPivotLongTailTest, BadCopyFieldTest, SyncSliceTest, HdfsChaosMonkeySafeLeaderTest, TestLazyCores, TestMaxScoreQueryParser, BasicZkTest, SpellCheckComponentTest, ResponseHeaderTest, DistributedFacetPivotLargeTest, TestQueryUtils, TestLMDirichletSimilarityFactory, BinaryUpdateRequestHandlerTest, TestPivotHelperCode, TestNonDefinedSimilarityFactory, TestTrie, DisMaxRequestHandlerTest, ParsingFieldUpdateProcessorsTest, CachingDirectoryFactoryTest, AnalyticsQueryTest, HighlighterMaxOffsetTest, PolyFieldTest, HdfsRecoveryZkTest, PreAnalyzedUpdateProcessorTest, TestPHPSerializedResponseWriter, PeerSyncTest, MergeStrategyTest, TestRestManager, TestShortCircuitedRequests, SearchHandlerTest, TestFieldCollectionResource, TestManagedResource, DistribDocExpirationUpdateProcessorTest, TestBM25SimilarityFactory, AddBlockUpdateTest, TestJoin, TestCSVLoader, XsltUpdateRequestHandlerTest, QueryResultKeyTest, TestCursorMarkWithoutUniqueKey, TestSolrXml, UUIDFieldTest, SimpleFacetsTest, ModifyConfFileTest, DebugComponentTest, TestRecoveryHdfs, SolrCoreTest, TestCSVResponseWriter, SolrIndexConfigTest, ResponseLogComponentTest, SortByFunctionTest, TestManagedSchemaFieldResource, TestFiltering, LeaderElectionTest, TestMiniSolrCloudCluster, SuggestComponentTest, TestClassNameShortening, QueryElevationComponentTest, TestAtomicUpdateErrorCases, TestStressRecovery, DistributedTermsComponentTest, JSONWriterTest, TestSolrQueryParserResource, EchoParamsTest, TestElisionMultitermQuery, TestFieldTypeCollectionResource, FileUtilsTest, StatsComponentTest, MultiThreadedOCPTest, UpdateRequestProcessorFactoryTest, TestCloudPivotFacet, BJQParserTest, TestSearchPerf, AbstractAnalyticsFacetTest, TestMultiCoreConfBootstrap, ClusterStateTest, TestManagedStopFilterFactory, TestHashPartitioner, TestDocBasedVersionConstraints, TestCollationFieldDocValues, HdfsBasicDistributedZkTest, TestManagedSynonymFilterFactory, HardAutoCommitTest, PrimitiveFieldTypeTest, PingRequestHandlerTest, CursorMarkTest, TestLFUCache, TestRequestStatusCollectionAPI, TestLRUCache, SignatureUpdateProcessorFactoryTest, FieldAnalysisRequestHandlerTest, SolrTestCaseJ4Test, NotRequiredUniqueKeyTest, TestPseudoReturnFields, CoreAdminHandlerTest, CacheHeaderTest, TestCollectionAPI, TestHighlightDedupGrouping, TestDistributedMissingSort, TestFoldingMultitermQuery, TestAddFieldRealTimeGet, TestInfoStreamLogging, TestNRTOpen, SharedFSAutoReplicaFailoverUtilsTest, DirectSolrConnectionTest, SolrPluginUtilsTest, TestCloudSchemaless, DistributedExpandComponentTest, OutputWriterTest, TestMissingGroups, PluginInfoTest, TestOverriddenPrefixQueryForCustomFieldType, OverriddenZkACLAndCredentialsProvidersTest, TestModifyConfFiles, TestImplicitCoreProperties, RecoveryZkTest, TestSolrQueryParserDefaultOperatorResource, TestReversedWildcardFilterFactory, AssignTest, NoCacheHeaderTest, TestSystemIdResolver, TestBinaryResponseWriter, DateMathParserTest, DistributedDebugComponentTest, OverseerTest, VMParamsZkACLAndCredentialsProvidersTest, LoggingHandlerTest, IndexSchemaRuntimeFieldTest, QueryFacetTest, SchemaVersionSpecificBehaviorTest, SampleTest, SpellPossibilityIteratorTest, DirectUpdateHandlerOptimizeTest, URLClassifyProcessorTest, TestPhraseSuggestions, TestNoOpRegenerator, TestSolrCoreProperties, HdfsDirectoryTest, TestSchemaSimilarityResource, RankQueryTest, TestCollationField, TestFuzzyAnalyzedSuggestions, BadComponentTest, LukeRequestHandlerTest, TestFunctionQuery, SystemInfoHandlerTest, SpellingQueryConverterTest, NumericFieldsTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest, OpenCloseCoreStressTest, LeaderElectionIntegrationTest, ShardRoutingTest, FullSolrCloudDistribCmdsTest, TestReplicationHandler, ClusterStateUpdateTest, TestRandomFaceting, TestRandomDVFaceting, ZkCLITest, ShardRoutingCustomTest, AutoCommitTest, TestDistributedGrouping, TestFaceting, TestRecovery, DistributedSpellCheckComponentTest, ZkControllerTest, TestStressReorder, TestReload, TestStressVersions, TestRangeQuery, TestSolr4Spatial, TestGroupingSearch, SolrCmdDistributorTest, ConvertedLegacyTest, TestSort, TestBadConfig, OverseerCollectionProcessorTest, TestIndexSearcher, HighlighterTest, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, DistributedQueryElevationComponentTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, SimplePostToolTest, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, SpellCheckCollatorTest, SolrRequestParserTest, DocValuesTest, SuggesterTSTTest, SuggesterTest, TestStressLucene, SpatialFilterTest, SuggesterWFSTTest, WordBreakSolrSpellCheckerTest, SolrCoreCheckLockOnStartupTest, TestUpdate, FieldMutatingUpdateProcessorTest, TestWordDelimiterFilterFactory, QueryEqualityTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, DistanceFunctionTest, TestSolrDeletionPolicy1, TestSurroundQueryParser, TestWriterPerf, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TestQueryTypes, TestOmitPositions, FileBasedSpellCheckerTest, TermsComponentTest, XmlUpdateRequestHandlerTest, RequestHandlersTest, TestValueSourceCache, PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, MoreLikeThisHandlerTest, RequiredFieldsTest, TestSolrQueryParser, TestArbitraryIndexDir, RegexBoostProcessorTest, MBeansHandlerTest, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, JsonLoaderTest, TestPartialUpdateDeduplication, CSVRequestHandlerTest, TestComponentsName, CoreContainerCoreInitFailuresTest, SOLR749Test, TestQuerySenderListener, UpdateParamsTest, AlternateDirectoryTest, TestQuerySenderNoQuery, CopyFieldTest, TestMergePolicyConfig, TestSolrDeletionPolicy2, MultiTermTest, TestBinaryField, MinimalSchemaTest, TestConfig, ExternalFileFieldSortTest, TestPostingsSolrHighlighter, TestLuceneMatchVersion, SynonymTokenizerTest, TestXIncludeConfig, TestDFRSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory, TestFastWriter, LegacyHTMLStripCharFilterTest, TestJmxMonitoredMap, TimeZoneUtilsTest, TestFastOutputStream, ScriptEngineTest, TestFastLRUCache, ChaosMonkeyNothingIsSafeTest, PreAnalyzedFieldTest, PrimUtilsTest, TestSuggestSpellingConverter, DateFieldTest, DOMUtilTest, TestUtils, TestDocumentBuilder, ZkNodePropsTest, SliceStateTest, CircularListTest, DistributedIntervalFacetingTest, TestTolerantSearch, NoFacetTest, RangeFacetTest, AliasIntegrationTest, AsyncMigrateRouteKeyTest, CollectionsAPIAsyncDistributedZkTest, DeleteInactiveReplicaTest, DeleteLastCustomShardedReplicaTest, DeleteReplicaTest]
   [junit4] Completed in 236.62s, 1 test, 1 error <<< FAILURES!

[...truncated 269 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:491: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:471: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:60: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/extra-targets.xml:39: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:189: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:496: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1358: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:961: There were test failures: 438 suites, 1830 tests, 3 suite-level errors, 5 errors, 50 ignored (22 assumptions)

Total time: 182 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 1838 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1838/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1
	at __randomizedtesting.SeedInfo.seed([177DFCBD6E4B9A3:809151D3A1BBD99F]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:550)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:583)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:205)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
	at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11424 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/init-core-data-001
   [junit4]   2> 3304357 T8459 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 3304358 T8459 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /gzlpr/qe
   [junit4]   2> 3304365 T8459 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 3304367 T8459 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3304369 T8460 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 3304377 T8459 oasc.ZkTestServer.run start zk server on port:51398
   [junit4]   2> 3304379 T8459 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 3304384 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3304391 T8466 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@373727ab name:ZooKeeperConnection Watcher:127.0.0.1:51398 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3304391 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3304392 T8459 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 3304393 T8459 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 3304400 T8459 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 3304402 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3304405 T8468 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f735d97 name:ZooKeeperConnection Watcher:127.0.0.1:51398/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3304405 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3304406 T8459 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 3304406 T8459 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 3304413 T8459 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 3304420 T8459 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 3304426 T8459 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 3304434 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 3304435 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 3304445 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 3304447 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 3304456 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3304457 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3304465 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 3304466 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 3304475 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 3304476 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 3304485 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 3304486 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 3304494 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 3304494 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 3304503 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3304506 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 3304514 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3304514 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3304521 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 3304522 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 3304528 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 3304528 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 3304539 T8459 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 3304541 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3304543 T8470 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5634ecfb name:ZooKeeperConnection Watcher:127.0.0.1:51398/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3304544 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3304545 T8459 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> Using old style solr.xml
   [junit4]   2> 3304549 T8459 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 3304550 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3304554 T8472 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@43445b82 name:ZooKeeperConnection Watcher:127.0.0.1:51398 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3304554 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3304555 T8459 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 3304555 T8459 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 3304563 T8459 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 3304565 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3304568 T8474 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d40263e name:ZooKeeperConnection Watcher:127.0.0.1:51398/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3304568 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3304569 T8459 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 3304572 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 3304574 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 3304583 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 3304584 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf2/schema.xml
   [junit4]   2> 3304592 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3304594 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 3304601 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 3304603 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 3304610 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 3304611 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf2/protwords.txt
   [junit4]   2> 3304620 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 3304621 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf2/currency.xml
   [junit4]   2> 3304629 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 3304630 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 3304638 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3304638 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3304647 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 3304648 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 3304658 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 3304659 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 3304669 T8459 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 3304670 T8459 oascc.SolrZkClient.makePath makePath: /configs/conf2/elevate.xml
   [junit4]   2> 3306848 T8459 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 3306858 T8459 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 3306863 T8459 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:51404
   [junit4]   2> 3306867 T8459 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 3306867 T8459 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3306868 T8459 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-002
   [junit4]   2> 3306868 T8459 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-002/'
   [junit4]   2> 3306918 T8459 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-002/solr.xml
   [junit4]   2> 3306930 T8459 oasc.CoreContainer.<init> New CoreContainer 1463402231
   [junit4]   2> 3306930 T8459 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-002/]
   [junit4]   2> 3306931 T8459 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 3306931 T8459 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 3306932 T8459 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 3306933 T8459 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 3306933 T8459 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 3306933 T8459 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 3306934 T8459 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 3306934 T8459 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 3306934 T8459 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 3306936 T8459 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
   [junit4]   2> 3306938 T8459 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3306938 T8459 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3306938 T8459 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 3306939 T8459 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51398/solr
   [junit4]   2> 3306939 T8459 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 3306939 T8459 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 3306941 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3306944 T8485 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62c5cb5f name:ZooKeeperConnection Watcher:127.0.0.1:51398 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3306944 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3306945 T8459 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 3306952 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3306955 T8487 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2440f293 name:ZooKeeperConnection Watcher:127.0.0.1:51398/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3306956 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3306959 T8459 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 3306967 T8459 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 3306974 T8459 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 3306984 T8459 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 3306991 T8459 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 3307001 T8459 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 3307007 T8459 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51404_gzlpr%2Fqe
   [junit4]   2> 3307009 T8459 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51404_gzlpr%2Fqe
   [junit4]   2> 3307018 T8459 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 3307023 T8459 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 3307029 T8459 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 3307035 T8459 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:51404_gzlpr%2Fqe
   [junit4]   2> 3307035 T8459 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 3307043 T8459 oasc.Overseer.start Overseer (id=92480082912542726-127.0.0.1:51404_gzlpr%2Fqe-n_0000000000) starting
   [junit4]   2> 3307052 T8459 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 3307082 T8459 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3307084 T8489 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 3307086 T8459 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 3307093 T8459 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 3307097 T8459 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3307108 T8488 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 3307113 T8459 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 3307113 T8459 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 3307117 T8459 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 3307118 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3307121 T8493 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3dda9c6c name:ZooKeeperConnection Watcher:127.0.0.1:51398/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3307121 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3307121 T8459 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 3307124 T8459 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3307133 T8459 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 3307133 T8459 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2
   [junit4]   2> 3307133 T8459 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 3307135 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3307137 T8495 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e2c0f98 name:ZooKeeperConnection Watcher:127.0.0.1:51398/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3307137 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3307138 T8459 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 3307143 T8496 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3307151 T8488 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: collection1
   [junit4]   2> 3307151 T8488 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 3307152 T8488 oasc.Overseer$ClusterStateUpdater.createCollection state version collection1 2
   [junit4]   2> 3307157 T8488 oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection /collections/collection1/state.json
   [junit4]   2> 3307161 T8497 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3307162 T8496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3309098 T8459 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 3309098 T8459 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 3309105 T8459 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 3309107 T8459 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:51409
   [junit4]   2> 3309110 T8459 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 3309110 T8459 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3309111 T8459 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-003
   [junit4]   2> 3309111 T8459 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-003/'
   [junit4]   2> 3309166 T8459 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-003/solr.xml
   [junit4]   2> 3309188 T8459 oasc.CoreContainer.<init> New CoreContainer 137745080
   [junit4]   2> 3309189 T8459 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-003/]
   [junit4]   2> 3309190 T8459 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 3309190 T8459 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 3309191 T8459 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 3309191 T8459 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 3309192 T8459 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 3309193 T8459 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 3309193 T8459 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 3309194 T8459 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 3309194 T8459 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 3309197 T8459 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
   [junit4]   2> 3309199 T8459 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3309200 T8459 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3309201 T8459 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 3309201 T8459 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51398/solr
   [junit4]   2> 3309202 T8459 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 3309202 T8459 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 3309204 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3309208 T8508 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a51babe name:ZooKeeperConnection Watcher:127.0.0.1:51398 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3309208 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3309209 T8459 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 3309214 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3309217 T8510 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@543aa19b name:ZooKeeperConnection Watcher:127.0.0.1:51398/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3309218 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3309238 T8459 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3310262 T8459 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51409_gzlpr%2Fqe
   [junit4]   2> 3310264 T8459 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51409_gzlpr%2Fqe
   [junit4]   2> 3310279 T8459 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 3310291 T8459 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 3310292 T8459 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 3312595 T8459 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 3312596 T8459 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 3312604 T8459 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 3312608 T8459 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:51412
   [junit4]   2> 3312613 T8459 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 3312614 T8459 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3312615 T8459 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-004
   [junit4]   2> 3312615 T8459 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-004/'
   [junit4]   2> 3312679 T8459 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-004/solr.xml
   [junit4]   2> 3312732 T8459 oasc.CoreContainer.<init> New CoreContainer 856105501
   [junit4]   2> 3312732 T8459 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-004/]
   [junit4]   2> 3312733 T8459 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 3312733 T8459 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 3312733 T8459 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 3312733 T8459 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 3312734 T8459 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 3312734 T8459 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 3312734 T8459 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 3312734 T8459 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 3312736 T8459 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 3312744 T8459 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
   [junit4]   2> 3312750 T8459 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3312751 T8459 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3312752 T8459 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 3312753 T8459 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51398/solr
   [junit4]   2> 3312753 T8459 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 3312754 T8459 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 3312756 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3312760 T8523 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7bbcd596 name:ZooKeeperConnection Watcher:127.0.0.1:51398 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3312760 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3312761 T8459 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 3312774 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3312776 T8525 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a6887e0 name:ZooKeeperConnection Watcher:127.0.0.1:51398/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3312777 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3312808 T8459 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3313830 T8459 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51412_gzlpr%2Fqe
   [junit4]   2> 3313832 T8459 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51412_gzlpr%2Fqe
   [junit4]   2> 3313847 T8459 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 3313862 T8459 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 3313862 T8459 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 3316090 T8459 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 3316092 T8459 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 3316102 T8459 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 3316108 T8459 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:51415
   [junit4]   2> 3316114 T8459 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 3316114 T8459 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3316115 T8459 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-005
   [junit4]   2> 3316116 T8459 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-005/'
   [junit4]   2> 3316159 T8459 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-005/solr.xml
   [junit4]   2> 3316174 T8459 oasc.CoreContainer.<init> New CoreContainer 1466057745
   [junit4]   2> 3316175 T8459 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-005/]
   [junit4]   2> 3316175 T8459 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 3316176 T8459 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 3316176 T8459 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 3316177 T8459 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 3316177 T8459 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 3316177 T8459 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 3316178 T8459 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 3316178 T8459 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 3316179 T8459 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 3316181 T8459 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
   [junit4]   2> 3316182 T8459 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3316183 T8459 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3316183 T8459 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 3316183 T8459 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51398/solr
   [junit4]   2> 3316184 T8459 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 3316184 T8459 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 3316185 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3316188 T8538 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@282288ec name:ZooKeeperConnection Watcher:127.0.0.1:51398 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3316188 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3316189 T8459 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 3316200 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3316212 T8540 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19bde358 name:ZooKeeperConnection Watcher:127.0.0.1:51398/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3316213 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3316242 T8459 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3317267 T8459 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51415_gzlpr%2Fqe
   [junit4]   2> 3317270 T8459 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51415_gzlpr%2Fqe
   [junit4]   2> 3317286 T8459 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 3317298 T8459 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 3317298 T8459 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 3319345 T8459 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
   [junit4]   2> 3319346 T8459 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 3319355 T8459 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 3319357 T8459 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:51418
   [junit4]   2> 3319361 T8459 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 3319362 T8459 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3319362 T8459 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-006
   [junit4]   2> 3319363 T8459 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-006/'
   [junit4]   2> 3319414 T8459 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-006/solr.xml
   [junit4]   2> 3319431 T8459 oasc.CoreContainer.<init> New CoreContainer 1698033424
   [junit4]   2> 3319432 T8459 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-006/]
   [junit4]   2> 3319432 T8459 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 3319433 T8459 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 3319434 T8459 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 3319434 T8459 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 3319434 T8459 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 3319434 T8459 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 3319435 T8459 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 3319435 T8459 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 3319436 T8459 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 3319438 T8459 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
   [junit4]   2> 3319441 T8459 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3319441 T8459 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3319442 T8459 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 3319442 T8459 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51398/solr
   [junit4]   2> 3319442 T8459 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 3319443 T8459 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 3319447 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3319451 T8553 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62dbe5c1 name:ZooKeeperConnection Watcher:127.0.0.1:51398 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3319451 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3319452 T8459 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 3319457 T8459 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3319459 T8555 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6f2d72b4 name:ZooKeeperConnection Watcher:127.0.0.1:51398/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3319459 T8459 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3319480 T8459 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3320499 T8459 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51418_gzlpr%2Fqe
   [junit4]   2> 3320502 T8459 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51418_gzlpr%2Fqe
   [junit4]   2> 3320524 T8459 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 3320537 T8459 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 3320537 T8459 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 3320647 T8501 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&collection.configName=conf1&router.field=myOwnField&name=solrj_collection&action=CREATE&numShards=2&wt=javabin&version=2
   [junit4]   2> 3320652 T8496 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3320658 T8489 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"solrj_collection",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "collection.configName":"conf1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "router.field":"myOwnField"}
   [junit4]   2> 3320660 T8558 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"solrj_collection",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "collection.configName":"conf1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "router.field":"myOwnField"}
   [junit4]   2> 3320666 T8558 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/solrj_collection 
   [junit4]   2> 3320670 T8558 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection
   [junit4]   2> 3320716 T8496 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3320749 T8488 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: solrj_collection
   [junit4]   2> 3320749 T8488 oasc.Overseer$ClusterStateUpdater.createCollection Create collection solrj_collection with shards [shard1, shard2]
   [junit4]   2> 3320750 T8488 oasc.Overseer$ClusterStateUpdater.createCollection state version solrj_collection 1
   [junit4]   2> 3320758 T8497 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 3320758 T8511 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 3320759 T8526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 3320760 T8541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 3320761 T8556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 3320762 T8496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 3320818 T8558 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection solrj_collection, shardNames [shard1, shard2] , replicationFactor : 2
   [junit4]   2> 3320819 T8558 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica1 as part of slice shard1 of collection solrj_collection on 127.0.0.1:51404_gzlpr%2Fqe
   [junit4]   2> 3320823 T8558 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica2 as part of slice shard1 of collection solrj_collection on 127.0.0.1:51418_gzlpr%2Fqe
   [junit4]   2> 3320833 T8558 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica1 as part of slice shard2 of collection solrj_collection on 127.0.0.1:51409_gzlpr%2Fqe
   [junit4]   2> 3320882 T8483 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard1_replica1&action=CREATE&numShards=2&collection=solrj_collection&shard=shard1&wt=javabin&version=2
   [junit4]   2> 3320886 T8558 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica2 as part of slice shard2 of collection solrj_collection on 127.0.0.1:51415_gzlpr%2Fqe
   [junit4]   2> 3320925 T8551 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard1_replica2&action=CREATE&numShards=2&collection=solrj_collection&shard=shard1&wt=javabin&version=2
   [junit4]   2> 3321000 T8551 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica2 state=down collection=solrj_collection
   [junit4]   2> 3321004 T8551 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 3321006 T8532 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard2_replica2&action=CREATE&numShards=2&collection=solrj_collection&shard=shard2&wt=javabin&version=2
   [junit4]   2> 3321017 T8496 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3321022 T8483 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica1 state=down collection=solrj_collection
   [junit4]   2> 3321026 T8505 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard2_replica1&action=CREATE&numShards=2&collection=solrj_collection&shard=shard2&wt=javabin&version=2
   [junit4]   2> 3321031 T8532 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica2 state=down collection=solrj_collection
   [junit4]   2> 3321035 T8505 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica1 state=down collection=solrj_collection
   [junit4]   2> 3321036 T8532 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 3321037 T8505 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 3321039 T8483 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 3321085 T8488 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"solrj_collection_shard1_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:51418/gzlpr/qe",
   [junit4]   2> 	  "node_name":"127.0.0.1:51418_gzlpr%2Fqe",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 3321105 T8488 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"solrj_collection_shard1_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:51404/gzlpr/qe",
   [junit4]   2> 	  "node_name":"127.0.0.1:51404_gzlpr%2Fqe",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 3321114 T8496 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3321127 T8488 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"solrj_collection_shard2_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:51415/gzlpr/qe",
   [junit4]   2> 	  "node_name":"127.0.0.1:51415_gzlpr%2Fqe",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 3321133 T8496 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3321138 T8488 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"solrj_collection_shard2_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:51409/gzlpr/qe",
   [junit4]   2> 	  "node_name":"127.0.0.1:51409_gzlpr%2Fqe",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"solrj_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 3321146 T8496 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3321274 T8511 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 3321274 T8497 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 3321274 T8541 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 3321276 T8556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 3321276 T8526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 3321279 T8496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 3322012 T8551 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica2
   [junit4]   2> 3322014 T8551 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 3322015 T8551 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 3322016 T8551 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 3322018 T8551 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3322018 T8551 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-006/solrj_collection_shard1_replica2/'
   [junit4]   2> 3322040 T8532 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica2
   [junit4]   2> 3322041 T8532 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 3322041 T8483 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica1
   [junit4]   2> 3322042 T8483 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 3322044 T8505 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica1
   [junit4]   2> 3322044 T8505 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
   [junit4]   2> 3322048 T8532 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 3322048 T8532 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 3322052 T8505 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 3322052 T8505 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 3322055 T8505 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3322056 T8505 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-003/solrj_collection_shard2_replica1/'
   [junit4]   2> 3322065 T8532 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3322065 T8532 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-005/solrj_collection_shard2_replica2/'
   [junit4]   2> 3322076 T8483 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 3322076 T8483 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 3322109 T8483 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3322112 T8483 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-002/solrj_collection_shard1_replica1/'
   [junit4]   2> 3322162 T8551 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 3322194 T8551 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3322199 T8551 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 3322206 T8505 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 3322207 T8551 oass.IndexSchema.readSchema [solrj_collection_shard1_replica2] Schema name=test
   [junit4]   2> 3322231 T8532 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 3322241 T8505 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3322281 T8532 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3322286 T8505 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 3322296 T8532 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 3322307 T8483 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 3322321 T8483 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3322323 T8532 oass.IndexSchema.readSchema [solrj_collection_shard2_replica2] Schema name=test
   [junit4]   2> 3322326 T8505 oass.IndexSchema.readSchema [solrj_collection_shard2_replica1] Schema name=test
   [junit4]   2> 3322335 T8483 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 3322344 T8483 oass.IndexSchema.readSchema [solrj_collection_shard1_replica1] Schema name=test
   [junit4]   2> 3322645 T8551 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3322650 T8505 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3322679 T8505 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 3322681 T8505 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 3322686 T8551 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 3322688 T8551 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 3322724 T8532 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3322733 T8532 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 3322735 T8532 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 3322747 T8505 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 3322752 T8551 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 3322753 T8505 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 3322761 T8505 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3322761 T8551 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 3322765 T8505 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 3322766 T8505 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 3322767 T8505 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3322769 T8551 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3322772 T8505 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 3322779 T8505 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 3322779 T8505 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica1' using configuration from collection solrj_collection
   [junit4]   2> 3322780 T8505 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 3322781 T8505 oasc.SolrCore.<init> [solrj_collection_shard2_replica1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-003/solrj_collection_shard2_replica1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-003/solrj_collection_shard2_replica1/data/
   [junit4]   2> 3322781 T8505 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7663b62b
   [junit4]   2> 3322782 T8505 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-003/solrj_collection_shard2_replica1/data
   [junit4]   2> 3322782 T8505 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-003/solrj_collection_shard2_replica1/data/index/
   [junit4]   2> 3322783 T8505 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-003/solrj_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3322774 T8551 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 3322785 T8551 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 3322785 T8551 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3322784 T8505 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-003/solrj_collection_shard2_replica1/data/index
   [junit4]   2> 3322788 T8505 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=31, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 3322789 T8505 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-c8b1528)),segFN=segments_1,generation=1}
   [junit4]   2> 3322790 T8505 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 3322793 T8505 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3322795 T8505 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3322795 T8505 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3322801 T8505 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3322796 T8551 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 3322804 T8551 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 3322804 T8551 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica2' using configuration from collection solrj_collection
   [junit4]   2> 3322805 T8551 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 3322806 T8551 oasc.SolrCore.<init> [solrj_collection_shard1_replica2] Opening new SolrCore at ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-006/solrj_collection_shard1_replica2/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-006/solrj_collection_shard1_replica2/data/
   [junit4]   2> 3322806 T8551 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7663b62b
   [junit4]   2> 3322803 T8505 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3322809 T8551 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-006/solrj_collection_shard1_replica2/data
   [junit4]   2> 3322810 T8551 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-006/solrj_collection_shard1_replica2/data/index/
   [junit4]   2> 3322810 T8551 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica2] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-006/solrj_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3322811 T8551 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-006/solrj_collection_shard1_replica2/data/index
   [junit4]   2> 3322812 T8551 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=31, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 3322813 T8551 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-78bd2325)),segFN=segments_1,generation=1}
   [junit4]   2> 3322815 T8551 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 3322810 T8505 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3322817 T8505 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3322818 T8505 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3322818 T8505 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3322819 T8505 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3322819 T8505 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3322820 T8505 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3322820 T8551 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3322822 T8551 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 3322805 T8483 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 3322827 T8532 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 3322823 T8551 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 3322829 T8551 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3322829 T8551 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 3322829 T8551 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 3322830 T8551 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3322831 T8551 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 3322831 T8551 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3322832 T8551 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3322833 T8551 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3322833 T8551 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 3322834 T8551 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 3322835 T8551 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 3322835 T8551 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 3322835 T8551 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 3322836 T8551 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 3322839 T8551 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 3322820 T8505 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 3322844 T8505 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 3322844 T8505 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 3322845 T8505 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 3322846 T8505 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 3322847 T8505 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 3322849 T8505 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 3322850 T8505 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 3322827 T8483 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 3322855 T8483 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 3322845 T8551 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 3322844 T8532 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 3322865 T8551 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 3322869 T8551 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3322872 T8551 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3322875 T8505 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3322878 T8505 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3322880 T8532 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3322881 T8505 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3322883 T8532 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 3322884 T8532 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 3322885 T8532 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 3322886 T8551 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3322889 T8532 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 3322889 T8532 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 3322889 T8551 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3322890 T8532 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica2' using configuration from collection solrj_collection
   [junit4]   2> 3322894 T8532 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 3322894 T8532 oasc.SolrCore.<init> [solrj_collection_shard2_replica2] Opening new SolrCore at ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-005/solrj_collection_shard2_replica2/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-005/solrj_collection_shard2_replica2/data/
   [junit4]   2> 3322895 T8532 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7663b62b
   [junit4]   2> 3322896 T8551 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 3322896 T8551 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 3322896 T8532 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-005/solrj_collection_shard2_replica2/data
   [junit4]   2> 3322899 T8532 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-005/solrj_collection_shard2_replica2/data/index/
   [junit4]   2> 3322900 T8532 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica2] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-005/solrj_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3322901 T8532 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001/tempDir-005/solrj_collection_shard2_replica2/data/index
   [junit4]   2> 3322901 T8532 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=31, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 3322902 T8532 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-2bd81d60)),segFN=segments_1,generation=1}
   [junit4]   2> 3322903 T8532 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 3322897 T8551 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=83.6044921875, floorSegmentMB=0.560546875, forceMergeDeletesPctAllowed=17.955457698485574, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4779036779837921
   [junit4]   2> 3322908 T8551 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-78bd2325)),segFN=segments_1,generation=1}
   [junit4]   2> 3322908 T8551 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 3322909 T8551 oass.SolrIndexSearcher.<init> Opening Searcher@768773da[solrj_collection_shard1_replica2] main
   [junit4]   2> 3322909 T8551 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 3322898 T8505 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3322916 T8551 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3322918 T8551 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3322920 T8551 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3322921 T8551 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 3322922 T8551 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3322926 T8505 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 3322927 T8505 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 3322928 T8505 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=83.6044921875, floorSegmentMB=0.560546875, forceMergeDeletesPctAllowed=17.955457698485574, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4779036779837921
   [junit4]   2> 3322929 T8551 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3322929 T8551 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3322931 T8551 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
   [junit4]   2> 3322938 T8532 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 3322939 T8505 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-c8b1528)),segFN=segments_1,generation=1}
   [junit4]   2> 3322940 T8505 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 3322941 T8505 oass.SolrIndexSearcher.<init> Opening Searcher@408f507b[solrj_collection_shard2_replica1] main
   [junit4]   2> 3322941 T8505 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
   [junit4]   2> 3322939 T8551 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3322943 T8551 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3322943 T8551 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 3322946 T8505 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 3322946 T8505 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 3322949 T8505 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 3322949 T8505 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 3322950 T8505 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 3322951 T8483 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 3322954 T8505 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
   [junit4]   2> 3322954 T8505 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/

[...truncated too long message...]

.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:344)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:383)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:126)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:333)
   [junit4]   2> 
   [junit4]   2> 4080309 T8459 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/gzlpr/qe,null}
   [junit4]   2> 4085300 T8459 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 4085304 T8459 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51398 51398
   [junit4]   2> 4085324 T8460 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51398 51398
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=177DFCBD6E4B9A3 -Dtests.slow=true -Dtests.locale=fr_FR -Dtests.timezone=Australia/North -Dtests.file.encoding=UTF-8
   [junit4] ERROR    781s | CollectionsAPIDistributedZkTest.testDistribSearch <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([177DFCBD6E4B9A3:809151D3A1BBD99F]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:550)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:583)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:205)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 4085683 T8459 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-177DFCBD6E4B9A3-001
   [junit4]   2> 781354 T8458 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene410: {other_tl1=PostingsFormat(name=Memory doPackFST= false), _version_=PostingsFormat(name=Memory doPackFST= false), multiDefault=PostingsFormat(name=SimpleText), a_t=PostingsFormat(name=Memory doPackFST= false), intDefault=PostingsFormat(name=Memory doPackFST= false), a_i1=PostingsFormat(name=Memory doPackFST= false), id=PostingsFormat(name=MockRandom), text=PostingsFormat(name=Lucene41VarGapFixedInterval), range_facet_l=PostingsFormat(name=Memory doPackFST= false), timestamp=PostingsFormat(name=Memory doPackFST= false)}, docValues:{timestamp=DocValuesFormat(name=Lucene410)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=fr_FR, timezone=Australia/North
   [junit4]   2> NOTE: Mac OS X 10.8.5 x86_64/Oracle Corporation 1.8.0_20 (64-bit)/cpus=2,threads=1,free=87921712,total=342777856
   [junit4]   2> NOTE: All tests run in this JVM: [TestReRankQParserPlugin, TestManagedSchemaFieldResource, TestUniqueKeyFieldResource, OverseerTest, DocValuesMultiTest, TestGroupingSearch, ChaosMonkeySafeLeaderTest, TestUpdate, TestChildDocTransformer, TestCollationFieldDocValues, TestFiltering, DirectUpdateHandlerTest, TestSearcherReuse, ReturnFieldsTest, MinimalSchemaTest, PluginInfoTest, DeleteLastCustomShardedReplicaTest, TestMultiCoreConfBootstrap, DocumentAnalysisRequestHandlerTest, TestSolrJ, CoreAdminRequestStatusTest, TestPerFieldSimilarity, QueryParsingTest, AlternateDirectoryTest, BasicFunctionalityTest, DocValuesTest, HighlighterMaxOffsetTest, UpdateParamsTest, TestDocSet, HdfsSyncSliceTest, InfoHandlerTest, FullSolrCloudDistribCmdsTest, SuggesterTSTTest, SSLMigrationTest, FileUtilsTest, ZkSolrClientTest, SpatialFilterTest, TermVectorComponentDistributedTest, HdfsBasicDistributedZk2Test, TestCopyFieldCollectionResource, CurrencyFieldOpenExchangeTest, UUIDFieldTest, NumericFieldsTest, StandardRequestHandlerTest, TestIntervalFaceting, TestCursorMarkWithoutUniqueKey, TestWriterPerf, DeleteShardTest, TestBM25SimilarityFactory, TestDistributedMissingSort, HdfsLockFactoryTest, CustomCollectionTest, TestElisionMultitermQuery, TestLeaderElectionZkExpiry, TriLevelCompositeIdRoutingTest, HighlighterConfigTest, OpenCloseCoreStressTest, TestPhraseSuggestions, TestLazyCores, TestSolr4Spatial2, TestLMDirichletSimilarityFactory, SolrIndexSplitterTest, TestSolrQueryParserResource, DeleteReplicaTest, FieldAnalysisRequestHandlerTest, SolrCoreCheckLockOnStartupTest, TestPartialUpdateDeduplication, TestFieldTypeResource, DocumentBuilderTest, TestIndexSearcher, TestPostingsSolrHighlighter, TestPHPSerializedResponseWriter, HttpPartitionTest, ShardRoutingCustomTest, TestFuzzyAnalyzedSuggestions, TestDynamicFieldResource, TestRealTimeGet, RequiredFieldsTest, BasicDistributedZkTest, TestRandomDVFaceting, PrimUtilsTest, TestBadConfig, TestCloudSchemaless, TestExceedMaxTermLength, SoftAutoCommitTest, TestCollapseQParserPlugin, SpellPossibilityIteratorTest, TestSuggestSpellingConverter, TestQueryUtils, TestCloudManagedSchemaConcurrent, RecoveryZkTest, SortByFunctionTest, OverseerRolesTest, TestPivotHelperCode, ShardRoutingTest, TestAtomicUpdateErrorCases, TestSchemaNameResource, CoreContainerCoreInitFailuresTest, TestInitQParser, XmlUpdateRequestHandlerTest, ChangedSchemaMergeTest, TestCloudManagedSchema, TestMaxScoreQueryParser, XsltUpdateRequestHandlerTest, TestSolrDeletionPolicy2, TestWordDelimiterFilterFactory, TestOmitPositions, TestSolr4Spatial, ChaosMonkeyNothingIsSafeTest, JSONWriterTest, TestManagedResource, TestOrdValues, ModifyConfFileTest, TestBinaryResponseWriter, TestJmxIntegration, TestFoldingMultitermQuery, TestFieldTypeCollectionResource, TestLuceneMatchVersion, RollingRestartTest, RAMDirectoryFactoryTest, DistributedSuggestComponentTest, ZkCLITest, TestRawResponseWriter, TestBinaryField, SolrIndexConfigTest, TestStandardQParsers, TestComponentsName, TestCSVResponseWriter, TestStressVersions, TestUtils, TestQueryTypes, SolrCmdDistributorTest, TestRemoteStreaming, TestAnalyzedSuggestions, TestSort, SliceStateTest, AddSchemaFieldsUpdateProcessorFactoryTest, QueryResultKeyTest, TestSchemaResource, DistributedDebugComponentTest, HardAutoCommitTest, TestCustomSort, HdfsDirectoryTest, SolrInfoMBeanTest, TestFastOutputStream, TestStressLucene, TestSolrXmlPersistence, TestCloudPivotFacet, TestSearchPerf, TestRestManager, CircularListTest, IndexSchemaRuntimeFieldTest, DistributedQueryElevationComponentTest, DistributedTermsComponentTest, OverseerStatusTest, DirectSolrSpellCheckerTest, RequestHandlersTest, IndexBasedSpellCheckerTest, HighlighterTest, BadIndexSchemaTest, AssignTest, TestShortCircuitedRequests, SchemaVersionSpecificBehaviorTest, TestSortingResponseWriter, DateMathParserTest, TestCodecSupport, ZkControllerTest, PeerSyncTest, SolrTestCaseJ4Test, TestNonNRTOpen, TestRandomFaceting, TestImplicitCoreProperties, TestCloudInspectUtil, DOMUtilTest, TestSweetSpotSimilarityFactory, NoCacheHeaderTest, SyncSliceTest, PreAnalyzedFieldTest, TestFieldSortValues, RemoteQueryErrorTest, DistributedQueryComponentCustomSortTest, HdfsUnloadDistributedZkTest, TestArbitraryIndexDir, TestSolrCoreProperties, TestCollectionAPI, ExternalFileFieldSortTest, QueryElevationComponentTest, TestManagedSchema, SignatureUpdateProcessorFactoryTest, UpdateRequestProcessorFactoryTest, TestMissingGroups, PingRequestHandlerTest, TestQuerySenderNoQuery, HdfsRecoveryZkTest, TestFastLRUCache, HdfsWriteToMultipleCollectionsTest, TestRandomMergePolicy, TestManagedSynonymFilterFactory, SystemInfoHandlerTest, TestValueSourceCache, TestStressReorder, TestCSVLoader, ClusterStateTest, DateRangeFieldTest, CoreAdminHandlerTest, CachingDirectoryFactoryTest, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed in 781.65s, 1 test, 1 error <<< FAILURES!

[...truncated 660 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:491: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:471: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:60: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/extra-targets.xml:39: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:189: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:496: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1358: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:961: There were test failures: 429 suites, 1772 tests, 1 error, 50 ignored (22 assumptions)

Total time: 163 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 1837 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1837/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

All tests passed

Build Log:
[...truncated 70576 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:491: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:78: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:548: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:554: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1959: Tidy was unable to process file /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/docs/solr-core/org/apache/solr/core/InitParams.html, 1 returned.

Total time: 223 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 1836 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1836/
Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseG1GC

All tests passed

Build Log:
[...truncated 60132 lines...]
-documentation-lint:
    [jtidy] Checking for broken html (such as invalid tags)...
   [delete] Deleting directory /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/jtidy_tmp
     [echo] Checking for broken links...
     [exec] 
     [exec] Crawl/parse...
     [exec] 
     [exec] Verify...
     [echo] Checking for malformed docs...
     [exec] 
     [exec] /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/docs/solr-analytics/overview-summary.html
     [exec]   missing: org.apache.solr.handler.component
     [exec] 
     [exec] Missing javadocs were found!

BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:491: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:78: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:548: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:564: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:2471: exec returned: 1

Total time: 228 minutes 42 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 1835 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1835/
Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseG1GC

All tests passed

Build Log:
[...truncated 55618 lines...]
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] Loading source files for package org.apache.solr.analytics.accumulator...
  [javadoc] Loading source files for package org.apache.solr.analytics.accumulator.facet...
  [javadoc] Loading source files for package org.apache.solr.analytics.expression...
  [javadoc] Loading source files for package org.apache.solr.analytics.plugin...
  [javadoc] Loading source files for package org.apache.solr.analytics.request...
  [javadoc] Loading source files for package org.apache.solr.analytics.statistics...
  [javadoc] Loading source files for package org.apache.solr.analytics.util...
  [javadoc] Loading source files for package org.apache.solr.analytics.util.valuesource...
  [javadoc] Loading source files for package org.apache.solr.handler.component...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 1.7.0_67
  [javadoc] Building tree for all the packages and classes...
  [javadoc] Building index for all the packages and classes...
  [javadoc] javadoc: error - Error while reading file /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/contrib/analytics/src/java/overview.html
  [javadoc] Building index for all classes...
  [javadoc] Generating /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/docs/solr-analytics/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal, @lucene.experimental
  [javadoc] 1 error

BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:491: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:78: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:548: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:463: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:440: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:243: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:291: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:2001: Javadoc returned 1

Total time: 226 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 1834 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1834/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.client.solrj.TestLBHttpSolrServer.testReliability

Error Message:
No live SolrServers available to handle this request

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request
	at __randomizedtesting.SeedInfo.seed([2E5BBD83E82DC359:EF9360C5494B12F0]:0)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:528)
	at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
	at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
	at org.apache.solr.client.solrj.TestLBHttpSolrServer.testReliability(TestLBHttpSolrServer.java:223)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:53542/solr
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:481)
	... 43 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:150)
	at java.net.SocketInputStream.read(SocketInputStream.java:121)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:682)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:448)
	... 46 more




Build Log:
[...truncated 12317 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.TestLBHttpSolrServer
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/init-core-data-001
   [junit4]   2> 15029 T31 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 15036 T31 oas.SolrTestCaseJ4.setUp ###Starting testReliability
   [junit4]   2> 15719 T31 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 16110 T31 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53538
   [junit4]   2> 16195 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 16209 T31 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 16209 T31 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001
   [junit4]   2> 16209 T31 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/'
   [junit4]   2> 16252 T31 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/solr.xml
   [junit4]   2> 16309 T31 oasc.CoreContainer.<init> New CoreContainer 147469593
   [junit4]   2> 16310 T31 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/]
   [junit4]   2> 16427 T31 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 16427 T31 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 16428 T31 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 16428 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 16430 T31 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 16430 T31 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 16430 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 16430 T31 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 16431 T31 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 16438 T31 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 16439 T31 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 16439 T31 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 16439 T31 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 16462 T43 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/'
   [junit4]   2> 16543 T43 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 16578 T43 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 16584 T43 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 16589 T43 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 16620 T43 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 16621 T43 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/
   [junit4]   2> 16621 T43 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockFSDirectoryFactory
   [junit4]   2> 16624 T43 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/
   [junit4]   2> 16624 T43 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 16682 T43 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data
   [junit4]   2> 16684 T43 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index/
   [junit4]   2> 16685 T43 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 16706 T43 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index
   [junit4]   2> 16734 T43 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 16735 T43 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 16738 T43 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 16739 T43 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 16740 T43 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 16740 T43 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 16741 T43 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 16741 T43 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 16742 T43 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 16746 T43 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 16747 T43 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 16840 T43 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 16847 T43 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 16849 T43 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 16851 T43 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 16852 T43 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 16854 T43 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 16854 T43 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 16911 T43 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 16911 T43 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 16912 T43 oass.SolrIndexSearcher.<init> Opening Searcher@4be867ad[collection1] main
   [junit4]   2> 16913 T43 oasr.ManagedResourceStorage$FileStorageIO.configure File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/conf
   [junit4]   2> 16913 T43 oasr.RestManager.init Initializing RestManager with initArgs: {storageDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/conf}
   [junit4]   2> 16913 T43 oasr.ManagedResourceStorage.load Reading _rest_managed.json using file:dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/conf
   [junit4]   2> 16914 T43 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 16916 T43 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using file:dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/conf
   [junit4]   2> 16916 T43 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 16937 T43 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 16937 T43 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 16937 T44 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4be867ad[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 16941 T31 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0
   [junit4]   2> 16941 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> ASYNC  NEW_CORE C0 name=collection1 org.apache.solr.core.SolrCore@701f8f5b
   [junit4]   2> 19477 T41 C0 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]} 0 1119
   [junit4]   2> ASYNC  NEW_CORE C1 name=collection1 org.apache.solr.core.SolrCore@701f8f5b
   [junit4]   2> 19654 T40 C1 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 20288 T40 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index),segFN=segments_2,generation=2}
   [junit4]   2> 20290 T40 C1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> ASYNC  NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@701f8f5b
   [junit4]   2> 20541 T40 C2 oass.SolrIndexSearcher.<init> Opening Searcher@7a345714[collection1] main
   [junit4]   2> 20547 T40 C2 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 20557 T44 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7a345714[collection1] main{UninvertingDirectoryReader(Uninverting(_0(5.0.0):c10))}
   [junit4]   2> 20558 T40 C2 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} {commit=} 0 905
   [junit4]   2> 20578 T31 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 20586 T31 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53540
   [junit4]   2> 20587 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 20588 T31 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 20588 T31 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001
   [junit4]   2> 20589 T31 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/'
   [junit4]   2> 20638 T31 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/solr.xml
   [junit4]   2> 20677 T31 oasc.CoreContainer.<init> New CoreContainer 510145216
   [junit4]   2> 20678 T31 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/]
   [junit4]   2> 20683 T31 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 20683 T31 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 20684 T31 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 20684 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 20684 T31 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 20685 T31 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 20686 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 20686 T31 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 20687 T31 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 20687 T31 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 20688 T31 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 20688 T31 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 20689 T31 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 20697 T55 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/'
   [junit4]   2> 20735 T55 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 20752 T55 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 20754 T55 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 20762 T55 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 20797 T55 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 20797 T55 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/
   [junit4]   2> 20798 T55 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockFSDirectoryFactory
   [junit4]   2> 20799 T55 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/
   [junit4]   2> 20800 T55 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 20805 T55 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data
   [junit4]   2> 20807 T55 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index/
   [junit4]   2> 20807 T55 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 20811 T55 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index
   [junit4]   2> 20826 T55 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 20826 T55 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 20854 T55 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 20855 T55 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 20855 T55 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 20855 T55 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 20856 T55 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 20856 T55 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 20856 T55 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 20857 T55 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 20858 T55 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 20864 T55 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 20888 T55 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 20890 T55 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 20901 T55 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 20902 T55 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 20904 T55 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 20905 T55 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 20912 T55 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 20913 T55 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 20914 T55 oass.SolrIndexSearcher.<init> Opening Searcher@689ec4fe[collection1] main
   [junit4]   2> 20914 T55 oasr.ManagedResourceStorage$FileStorageIO.configure File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/conf
   [junit4]   2> 20914 T55 oasr.RestManager.init Initializing RestManager with initArgs: {storageDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/conf}
   [junit4]   2> 20915 T55 oasr.ManagedResourceStorage.load Reading _rest_managed.json using file:dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/conf
   [junit4]   2> 20916 T55 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 20917 T55 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using file:dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/conf
   [junit4]   2> 20918 T55 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 20918 T55 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 20919 T56 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@689ec4fe[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 20920 T55 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 20923 T31 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0
   [junit4]   2> 20924 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> ASYNC  NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@39299d2f
   [junit4]   2> 20938 T49 C3 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]} 0 7
   [junit4]   2> 20944 T50 C3 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> ASYNC  NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@39299d2f
   [junit4]   2> 20996 T50 C4 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index),segFN=segments_2,generation=2}
   [junit4]   2> 21000 T50 C4 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 21043 T50 C4 oass.SolrIndexSearcher.<init> Opening Searcher@462b09df[collection1] main
   [junit4]   2> 21043 T50 C4 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 21045 T56 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@462b09df[collection1] main{UninvertingDirectoryReader(Uninverting(_0(5.0.0):c10))}
   [junit4]   2> 21048 T50 C4 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} {commit=} 0 104
   [junit4]   2> 21069 T31 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 21076 T31 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53542
   [junit4]   2> 21078 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 21078 T31 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 21078 T31 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001
   [junit4]   2> 21078 T31 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/'
   [junit4]   2> 21134 T31 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/solr.xml
   [junit4]   2> 21166 T31 oasc.CoreContainer.<init> New CoreContainer 153560465
   [junit4]   2> 21166 T31 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/]
   [junit4]   2> 21168 T31 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 21168 T31 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 21169 T31 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 21169 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 21169 T31 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 21169 T31 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 21170 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 21170 T31 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 21171 T31 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 21171 T31 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 21173 T31 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 21174 T31 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 21174 T31 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 21182 T67 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/'
   [junit4]   2> 21217 T67 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 21243 T67 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 21244 T67 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 21260 T67 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 21268 T67 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 21269 T67 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/
   [junit4]   2> 21269 T67 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockFSDirectoryFactory
   [junit4]   2> 21270 T67 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/
   [junit4]   2> 21270 T67 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 21289 T67 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data
   [junit4]   2> 21298 T67 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index/
   [junit4]   2> 21300 T67 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 21302 T67 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index
   [junit4]   2> 21308 T67 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 21309 T67 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 21313 T67 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 21313 T67 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 21313 T67 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 21314 T67 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 21314 T67 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 21315 T67 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 21315 T67 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 21315 T67 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 21316 T67 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 21316 T67 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 21354 T67 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 21356 T67 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 21357 T67 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 21359 T67 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 21360 T67 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 21361 T67 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 21393 T67 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 21393 T67 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 21394 T67 oass.SolrIndexSearcher.<init> Opening Searcher@3f33b98c[collection1] main
   [junit4]   2> 21395 T67 oasr.ManagedResourceStorage$FileStorageIO.configure File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/conf
   [junit4]   2> 21395 T67 oasr.RestManager.init Initializing RestManager with initArgs: {storageDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/conf}
   [junit4]   2> 21395 T67 oasr.ManagedResourceStorage.load Reading _rest_managed.json using file:dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/conf
   [junit4]   2> 21396 T67 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 21397 T67 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using file:dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/conf
   [junit4]   2> 21397 T67 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 21398 T67 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 21398 T67 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 21398 T68 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3f33b98c[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 21400 T31 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0
   [junit4]   2> 21400 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> ASYNC  NEW_CORE C5 name=collection1 org.apache.solr.core.SolrCore@76ad079c
   [junit4]   2> 21413 T61 C5 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]} 0 9
   [junit4]   2> 21428 T62 C5 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 21469 T62 C5 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index),segFN=segments_2,generation=2}
   [junit4]   2> 21471 T62 C5 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 21482 T62 C5 oass.SolrIndexSearcher.<init> Opening Searcher@fdea7f8[collection1] main
   [junit4]   2> 21483 T68 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@fdea7f8[collection1] main{UninvertingDirectoryReader(Uninverting(_0(5.0.0):c10))}
   [junit4]   2> 21484 T62 C5 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 21484 T62 C5 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} {commit=} 0 58
   [junit4]   2> 21503 T31 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=510145216
   [junit4]   2> 21509 T31 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@39299d2f
   [junit4]   2> 21511 T31 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=10,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 21516 T31 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 21516 T31 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 21516 T31 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 21526 T31 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 21536 T31 oasc.CachingDirectoryFactory.close Closing MockFSDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 21537 T31 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index [CachedDir<<refCount=0;path=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index;done=false>>]
   [junit4]   2> 21537 T31 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index
   [junit4]   2> 21968 T31 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data [CachedDir<<refCount=0;path=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data;done=false>>]
   [junit4]   2> 21969 T31 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data
   [junit4]   2> 21973 T31 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> 27085 T31 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=147469593
   [junit4]   2> 27090 T31 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> ASYNC  NEW_CORE C6 name=collection1 org.apache.solr.core.SolrCore@76ad079c
   [junit4]   2> 27206 T65 C6 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=10 status=0 QTime=391 
   [junit4]   2> ASYNC  NEW_CORE C7 name=collection1 org.apache.solr.core.SolrCore@701f8f5b
   [junit4]   2> 27207 T41 C7 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=10 status=0 QTime=672 
   [junit4]   2> 27269 T41 C7 oasc.SolrException.log ERROR null:org.eclipse.jetty.io.EofException
   [junit4]   2> 		at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:142)
   [junit4]   2> 		at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:107)
   [junit4]   2> 		at org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:214)
   [junit4]   2> 		at org.apache.solr.common.util.FastOutputStream.flushBuffer(FastOutputStream.java:207)
   [junit4]   2> 		at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:98)
   [junit4]   2> 		at org.apache.solr.response.BinaryResponseWriter.write(BinaryResponseWriter.java:52)
   [junit4]   2> 		at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:50)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:771)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:424)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:202)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
   [junit4]   2> 		at org.eclipse.jetty.server.Server.handle(Server.java:368)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
   [junit4]   2> 		at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
   [junit4]   2> 		at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
   [junit4]   2> 		at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 27271 T41 C7 oasc.SolrException.log ERROR null:org.eclipse.jetty.io.EofException
   [junit4]   2> 		at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:142)
   [junit4]   2> 		at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:107)
   [junit4]   2> 		at org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:214)
   [junit4]   2> 		at org.apache.solr.common.util.FastOutputStream.flushBuffer(FastOutputStream.java:207)
   [junit4]   2> 		at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:98)
   [junit4]   2> 		at org.apache.solr.response.BinaryResponseWriter.write(BinaryResponseWriter.java:52)
   [junit4]   2> 		at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:50)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:771)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:424)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:202)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
   [junit4]   2> 		at org.eclipse.jetty.server.Server.handle(Server.java:368)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
   [junit4]   2> 		at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
   [junit4]   2> 		at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
   [junit4]   2> 		at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 27297 T41 C7 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@701f8f5b
   [junit4]   2> 27298 T41 C7 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=10,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 27301 T41 C7 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 27302 T41 C7 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 27304 T41 C7 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 27343 T41 C7 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 27347 T41 C7 oasc.CachingDirectoryFactory.close Closing MockFSDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 27347 T41 C7 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data [CachedDir<<refCount=0;path=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data;done=false>>]
   [junit4]   2> 27348 T41 C7 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data
   [junit4]   2> 27350 T41 C7 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index [CachedDir<<refCount=0;path=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index;done=false>>]
   [junit4]   2> 27350 T41 C7 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index
   [junit4]   2> 27508 T61 C6 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=*:*&distrib=false&sort=_docid_+asc&rows=0&wt=javabin&version=2} hits=10 status=0 QTime=138 
   [junit4]   2> 32103 T31 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=153560465
   [junit4]   2> 32105 T31 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@76ad079c
   [junit4]   2> 32106 T31 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=10,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 32108 T31 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 32108 T31 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 32109 T31 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 32117 T31 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 32122 T31 oasc.CachingDirectoryFactory.close Closing MockFSDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 32122 T31 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data [CachedDir<<refCount=0;path=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data;done=false>>]
   [junit4]   2> 32122 T31 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data
   [junit4]   2> 32123 T31 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index [CachedDir<<refCount=0;path=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index;done=false>>]
   [junit4]   2> 32124 T31 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index
   [junit4]   2> 32159 T31 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> 37107 T31 oas.SolrTestCaseJ4.tearDown ###Ending testReliability
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLBHttpSolrServer -Dtests.method=testReliability -Dtests.seed=2E5BBD83E82DC359 -Dtests.slow=true -Dtests.locale=nl_BE -Dtests.timezone=Poland -Dtests.file.encoding=UTF-8
   [junit4] ERROR   22.2s | TestLBHttpSolrServer.testReliability <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([2E5BBD83E82DC359:EF9360C5494B12F0]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:528)
   [junit4]    > 	at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
   [junit4]    > 	at org.apache.solr.client.solrj.TestLBHttpSolrServer.testReliability(TestLBHttpSolrServer.java:223)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:53542/solr
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:481)
   [junit4]    > 	... 43 more
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:150)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:121)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
   [junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
   [junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
   [junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
   [junit4]    > 	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:682)
   [junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)
   [junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:448)
   [junit4]    > 	... 46 more
   [junit4]   2> 37260 T31 oas.SolrTestCaseJ4.setUp ###Starting testTwoServers
   [junit4]   2> 37274 T31 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 37286 T31 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53588
   [junit4]   2> 37288 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 37289 T31 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 37289 T31 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001
   [junit4]   2> 37290 T31 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/'
   [junit4]   2> 37372 T31 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/solr.xml
   [junit4]   2> 37436 T31 oasc.CoreContainer.<init> New CoreContainer 1435178801
   [junit4]   2> 37436 T31 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/]
   [junit4]   2> 37437 T31 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 37438 T31 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 37438 T31 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 37439 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 37441 T31 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 37447 T31 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 37448 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 37448 T31 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 37449 T31 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 37450 T31 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 37451 T31 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 37459 T31 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 37460 T31 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 37468 T80 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/'
   [junit4]   2> 37520 T80 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 37540 T80 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 37541 T80 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 37550 T80 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 37565 T80 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 37567 T80 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/
   [junit4]   2> 37567 T80 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockFSDirectoryFactory
   [junit4]   2> 37567 T80 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/
   [junit4]   2> 37567 T80 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 37575 T80 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data
   [junit4]   2> 37576 T80 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index/
   [junit4]   2> 37576 T80 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 37579 T80 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index
   [junit4]   2> 37585 T80 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 37585 T80 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 37589 T80 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 37590 T80 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 37590 T80 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 37590 T80 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 37591 T80 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 37591 T80 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 37591 T80 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 37591 T80 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 37591 T80 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 37592 T80 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 37601 T80 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 37606 T80 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 37613 T80 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 37614 T80 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 37616 T80 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 37617 T80 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 37623 T80 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 37623 T80 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 37625 T80 oass.SolrIndexSearcher.<init> Opening Searcher@53144aa5[collection1] main
   [junit4]   2> 37625 T80 oasr.ManagedResourceStorage$FileStorageIO.configure File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/conf
   [junit4]   2> 37626 T80 oasr.RestManager.init Initializing RestManager with initArgs: {storageDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/conf}
   [junit4]   2> 37626 T80 oasr.ManagedResourceStorage.load Reading _rest_managed.json using file:dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/conf
   [junit4]   2> 37627 T80 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 37628 T80 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using file:dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/conf
   [junit4]   2> 37629 T80 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 37629 T80 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 37630 T81 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@53144aa5[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 37630 T80 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 37641 T31 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0
   [junit4]   2> 37642 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> ASYNC  NEW_CORE C8 name=collection1 org.apache.solr.core.SolrCore@4a570d65
   [junit4]   2> 37678 T75 C8 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]} 0 29
   [junit4]   2> 37687 T76 C8 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 37733 T76 C8 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index),segFN=segments_2,generation=2}
   [junit4]   2> 37734 T76 C8 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 37754 T76 C8 oass.SolrIndexSearcher.<init> Opening Searcher@3e3eec63[collection1] main
   [junit4]   2> 37755 T76 C8 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 37755 T81 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3e3eec63[collection1] main{UninvertingDirectoryReader(Uninverting(_0(5.0.0):c10))}
   [junit4]   2> 37760 T76 C8 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} {commit=} 0 73
   [junit4]   2> 37775 T31 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 37785 T31 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53592
   [junit4]   2> 37786 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 37786 T31 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 37786 T31 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001
   [junit4]   2> 37787 T31 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/'
   [junit4]   2> 37809 T31 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/solr.xml
   [junit4]   2> 37846 T31 oasc.CoreContainer.<init> New CoreContainer 1130257930
   [junit4]   2> 37849 T31 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/]
   [junit4]   2> 37850 T31 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 37850 T31 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 37850 T31 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 37851 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 37851 T31 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 37852 T31 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 37853 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 37853 T31 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 37853 T31 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 37854 T31 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 37854 T31 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 37854 T31 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 37855 T31 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 37866 T92 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/'
   [junit4]   2> 37905 T92 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 37930 T92 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 37932 T92 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 37937 T92 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 37945 T92 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 37947 T92 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/
   [junit4]   2> 37948 T92 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockFSDirectoryFactory
   [junit4]   2> 37948 T92 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/
   [junit4]   2> 37949 T92 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 37951 T92 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data
   [junit4]   2> 37952 T92 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index/
   [junit4]   2> 37952 T92 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 37954 T92 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index
   [junit4]   2> 37962 T92 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 37962 T92 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 37965 T92 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 37966 T92 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 37966 T92 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 37967 T92 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 37967 T92 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 37969 T92 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 37969 T92 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 37969 T92 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 37970 T92 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 37970 T92 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 37972 T92 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 37973 T92 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 37974 T92 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 37976 T92 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 37978 T92 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 37978 T92 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 37983 T92 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(MMapDirectory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index lockFactory=NativeFSLockFactory@/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 37983 T92 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 37984 T92 oass.SolrIndexSearcher.<init> Opening Searcher@bf9669f[collection1] main
   [junit4]   2> 37984 T92 oasr.ManagedResourceStorage$FileStorageIO.configure File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/conf
   [junit4]   2> 37985 T92 oasr.RestManager.init Initializing RestManager with initArgs: {storageDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/conf}
   [junit4]   2> 37985 T92 oasr.Ma

[...truncated too long message...]

ve_adds=10,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 67616 T31 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 67617 T31 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 67617 T31 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 67624 T31 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 67625 T31 oasc.CachingDirectoryFactory.close Closing MockFSDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 67626 T31 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index [CachedDir<<refCount=0;path=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index;done=false>>]
   [junit4]   2> 67627 T31 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data/index
   [junit4]   2> 67643 T31 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data [CachedDir<<refCount=0;path=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data;done=false>>]
   [junit4]   2> 67644 T31 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-0-001/collection1/data
   [junit4]   2> 67651 T31 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> 72628 T31 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=582649744
   [junit4]   2> 72632 T31 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@6907c2fd
   [junit4]   2> 72633 T31 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 72635 T31 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 72636 T31 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 72636 T31 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 72651 T31 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 72653 T31 oasc.CachingDirectoryFactory.close Closing MockFSDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 72653 T31 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data [CachedDir<<refCount=0;path=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data;done=false>>]
   [junit4]   2> 72654 T31 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data
   [junit4]   2> 72654 T31 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index [CachedDir<<refCount=0;path=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index;done=false>>]
   [junit4]   2> 72655 T31 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-1-001/collection1/data/index
   [junit4]   2> 72675 T31 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> 77647 T31 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=657395488
   [junit4]   2> 77650 T31 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@2e900bea
   [junit4]   2> 77650 T31 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=10,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 77652 T31 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 77653 T31 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 77653 T31 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 77661 T31 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 77662 T31 oasc.CachingDirectoryFactory.close Closing MockFSDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 77663 T31 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data [CachedDir<<refCount=0;path=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data;done=false>>]
   [junit4]   2> 77663 T31 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data
   [junit4]   2> 77664 T31 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index [CachedDir<<refCount=0;path=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index;done=false>>]
   [junit4]   2> 77664 T31 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001/instance-2-001/collection1/data/index
   [junit4]   2> 77679 T31 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> 82641 T31 oas.SolrTestCaseJ4.tearDown ###Ending testSimple
   [junit4]   2> 82643 T31 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrServer-2E5BBD83E82DC359-001
   [junit4]   2> NOTE: test params are: codec=Lucene410: {name=PostingsFormat(name=SimpleText), id=PostingsFormat(name=SimpleText)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=nl_BE, timezone=Poland
   [junit4]   2> NOTE: Mac OS X 10.8.5 x86_64/Oracle Corporation 1.8.0_20 (64-bit)/cpus=2,threads=4,free=8372592,total=40632320
   [junit4]   2> NOTE: All tests run in this JVM: [DocumentAnalysisResponseTest, TestXMLEscaping, TestEmbeddedSolrServer, TestLBHttpSolrServer]
   [junit4] Completed in 67.66s, 3 tests, 1 error <<< FAILURES!

[...truncated 150 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:491: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:471: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:60: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/extra-targets.xml:39: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:202: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:496: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1358: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:961: There were test failures: 53 suites, 313 tests, 1 error

Total time: 158 minutes 12 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 1833 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1833/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
REGRESSION:  org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch

Error Message:


Stack Trace:
java.lang.NullPointerException
	at __randomizedtesting.SeedInfo.seed([9D91BDE760C23FDB:1C7733FF179D5FE7]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:706)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601)
	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:124)
	at org.apache.solr.client.solrj.SolrServer.deleteByQuery(SolrServer.java:285)
	at org.apache.solr.client.solrj.SolrServer.deleteByQuery(SolrServer.java:271)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.del(AbstractFullDistribZkTestBase.java:728)
	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.tryDelete(ChaosMonkeySafeLeaderTest.java:194)
	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:112)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
	at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11288 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/init-core-data-001
   [junit4]   2> 2658301 T7128 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 2658302 T7128 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 2658309 T7128 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 2658313 T7128 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2658315 T7129 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 2658415 T7128 oasc.ZkTestServer.run start zk server on port:51023
   [junit4]   2> 2658416 T7128 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2658422 T7128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2658430 T7135 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@405ed72e name:ZooKeeperConnection Watcher:127.0.0.1:51023 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2658431 T7128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2658431 T7128 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2658433 T7128 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 2658444 T7128 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2658448 T7128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2658451 T7137 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6d90284e name:ZooKeeperConnection Watcher:127.0.0.1:51023/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2658451 T7128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2658452 T7128 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2658452 T7128 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 2658457 T7128 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 2658464 T7128 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 2658467 T7128 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 2658473 T7128 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2658473 T7128 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2658481 T7128 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 2658482 T7128 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 2658489 T7128 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2658489 T7128 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2658497 T7128 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2658498 T7128 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2658506 T7128 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2658507 T7128 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2658512 T7128 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2658512 T7128 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 2658520 T7128 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2658520 T7128 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2658528 T7128 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2658529 T7128 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2658536 T7128 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2658536 T7128 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2658558 T7128 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2658559 T7128 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2658564 T7128 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2658568 T7128 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2658576 T7128 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2658579 T7128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2658580 T7139 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21519232 name:ZooKeeperConnection Watcher:127.0.0.1:51023/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2658580 T7128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2658580 T7128 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2660459 T7128 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2660471 T7128 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 2660473 T7128 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:51027
   [junit4]   2> 2660476 T7128 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2660478 T7128 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2660479 T7128 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-002
   [junit4]   2> 2660479 T7128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-002/'
   [junit4]   2> 2660519 T7128 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-002/solr.xml
   [junit4]   2> 2660537 T7128 oasc.CoreContainer.<init> New CoreContainer 668295376
   [junit4]   2> 2660538 T7128 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-002/]
   [junit4]   2> 2660539 T7128 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2660539 T7128 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2660539 T7128 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2660540 T7128 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2660540 T7128 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2660540 T7128 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2660540 T7128 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2660541 T7128 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2660541 T7128 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2660543 T7128 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 2660545 T7128 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2660546 T7128 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2660546 T7128 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2660546 T7128 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51023/solr
   [junit4]   2> 2660547 T7128 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2660548 T7128 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2660550 T7128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2660552 T7150 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7abb9609 name:ZooKeeperConnection Watcher:127.0.0.1:51023 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2660554 T7128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2660555 T7128 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2660559 T7128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2660560 T7152 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d2e38f7 name:ZooKeeperConnection Watcher:127.0.0.1:51023/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2660560 T7128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2660563 T7128 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 2660568 T7128 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 2660573 T7128 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 2660578 T7128 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 2660583 T7128 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 2660590 T7128 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 2660595 T7128 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51027_
   [junit4]   2> 2660597 T7128 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51027_
   [junit4]   2> 2660606 T7128 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 2660610 T7128 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 2660615 T7128 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 2660620 T7128 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:51027_
   [junit4]   2> 2660620 T7128 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 2660627 T7128 oasc.Overseer.start Overseer (id=92458663216218116-127.0.0.1:51027_-n_0000000000) starting
   [junit4]   2> 2660635 T7128 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 2660665 T7128 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2660666 T7154 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 2660669 T7128 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 2660675 T7128 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 2660679 T7128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2660693 T7153 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 2660700 T7156 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 2660701 T7156 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2660703 T7157 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2660707 T7156 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2660711 T7153 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:51027",
   [junit4]   2> 	  "node_name":"127.0.0.1:51027_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2660711 T7153 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 2660711 T7153 oasc.Overseer$ClusterStateUpdater.createCollection state version control_collection 1
   [junit4]   2> 2660711 T7153 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 2660715 T7157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2661708 T7156 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2661709 T7156 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 2661712 T7156 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2661712 T7156 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 2661718 T7156 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2661718 T7156 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-002/collection1/'
   [junit4]   2> 2661720 T7156 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-002/collection1/lib/.svn/' to classloader
   [junit4]   2> 2661722 T7156 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-002/collection1/lib/classes/' to classloader
   [junit4]   2> 2661723 T7156 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-002/collection1/lib/README' to classloader
   [junit4]   2> 2661770 T7156 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 2661794 T7156 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2661796 T7156 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2661804 T7156 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 2661915 T7156 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2661916 T7156 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2661917 T7156 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2661926 T7156 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2661990 T7156 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2661991 T7156 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 2661991 T7156 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-002/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/control/data/
   [junit4]   2> 2661991 T7156 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@65527925
   [junit4]   2> 2661994 T7156 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/control/data
   [junit4]   2> 2661994 T7156 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/control/data/index/
   [junit4]   2> 2661995 T7156 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2661996 T7156 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/control/data/index
   [junit4]   2> 2661997 T7156 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=87.1201171875, floorSegmentMB=1.4130859375, forceMergeDeletesPctAllowed=11.502438378813286, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.13592693577772286
   [junit4]   2> 2662001 T7156 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 2662002 T7156 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2662006 T7156 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2662006 T7156 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2662006 T7156 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2662007 T7156 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2662007 T7156 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2662008 T7156 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2662008 T7156 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2662009 T7156 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2662009 T7156 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2662009 T7156 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2662010 T7156 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2662011 T7156 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2662012 T7156 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 2662012 T7156 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 2662013 T7156 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 2662013 T7156 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 2662014 T7156 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2662014 T7156 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2662015 T7156 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 2662015 T7156 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 2662023 T7156 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2662025 T7156 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2662027 T7156 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2662029 T7156 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2662042 T7156 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2662042 T7156 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2662043 T7156 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=22, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 2662045 T7156 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 2662045 T7156 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2662045 T7156 oass.SolrIndexSearcher.<init> Opening Searcher@200b6d83[collection1] main
   [junit4]   2> 2662046 T7156 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 2662050 T7156 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2662050 T7156 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2662051 T7156 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2662052 T7156 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 2662052 T7156 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2662053 T7156 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2662054 T7156 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2662055 T7156 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
   [junit4]   2> 2662060 T7156 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2662061 T7156 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2662061 T7156 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 2662064 T7158 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@200b6d83[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 2662066 T7156 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 2662069 T7161 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:51027 collection:control_collection shard:shard1
   [junit4]   2> 2662070 T7128 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 2662070 T7128 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2662071 T7161 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2662074 T7128 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2662075 T7128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2662080 T7163 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f5f991c name:ZooKeeperConnection Watcher:127.0.0.1:51023/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2662080 T7128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2662081 T7128 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2662084 T7128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2662088 T7161 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 2662094 T7128 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2662101 T7157 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2662105 T7161 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 2662106 T7161 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C4938 name=collection1 org.apache.solr.core.SolrCore@166ee827 url=https://127.0.0.1:51027/collection1 node=127.0.0.1:51027_ C4938_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=https://127.0.0.1:51027, node_name=127.0.0.1:51027_, state=down}
   [junit4]   2> 2662106 T7161 C4938 P51027 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:51027/collection1/
   [junit4]   2> 2662107 T7161 C4938 P51027 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 2662107 T7161 C4938 P51027 oasc.SyncStrategy.syncToMe https://127.0.0.1:51027/collection1/ has no replicas
   [junit4]   2> 2662107 T7161 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:51027/collection1/ shard1
   [junit4]   2> 2662107 T7161 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2662161 T7157 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2662266 T7157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2662267 T7164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2662303 T7161 oasc.ZkController.register We are https://127.0.0.1:51027/collection1/ and leader is https://127.0.0.1:51027/collection1/
   [junit4]   2> 2662303 T7161 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:51027
   [junit4]   2> 2662303 T7161 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 2662304 T7161 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 2662304 T7161 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2662311 T7157 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2662326 T7153 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:51027",
   [junit4]   2> 	  "node_name":"127.0.0.1:51027_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2662434 T7164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2662434 T7157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2663891 T7128 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 2663892 T7128 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2663901 T7128 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 2663903 T7128 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:51031
   [junit4]   2> 2663909 T7128 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2663909 T7128 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2663910 T7128 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-003
   [junit4]   2> 2663910 T7128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-003/'
   [junit4]   2> 2663946 T7128 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-003/solr.xml
   [junit4]   2> 2663963 T7128 oasc.CoreContainer.<init> New CoreContainer 1316883938
   [junit4]   2> 2663964 T7128 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-003/]
   [junit4]   2> 2663965 T7128 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2663965 T7128 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2663966 T7128 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2663966 T7128 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2663967 T7128 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2663967 T7128 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2663967 T7128 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2663968 T7128 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2663968 T7128 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2663970 T7128 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 2663972 T7128 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2663972 T7128 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2663974 T7128 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2663974 T7128 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51023/solr
   [junit4]   2> 2663974 T7128 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2663974 T7128 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2663976 T7128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2663981 T7175 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32445441 name:ZooKeeperConnection Watcher:127.0.0.1:51023 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2663982 T7128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2663983 T7128 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2663985 T7128 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2663988 T7177 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7de157ab name:ZooKeeperConnection Watcher:127.0.0.1:51023/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2663989 T7128 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2664008 T7128 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2665025 T7128 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51031_
   [junit4]   2> 2665033 T7128 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51031_
   [junit4]   2> 2665049 T7128 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 2665061 T7179 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 2665062 T7179 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2665064 T7157 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2665064 T7179 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2665076 T7153 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:51031",
   [junit4]   2> 	  "node_name":"127.0.0.1:51031_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2665077 T7153 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
   [junit4]   2> 2665077 T7153 oasc.Overseer$ClusterStateUpdater.createCollection state version collection1 1
   [junit4]   2> 2665077 T7153 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 2665085 T7164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2665086 T7178 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2665086 T7157 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2666066 T7179 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2666067 T7179 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 2666071 T7179 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2666072 T7179 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 2666075 T7179 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2666076 T7179 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-003/collection1/'
   [junit4]   2> 2666078 T7179 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-003/collection1/lib/.svn/' to classloader
   [junit4]   2> 2666079 T7179 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-003/collection1/lib/classes/' to classloader
   [junit4]   2> 2666080 T7179 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-003/collection1/lib/README' to classloader
   [junit4]   2> 2666138 T7179 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 2666163 T7179 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2666166 T7179 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2666170 T7179 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 2666296 T7179 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2666297 T7179 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2666298 T7179 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2666304 T7179 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2666381 T7179 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 2666382 T7179 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 2666383 T7179 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-003/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/jetty1/
   [junit4]   2> 2666383 T7179 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@65527925
   [junit4]   2> 2666384 T7179 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/jetty1
   [junit4]   2> 2666386 T7179 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/jetty1/index/
   [junit4]   2> 2666386 T7179 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 2666388 T7179 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/jetty1/index
   [junit4]   2> 2666389 T7179 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=87.1201171875, floorSegmentMB=1.4130859375, forceMergeDeletesPctAllowed=11.502438378813286, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.13592693577772286
   [junit4]   2> 2666393 T7179 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
   [junit4]   2> 2666393 T7179 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2666399 T7179 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2666400 T7179 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2666400 T7179 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2666400 T7179 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2666403 T7179 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2666403 T7179 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2666404 T7179 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2666404 T7179 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2666404 T7179 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2666405 T7179 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2666406 T7179 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2666406 T7179 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2666407 T7179 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 2666408 T7179 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 2666408 T7179 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 2666409 T7179 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 2666409 T7179 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2666410 T7179 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2666410 T7179 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 2666411 T7179 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 2666417 T7179 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2666418 T7179 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2666419 T7179 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2666421 T7179 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2666424 T7179 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2666424 T7179 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2666425 T7179 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=22, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 2666426 T7179 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
   [junit4]   2> 2666426 T7179 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2666427 T7179 oass.SolrIndexSearcher.<init> Opening Searcher@570a0ed1[collection1] main
   [junit4]   2> 2666427 T7179 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 2666431 T7179 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2666431 T7179 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2666433 T7179 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2666434 T7179 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 2666434 T7179 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2666438 T7179 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2666438 T7179 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2666439 T7179 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
   [junit4]   2> 2666439 T7179 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 2666446 T7180 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@570a0ed1[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 2666449 T7179 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 2666453 T7183 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:51031 collection:collection1 shard:shard1
   [junit4]   2> 2666453 T7128 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 2666454 T7128 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2666456 T7183 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 2666467 T7128 oasc.AbstractFullDistribZkTestBase.randomlyEnableAutoSoftCommit Not turning on auto soft commit
   [junit4]   2> 2666476 T7183 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 2666480 T7183 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 2666480 T7183 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C4939 name=collection1 org.apache.solr.core.SolrCore@7bb875df url=https://127.0.0.1:51031/collection1 node=127.0.0.1:51031_ C4939_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=https://127.0.0.1:51031, node_name=127.0.0.1:51031_, state=down}
   [junit4]   2> 2666480 T7183 C4939 P51031 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:51031/collection1/
   [junit4]   2> 2666480 T7183 C4939 P51031 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 2666480 T7183 C4939 P51031 oasc.SyncStrategy.syncToMe https://127.0.0.1:51031/collection1/ has no replicas
   [junit4]   2> 2666481 T7183 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:51031/collection1/ shard1
   [junit4]   2> 2666481 T7183 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2666488 T7157 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2666499 T7157 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> ASYNC  NEW_CORE C4940 name=collection1 org.apache.solr.core.SolrCore@166ee827 url=https://127.0.0.1:51027/collection1 node=127.0.0.1:51027_ C4940_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=https://127.0.0.1:51027, node_name=127.0.0.1:51027_, state=active, leader=true}
   [junit4]   2> 2666509 T7148 C4940 P51027 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1479338620036841472)} 0 7
   [junit4]   2> 2666511 T7128 oascsi.CloudSolrServer.requestWithRetryOnStaleState ERROR Request to collection collection1 failed due to (510) org.apache.solr.common.SolrException: Not enough nodes to handle the request, retry? 0
   [junit4]   2> 2666518 T7128 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 51027
   [junit4]   2> 2666518 T7128 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=668295376
   [junit4]   2> 2666518 T7128 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 2666519 T7128 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2666520 T7157 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2666521 T7128 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@166ee827
   [junit4]   2> 2666522 T7153 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:51027",
   [junit4]   2> 	  "node_name":"127.0.0.1:51027_",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2666525 T7128 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=1,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 2666526 T7128 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 2666528 T7128 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2666528 T7128 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2666529 T7128 C4940 P51027 oasu.DirectUpdateHandler2.closeWriter Committing on IndexWriter close.
   [junit4]   2> 2666534 T7128 C4940 P51027 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/control/data/index,segFN=segments_2,generation=2}
   [junit4]   2> 2666535 T7128 C4940 P51027 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 2666537 T7128 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 2666537 T7128 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2666538 T7128 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/control/data/index [CachedDir<<refCount=0;path=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/control/data/index;done=false>>]
   [junit4]   2> 2666538 T7128 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/control/data/index
   [junit4]   2> 2666539 T7128 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/control/data [CachedDir<<refCount=0;path=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/control/data;done=false>>]
   [junit4]   2> 2666539 T7128 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/control/data
   [junit4]   2> 2666540 T7128 oasc.Overseer.close Overseer (id=92458663216218116-127.0.0.1:51027_-n_0000000000) closing
   [junit4]   2> 2666540 T7153 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:51027_
   [junit4]   2> 2666546 T7157 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 2666552 T7185 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:51031_
   [junit4]   2> 2666552 T7185 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 2666559 T7185 oasc.Overseer.start Overseer (id=92458663216218119-127.0.0.1:51031_-n_0000000001) starting
   [junit4]   2> 2666562 T7128 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
   [junit4]   2> 2666602 T7185 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2666605 T7187 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 2666622 T7186 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
   [junit4]   2> 2666626 T7185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2666648 T7186 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:51027",
   [junit4]   2> 	  "node_name":"127.0.0.1:51027_",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2666660 T7186 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 2666703 T7183 oasc.ZkController.register We are https://127.0.0.1:51031/collection1/ and leader is https://127.0.0.1:51031/collection1/
   [junit4]   2> 2666703 T7183 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:51031
   [junit4]   2> 2666704 T7183 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 2666705 T7183 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 2666705 T7183 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2666710 T7185 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2666730 T7186 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:51031",
   [junit4]   2> 	  "node_name":"127.0.0.1:51031_",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2666843 T7185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2668045 T7184 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:347)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:344)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:344)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:346)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:296)
   [junit4]   2> 
   [junit4]   2> 2671552 T7128 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 51031
   [junit4]   2> 2671553 T7128 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1316883938
   [junit4]   2> 2671554 T7128 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 2671556 T7128 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2671561 T7128 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@7bb875df
   [junit4]   2> 2671561 T7185 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2671574 T7186 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:51031",
   [junit4]   2> 	  "node_name":"127.0.0.1:51031_",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2671578 T7128 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 2671578 T7128 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 2671578 T7128 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2671579 T7128 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2671579 T7185 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2671583 T7128 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 2671583 T7128 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2671584 T7128 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/jetty1/index [CachedDir<<refCount=0;path=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/jetty1/index;done=false>>]
   [junit4]   2> 2671585 T7128 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/jetty1/index
   [junit4]   2> 2671585 T7128 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/jetty1 [CachedDir<<refCount=0;path=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/jetty1;done=false>>]
   [junit4]   2> 2671585 T7128 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001/tempDir-001/jetty1
   [junit4]   2> 2671587 T7128 oasc.Overseer.close Overseer (id=92458663216218119-127.0.0.1:51031_-n_0000000001) closing
   [junit4]   2> 2671587 T7186 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:51031_
   [junit4]   2> 2671592 T7177 oaz.ClientCnxn$EventThread.processEvent ERROR Error while calling watcher  java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1a8b885f rejected from java.util.concurrent.ThreadPoolExecutor@5f89056a[Shutting down, pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 8]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:263)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 
   [junit4]   2> 2673090 T7185 oascc.ZkStateReader.getIndividualColls WARN Error reading collections nodes org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:313)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:310)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:310)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.getIndividualColls(ZkStateReader.java:483)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.constructState(ZkStateReader.java:448)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.access$200(ZkStateReader.java:55)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader$2.process(ZkStateReader.java:315)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:266)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 2673091 T7185 oascc.ZkStateReader.getIndividualColls WARN Error reading collections nodes org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/control_collection/state.json
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:313)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:310)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:310)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.getIndividualColls(ZkStateReader.java:483)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.constructState(ZkStateReader.java:448)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.access$200(ZkStateReader.java:55)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader$2.process(ZkStateReader.java:315)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:266)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 2673093 T7189 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:347)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:344)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:344)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:346)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:296)
   [junit4]   2> 
   [junit4]   2> 2673115 T7128 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
   [junit4]   2> 2678101 T7128 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 2678107 T7128 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51023 51023
   [junit4]   2> 2678286 T7129 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51023 51023
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.method=testDistribSearch -Dtests.seed=9D91BDE760C23FDB -Dtests.slow=true -Dtests.locale=vi -Dtests.timezone=Pacific/Palau -Dtests.file.encoding=UTF-8
   [junit4] ERROR   20.3s | ChaosMonkeySafeLeaderTest.testDistribSearch <<<
   [junit4]    > Throwable #1: java.lang.NullPointerException
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9D91BDE760C23FDB:1C7733FF179D5FE7]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:706)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601)
   [junit4]    > 	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:124)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrServer.deleteByQuery(SolrServer.java:285)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrServer.deleteByQuery(SolrServer.java:271)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.del(AbstractFullDistribZkTestBase.java:728)
   [junit4]    > 	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.tryDelete(ChaosMonkeySafeLeaderTest.java:194)
   [junit4]    > 	at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:112)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2678569 T7128 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-9D91BDE760C23FDB-001
   [junit4]   2> NOTE: test params are: codec=Lucene410: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=vi, timezone=Pacific/Palau
   [junit4]   2> NOTE: Mac OS X 10.8.5 x86_64/Oracle Corporation 1.8.0_20 (64-bit)/cpus=2,threads=1,free=319795856,total=461049856
   [junit4]   2> NOTE: All tests run in this JVM: [TestManagedSchema, DistributedQueryElevationComponentTest, MultiTermTest, SpellingQueryConverterTest, TestStandardQParsers, TestSimpleQParserPlugin, ReturnFieldsTest, PathHierarchyTokenizerFactoryTest, TestPostingsSolrHighlighter, TestRemoteStreaming, TestCodecSupport, BlockDirectoryTest, SolrPluginUtilsTest, TestDocSet, TestPartialUpdateDeduplication, UpdateParamsTest, MoreLikeThisHandlerTest, TimeZoneUtilsTest, TestValueSourceCache, MergeStrategyTest, ExternalCollectionsTest, TestReRankQParserPlugin, TestDefaultSearchFieldResource, SampleTest, RankQueryTest, TestQuerySenderListener, TestFieldResource, TestSolrCoreProperties, TestManagedSchemaDynamicFieldResource, NotRequiredUniqueKeyTest, TestIndexSearcher, BadComponentTest, PrimUtilsTest, FullSolrCloudDistribCmdsTest, TestManagedResource, TestShardHandlerFactory, TestClassNameShortening, FastVectorHighlighterTest, TestSolrQueryParserDefaultOperatorResource, TestMissingGroups, TestUniqueKeyFieldResource, OverriddenZkACLAndCredentialsProvidersTest, StressHdfsTest, HighlighterMaxOffsetTest, TestSortingResponseWriter, DirectUpdateHandlerOptimizeTest, TestCustomSort, HardAutoCommitTest, TestFieldTypeCollectionResource, SSLMigrationTest, TestRangeQuery, TestDistributedMissingSort, QueryParsingTest, ReplicationFactorTest, TestConfigSets, TestSolr4Spatial2, TestSolr4Spatial, TestDynamicFieldResource, RemoteQueryErrorTest, TestStressRecovery, TestManagedSchemaFieldTypeResource, TestJmxMonitoredMap, FieldAnalysisRequestHandlerTest, HdfsChaosMonkeySafeLeaderTest, CurrencyFieldXmlFileTest, TermsComponentTest, TestFaceting, InfoHandlerTest, TestBM25SimilarityFactory, HdfsSyncSliceTest, ScriptEngineTest, AddSchemaFieldsUpdateProcessorFactoryTest, DistributedQueryComponentOptimizationTest, QueryElevationComponentTest, CoreAdminRequestStatusTest, TestSolrIndexConfig, SolrRequestParserTest, TestCollationField, TestDefaultSimilarityFactory, DeleteLastCustomShardedReplicaTest, TestQueryTypes, TestBadConfig, TestReload, TestAddFieldRealTimeGet, OverseerTest, StatelessScriptUpdateProcessorFactoryTest, TestLeaderElectionZkExpiry, FieldFacetExtrasTest, TestCharFilters, DocValuesMissingTest, TestLFUCache, DeleteReplicaTest, CopyFieldTest, QueryResultKeyTest, TestComponentsName, SuggesterFSTTest, BasicZkTest, TestFieldCollectionResource, SuggesterTSTTest, TestNRTOpen, BasicFunctionalityTest, TestSolrQueryParserResource, HdfsUnloadDistributedZkTest, TestStressUserVersions, QueryEqualityTest, DocValuesTest, TestReloadAndDeleteDocs, HdfsWriteToMultipleCollectionsTest, SolrTestCaseJ4Test, CircularListTest, CoreAdminCreateDiscoverTest, PingRequestHandlerTest, BlockCacheTest, AnalysisErrorHandlingTest, LeaderElectionIntegrationTest, TestSurroundQueryParser, NumericFieldsTest, ExternalFileFieldSortTest, HighlighterTest, DistribDocExpirationUpdateProcessorTest, LegacyHTMLStripCharFilterTest, TestTolerantSearch, SoftAutoCommitTest, TestCloudPivotFacet, CursorPagingTest, TestSearcherReuse, TestGroupingSearch, TestHighFrequencyDictionaryFactory, PluginInfoTest, EnumFieldTest, RollingRestartTest, TestImplicitCoreProperties, TestShortCircuitedRequests, OverseerCollectionProcessorTest, AlternateDirectoryTest, PreAnalyzedFieldTest, TestCollapseQParserPlugin, LeaderElectionTest, SignatureUpdateProcessorFactoryTest, TestLRUCache, SolrCoreCheckLockOnStartupTest, RAMDirectoryFactoryTest, TestXIncludeConfig, TestHighlightDedupGrouping, SliceStateTest, SyncSliceTest, TestRealTimeGet, TestSort, TestFunctionQuery, TestMaxScoreQueryParser, TestStressVersions, DocExpirationUpdateProcessorFactoryTest, TestPerFieldSimilarity, CustomCollectionTest, TestFileDictionaryLookup, ShowFileRequestHandlerTest, TestAtomicUpdateErrorCases, IndexSchemaRuntimeFieldTest, BasicDistributedZk2Test, TestParamSet, TestDistributedGrouping, TestSolrDeletionPolicy1, TestManagedSynonymFilterFactory, CurrencyFieldOpenExchangeTest, RecoveryZkTest, TestMultiCoreConfBootstrap, SharedFSAutoReplicaFailoverTest, RequiredFieldsTest, TestExtendedDismaxParser, DistributedSuggestComponentTest, TestPhraseSuggestions, PolyFieldTest, TestCloudManagedSchema, DateMathParserTest, TestNonDefinedSimilarityFactory, HdfsCollectionsAPIDistributedZkTest, WordBreakSolrSpellCheckerTest, DistributedFacetPivotSmallTest, DocumentAnalysisRequestHandlerTest, TestHashPartitioner, TestLazyCores, SOLR749Test, HdfsLockFactoryTest, OverseerRolesTest, TestSchemaSimilarityResource, BasicDistributedZkTest, DOMUtilTest, SystemInfoHandlerTest, SpellPossibilityIteratorTest, EchoParamsTest, TestOrdValues, AbstractAnalyticsFacetTest, TermVectorComponentTest, TestRestManager, TestDynamicFieldCollectionResource, MBeansHandlerTest, TestRandomMergePolicy, BJQParserTest, TestNonNRTOpen, DistributedExpandComponentTest, HdfsBasicDistributedZkTest, TestSolrJ, TestNoOpRegenerator, TestPivotHelperCode, ShardRoutingCustomTest, DebugComponentTest, DistanceFunctionTest, TestUtils, HttpPartitionTest, OverseerStatusTest, TestCloudSchemaless, TestSolrDeletionPolicy2, BadCopyFieldTest, SolrCmdDistributorTest, ChaosMonkeySafeLeaderTest]
   [junit4] Completed in 20.31s, 1 test, 1 error <<< FAILURES!

[...truncated 686 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:491: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:471: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:60: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/extra-targets.xml:39: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:189: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:496: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1358: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:961: There were test failures: 438 suites, 1830 tests, 1 error, 50 ignored (22 assumptions)

Total time: 147 minutes 15 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 1832 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1832/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

2 tests failed.
REGRESSION:  org.apache.solr.cloud.DeleteShardTest.testDistribSearch

Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:52376/fq_grh

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:52376/fq_grh
	at __randomizedtesting.SeedInfo.seed([C8AF289A6ACC0F0C:4949A6821D936F30]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
	at org.apache.solr.cloud.DeleteShardTest.deleteShard(DeleteShardTest.java:152)
	at org.apache.solr.cloud.DeleteShardTest.doTest(DeleteShardTest.java:94)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
	at sun.reflect.GeneratedMethodAccessor37.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:150)
	at java.net.SocketInputStream.read(SocketInputStream.java:121)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
	at sun.security.ssl.InputRecord.read(InputRecord.java:503)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:954)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:911)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:682)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:448)
	... 44 more


REGRESSION:  org.apache.solr.core.TestNonNRTOpen.testReaderIsNotNRT

Error Message:
SOLR-5815? : wrong maxDoc: core=org.apache.solr.core.SolrCore@192c78ef searcher=Searcher@56653bf3[collection1] main{UninvertingDirectoryReader(Uninverting(_1(5.0.0):C1) Uninverting(_2(5.0.0):C1))} expected:<3> but was:<2>

Stack Trace:
java.lang.AssertionError: SOLR-5815? : wrong maxDoc: core=org.apache.solr.core.SolrCore@192c78ef searcher=Searcher@56653bf3[collection1] main{UninvertingDirectoryReader(Uninverting(_1(5.0.0):C1) Uninverting(_2(5.0.0):C1))} expected:<3> but was:<2>
	at __randomizedtesting.SeedInfo.seed([C8AF289A6ACC0F0C:7D29491DD50DBDF8]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.apache.solr.core.TestNonNRTOpen.assertNotNRT(TestNonNRTOpen.java:142)
	at org.apache.solr.core.TestNonNRTOpen.testReaderIsNotNRT(TestNonNRTOpen.java:100)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11764 lines...]
   [junit4] Suite: org.apache.solr.cloud.DeleteShardTest
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/init-core-data-001
   [junit4]   2> 4635310 T12138 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 4635310 T12138 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /fq_grh/
   [junit4]   2> 4635316 T12138 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 4635317 T12138 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 4635319 T12139 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 4635420 T12138 oasc.ZkTestServer.run start zk server on port:52365
   [junit4]   2> 4635421 T12138 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 4635424 T12138 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 4635434 T12145 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e1f3e0d name:ZooKeeperConnection Watcher:127.0.0.1:52365 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4635436 T12138 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 4635438 T12138 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 4635438 T12138 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 4635452 T12138 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 4635454 T12138 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 4635457 T12147 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62a2b74b name:ZooKeeperConnection Watcher:127.0.0.1:52365/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4635458 T12138 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 4635458 T12138 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 4635459 T12138 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 4635470 T12138 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 4635479 T12138 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 4635487 T12138 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 4635496 T12138 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 4635497 T12138 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 4635510 T12138 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 4635511 T12138 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 4635518 T12138 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4635519 T12138 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 4635526 T12138 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 4635528 T12138 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 4635534 T12138 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 4635535 T12138 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 4635543 T12138 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 4635543 T12138 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 4635555 T12138 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 4635555 T12138 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 4635564 T12138 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4635565 T12138 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 4635573 T12138 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4635574 T12138 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 4635582 T12138 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 4635582 T12138 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 4635590 T12138 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 4635591 T12138 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 4635601 T12138 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 4635603 T12138 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 4635605 T12149 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@54664ff8 name:ZooKeeperConnection Watcher:127.0.0.1:52365/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4635605 T12138 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 4635606 T12138 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 4637490 T12138 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 4637498 T12138 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 4637499 T12138 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:52369
   [junit4]   2> 4637502 T12138 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 4637503 T12138 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4637504 T12138 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-002
   [junit4]   2> 4637505 T12138 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-002/'
   [junit4]   2> 4637545 T12138 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-002/solr.xml
   [junit4]   2> 4637559 T12138 oasc.CoreContainer.<init> New CoreContainer 1437503719
   [junit4]   2> 4637560 T12138 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-002/]
   [junit4]   2> 4637561 T12138 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 4637562 T12138 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 4637562 T12138 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 4637562 T12138 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 4637562 T12138 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 4637562 T12138 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 4637563 T12138 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 4637563 T12138 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 4637564 T12138 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 4637566 T12138 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 4637568 T12138 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4637568 T12138 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4637569 T12138 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 4637569 T12138 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52365/solr
   [junit4]   2> 4637569 T12138 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 4637570 T12138 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 4637572 T12138 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 4637575 T12160 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2214a04 name:ZooKeeperConnection Watcher:127.0.0.1:52365 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4637576 T12138 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 4637577 T12138 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 4637580 T12138 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 4637584 T12162 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5830a458 name:ZooKeeperConnection Watcher:127.0.0.1:52365/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4637585 T12138 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 4637588 T12138 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 4637593 T12138 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 4637598 T12138 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 4637606 T12138 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 4637612 T12138 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 4637622 T12138 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 4637628 T12138 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52369_fq_grh
   [junit4]   2> 4637630 T12138 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52369_fq_grh
   [junit4]   2> 4637641 T12138 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 4637647 T12138 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 4637653 T12138 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 4637659 T12138 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:52369_fq_grh
   [junit4]   2> 4637660 T12138 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 4637666 T12138 oasc.Overseer.start Overseer (id=92455780472193028-127.0.0.1:52369_fq_grh-n_0000000000) starting
   [junit4]   2> 4637675 T12138 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 4637706 T12138 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 4637708 T12164 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 4637710 T12138 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 4637716 T12138 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 4637721 T12138 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 4637735 T12163 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 4637739 T12166 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 4637739 T12166 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 4637741 T12166 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 4637741 T12167 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 4637750 T12163 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:52369/fq_grh",
   [junit4]   2> 	  "node_name":"127.0.0.1:52369_fq_grh",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 4637750 T12163 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 4637751 T12163 oasc.Overseer$ClusterStateUpdater.createCollection state version control_collection 1
   [junit4]   2> 4637751 T12163 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 4637758 T12167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4638743 T12166 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 4638743 T12166 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 4638748 T12166 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 4638748 T12166 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 4638755 T12166 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4638756 T12166 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-002/collection1/'
   [junit4]   2> 4638758 T12166 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-002/collection1/lib/.svn/' to classloader
   [junit4]   2> 4638760 T12166 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-002/collection1/lib/classes/' to classloader
   [junit4]   2> 4638761 T12166 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-002/collection1/lib/README' to classloader
   [junit4]   2> 4638826 T12166 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 4638834 T12166 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4638841 T12166 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 4638850 T12166 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 4638948 T12166 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 4638959 T12166 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 4638960 T12166 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 4638994 T12166 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 4639000 T12166 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 4639006 T12166 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4639009 T12166 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 4639009 T12166 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 4639009 T12166 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4639012 T12166 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 4639012 T12166 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 4639013 T12166 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 4639013 T12166 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 4639013 T12166 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-002/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-001/control/data/
   [junit4]   2> 4639014 T12166 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@31f4a70e
   [junit4]   2> 4639014 T12166 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-001/control/data
   [junit4]   2> 4639015 T12166 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-001/control/data/index/
   [junit4]   2> 4639015 T12166 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 4639016 T12166 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-001/control/data/index
   [junit4]   2> 4639016 T12166 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=50, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8547504380703527]
   [junit4]   2> 4639017 T12166 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@d717ea3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@304c6807),segFN=segments_1,generation=1}
   [junit4]   2> 4639018 T12166 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 4639023 T12166 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 4639023 T12166 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 4639024 T12166 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 4639024 T12166 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4639024 T12166 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4639025 T12166 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 4639025 T12166 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4639025 T12166 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4639026 T12166 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 4639026 T12166 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 4639026 T12166 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 4639028 T12166 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 4639028 T12166 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 4639029 T12166 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 4639029 T12166 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 4639029 T12166 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 4639029 T12166 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 4639031 T12166 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 4639031 T12166 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 4639032 T12166 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 4639041 T12166 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4639043 T12166 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4639044 T12166 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4639045 T12166 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4639048 T12166 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 4639048 T12166 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 4639049 T12166 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@1535b809
   [junit4]   2> 4639049 T12166 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@d717ea3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@304c6807),segFN=segments_1,generation=1}
   [junit4]   2> 4639050 T12166 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 4639050 T12166 oass.SolrIndexSearcher.<init> Opening Searcher@5527440b[collection1] main
   [junit4]   2> 4639050 T12166 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 4639053 T12166 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4639054 T12166 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 4639056 T12166 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 4639056 T12166 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 4639056 T12166 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4639058 T12166 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 4639059 T12166 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4639061 T12166 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
   [junit4]   2> 4639067 T12166 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
   [junit4]   2> 4639067 T12166 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4639067 T12166 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 4639070 T12168 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5527440b[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 4639073 T12166 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 4639076 T12171 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:52369/fq_grh collection:control_collection shard:shard1
   [junit4]   2> 4639077 T12138 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 4639077 T12138 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 4639078 T12171 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 4639083 T12138 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 4639085 T12138 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 4639088 T12173 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d63951a name:ZooKeeperConnection Watcher:127.0.0.1:52365/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4639089 T12138 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 4639090 T12138 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 4639093 T12138 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 4639094 T12171 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 4639099 T12167 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 4639101 T12171 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 4639102 T12171 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C2906 name=collection1 org.apache.solr.core.SolrCore@45d8b20d url=https://127.0.0.1:52369/fq_grh/collection1 node=127.0.0.1:52369_fq_grh C2906_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=https://127.0.0.1:52369/fq_grh, node_name=127.0.0.1:52369_fq_grh, state=down}
   [junit4]   2> 4639103 T12171 C2906 P52369 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:52369/fq_grh/collection1/
   [junit4]   2> 4639103 T12171 C2906 P52369 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 4639104 T12171 C2906 P52369 oasc.SyncStrategy.syncToMe https://127.0.0.1:52369/fq_grh/collection1/ has no replicas
   [junit4]   2> 4639105 T12171 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:52369/fq_grh/collection1/ shard1
   [junit4]   2> 4639105 T12171 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 4639106 T12138 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 4639124 T12167 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 4639235 T12167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4639236 T12174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4639282 T12171 oasc.ZkController.register We are https://127.0.0.1:52369/fq_grh/collection1/ and leader is https://127.0.0.1:52369/fq_grh/collection1/
   [junit4]   2> 4639282 T12171 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:52369/fq_grh
   [junit4]   2> 4639282 T12171 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 4639282 T12171 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 4639283 T12171 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 4639286 T12167 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 4639297 T12163 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:52369/fq_grh",
   [junit4]   2> 	  "node_name":"127.0.0.1:52369_fq_grh",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 4639404 T12174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4639405 T12167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4640997 T12138 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 4640998 T12138 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 4641004 T12138 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 4641007 T12138 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:52373
   [junit4]   2> 4641010 T12138 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 4641011 T12138 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4641012 T12138 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-003
   [junit4]   2> 4641012 T12138 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-003/'
   [junit4]   2> 4641053 T12138 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-003/solr.xml
   [junit4]   2> 4641068 T12138 oasc.CoreContainer.<init> New CoreContainer 1144636110
   [junit4]   2> 4641069 T12138 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-003/]
   [junit4]   2> 4641070 T12138 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 4641070 T12138 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 4641071 T12138 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 4641071 T12138 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 4641071 T12138 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 4641072 T12138 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 4641072 T12138 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 4641072 T12138 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 4641073 T12138 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 4641076 T12138 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 4641078 T12138 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4641078 T12138 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4641079 T12138 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 4641079 T12138 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52365/solr
   [junit4]   2> 4641080 T12138 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 4641080 T12138 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 4641082 T12138 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 4641084 T12185 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@588ce911 name:ZooKeeperConnection Watcher:127.0.0.1:52365 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4641086 T12138 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 4641086 T12138 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 4641090 T12138 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 4641093 T12187 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f5a3ace name:ZooKeeperConnection Watcher:127.0.0.1:52365/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4641094 T12138 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 4641119 T12138 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 4642138 T12138 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52373_fq_grh
   [junit4]   2> 4642142 T12138 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52373_fq_grh
   [junit4]   2> 4642159 T12138 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 4642171 T12189 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 4642171 T12189 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 4642173 T12167 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 4642173 T12189 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 4642185 T12163 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:52373/fq_grh",
   [junit4]   2> 	  "node_name":"127.0.0.1:52373_fq_grh",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 4642185 T12163 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 4642185 T12163 oasc.Overseer$ClusterStateUpdater.createCollection state version collection1 1
   [junit4]   2> 4642185 T12163 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 4642194 T12167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 4642194 T12188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 4642194 T12174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 4643176 T12189 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 4643177 T12189 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 4643180 T12189 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 4643181 T12189 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 4643186 T12189 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4643187 T12189 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-003/collection1/'
   [junit4]   2> 4643189 T12189 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-003/collection1/lib/.svn/' to classloader
   [junit4]   2> 4643190 T12189 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-003/collection1/lib/classes/' to classloader
   [junit4]   2> 4643191 T12189 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-003/collection1/lib/README' to classloader
   [junit4]   2> 4643237 T12189 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 4643245 T12189 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4643252 T12189 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 4643257 T12189 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 4643378 T12189 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 4643386 T12189 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 4643386 T12189 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 4643416 T12189 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 4643421 T12189 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 4643425 T12189 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4643429 T12189 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 4643429 T12189 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 4643430 T12189 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4643433 T12189 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 4643434 T12189 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 4643434 T12189 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 4643435 T12189 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 4643435 T12189 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-003/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-001/jetty1/
   [junit4]   2> 4643435 T12189 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@31f4a70e
   [junit4]   2> 4643437 T12189 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-001/jetty1
   [junit4]   2> 4643437 T12189 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-001/jetty1/index/
   [junit4]   2> 4643438 T12189 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 4643438 T12189 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-001/jetty1/index
   [junit4]   2> 4643439 T12189 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=50, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8547504380703527]
   [junit4]   2> 4643440 T12189 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2f0a72e9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1a79ebd3),segFN=segments_1,generation=1}
   [junit4]   2> 4643440 T12189 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 4643447 T12189 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 4643447 T12189 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 4643447 T12189 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 4643448 T12189 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4643448 T12189 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4643448 T12189 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 4643449 T12189 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4643450 T12189 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4643450 T12189 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 4643451 T12189 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 4643451 T12189 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 4643452 T12189 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 4643452 T12189 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 4643453 T12189 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 4643453 T12189 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 4643454 T12189 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 4643454 T12189 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 4643455 T12189 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 4643455 T12189 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 4643456 T12189 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 4643464 T12189 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4643466 T12189 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4643467 T12189 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4643468 T12189 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4643471 T12189 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 4643471 T12189 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 4643472 T12189 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@4fca0168
   [junit4]   2> 4643472 T12189 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2f0a72e9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1a79ebd3),segFN=segments_1,generation=1}
   [junit4]   2> 4643472 T12189 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 4643473 T12189 oass.SolrIndexSearcher.<init> Opening Searcher@73bac3b3[collection1] main
   [junit4]   2> 4643473 T12189 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 4643476 T12189 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4643477 T12189 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 4643478 T12189 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 4643479 T12189 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 4643479 T12189 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4643484 T12189 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
   [junit4]   2> 4643484 T12189 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4643485 T12189 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
   [junit4]   2> 4643487 T12189 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 4643493 T12190 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@73bac3b3[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 4643499 T12189 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 4643502 T12193 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:52373/fq_grh collection:collection1 shard:shard2
   [junit4]   2> 4643503 T12138 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 4643503 T12138 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 4643504 T12193 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 4643519 T12193 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 4643525 T12193 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 4643525 T12193 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C2907 name=collection1 org.apache.solr.core.SolrCore@5fdeb546 url=https://127.0.0.1:52373/fq_grh/collection1 node=127.0.0.1:52373_fq_grh C2907_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=https://127.0.0.1:52373/fq_grh, node_name=127.0.0.1:52373_fq_grh, state=down}
   [junit4]   2> 4643525 T12193 C2907 P52373 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:52373/fq_grh/collection1/
   [junit4]   2> 4643526 T12193 C2907 P52373 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 4643526 T12193 C2907 P52373 oasc.SyncStrategy.syncToMe https://127.0.0.1:52373/fq_grh/collection1/ has no replicas
   [junit4]   2> 4643526 T12193 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:52373/fq_grh/collection1/ shard2
   [junit4]   2> 4643527 T12193 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 4643530 T12167 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 4643571 T12174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 4643572 T12167 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 4643573 T12188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 4643573 T12194 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 4643610 T12193 oasc.ZkController.register We are https://127.0.0.1:52373/fq_grh/collection1/ and leader is https://127.0.0.1:52373/fq_grh/collection1/
   [junit4]   2> 4643610 T12193 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:52373/fq_grh
   [junit4]   2> 4643610 T12193 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 4643611 T12193 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 4643611 T12193 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 4643614 T12194 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 4643625 T12163 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:52373/fq_grh",
   [junit4]   2> 	  "node_name":"127.0.0.1:52373_fq_grh",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 4643735 T12174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 4643735 T12194 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 4643737 T12188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 4645392 T12138 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 4645393 T12138 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 4645403 T12138 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 4645404 T12138 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:52376
   [junit4]   2> 4645407 T12138 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 4645407 T12138 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4645408 T12138 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-004
   [junit4]   2> 4645408 T12138 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-004/'
   [junit4]   2> 4645454 T12138 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-004/solr.xml
   [junit4]   2> 4645470 T12138 oasc.CoreContainer.<init> New CoreContainer 1802668282
   [junit4]   2> 4645471 T12138 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-004/]
   [junit4]   2> 4645471 T12138 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 4645472 T12138 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 4645472 T12138 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 4645472 T12138 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 4645473 T12138 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 4645473 T12138 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 4645474 T12138 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 4645474 T12138 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 4645474 T12138 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 4645477 T12138 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 4645479 T12138 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4645480 T12138 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4645480 T12138 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 4645480 T12138 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52365/solr
   [junit4]   2> 4645480 T12138 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 4645481 T12138 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 4645482 T12138 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 4645484 T12205 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@733cb183 name:ZooKeeperConnection Watcher:127.0.0.1:52365 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4645486 T12138 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 4645486 T12138 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 4645490 T12138 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 4645492 T12207 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61d9c451 name:ZooKeeperConnection Watcher:127.0.0.1:52365/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 4645493 T12138 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 4645515 T12138 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 4646531 T12138 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52376_fq_grh
   [junit4]   2> 4646536 T12138 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52376_fq_grh
   [junit4]   2> 4646555 T12138 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 4646567 T12209 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 4646568 T12209 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 4646570 T12209 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 4646570 T12194 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 4646582 T12163 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:52376/fq_grh",
   [junit4]   2> 	  "node_name":"127.0.0.1:52376_fq_grh",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 4646582 T12163 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 4646582 T12163 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 4646591 T12194 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 4646591 T12188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 4646591 T12174 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 4646591 T12208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 4647572 T12209 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 4647572 T12209 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 4647574 T12209 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 4647574 T12209 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 4647578 T12209 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4647578 T12209 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-004/collection1/'
   [junit4]   2> 4647579 T12209 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-004/collection1/lib/.svn/' to classloader
   [junit4]   2> 4647580 T12209 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-004/collection1/lib/classes/' to classloader
   [junit4]   2> 4647581 T12209 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-004/collection1/lib/README' to classloader
   [junit4]   2> 4647620 T12209 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 4647629 T12209 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4647633 T12209 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 4647637 T12209 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 4647749 T12209 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 4647756 T12209 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 4647758 T12209 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 4647784 T12209 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 4647787 T12209 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 4647792 T12209 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4647796 T12209 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 4647797 T12209 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 4647797 T12209 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 4647800 T12209 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 4647801 T12209 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 4647801 T12209 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 4647801 T12209 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 4647802 T12209 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-004/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-001/jetty2/
   [junit4]   2> 4647802 T12209 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@31f4a70e
   [junit4]   2> 4647803 T12209 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-001/jetty2
   [junit4]   2> 4647803 T12209 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-001/jetty2/index/
   [junit4]   2> 4647803 T12209 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 4647804 T12209 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteShardTest-C8AF289A6ACC0F0C-001/tempDir-001/jetty2/index
   [junit4]   2> 4647804 T12209 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=50, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8547504380703527]
   [junit4]   2> 4647805 T12209 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2e418c35 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@293ac72),segFN=segments_1,generation=1}
   [junit4]   2> 4647805 T12209 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 4647809 T12209 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 4647810 T12209 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 4647811 T12209 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 4647811 T12209 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4647811 T12209 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4647812 T12209 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 4647812 T12209 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4647813 T12209 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4647813 T12209 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 4647813 T12209 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 4647814 T12209 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 4647814 T12209 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 4647815 T12209 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 4647815 T12209 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 4647815 T12209 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 4647816 T12209 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 4647816 T12209 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 4647817 T12209 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 4647817 T12209 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 4647818 T12209 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 4647825 T12209 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4647827 T12209 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4647829 T12209 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4647831 T12209 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4647833 T12209 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 4647834 T12209 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 4647834 T12209 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@5ddcd4ae
   [junit4]   2> 4647835 T12209 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2e418c35 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@293ac72),segFN=segments_1,generation=1}
   [junit4]   2> 4647835 T12209 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 4647836 T12209 oass.SolrIndexSearcher.<init> Opening Searcher@5870159[collection1] main
   [junit4]   2> 4647836 T12209 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 4647839 T12209 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4647839 T12209 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 4647841 T12209 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 4647841 T12209 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 4647842 T12209 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4647845 T12209 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
   [junit4]   2> 4647845 T12209 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 4647845 T12209 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
   [junit4]   2> 4647846 T12209 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 4647849 T12210 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5870159[collection1] main{UninvertingDirectoryReader()}
   [junit4]   2> 4647851 T12209 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 4647853 T12213 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:52376/fq_grh collection:collection1 shard:shard1
   [junit4]   2> 4647853 T12138 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 4647854 T12138 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 4647855 T12213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 4647864 T12213 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 4647873 T12194 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 4647886 T12213 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 4647886 T12213 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C2908 name=collection1 org.apache.solr.core.SolrCore@3a9b7dc8 url=https://127.0.0.1:52376/fq_grh/collection1 node=127.0.0.1:52376_fq_grh C2908_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=https://127.0.0.1:52376/fq_grh, node_name=127.0.0.1:52376_fq_grh, state=down}
   [junit4]   2> 4647887 T12213 C2908 P52376 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:52376/fq_grh/collection1/
   [junit4]   2> 4647887 T12213 C2908 P52376 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 4647887 T12213 C2908 P52376 oasc.SyncStrategy.syncToMe https://127.0.0.1:52376/fq_grh/collection1/ has no replicas
   [junit4]   2> 4647888 T12213 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:52376/fq_grh/collection1/ shard1
   [junit4]   2> 4647888 T12213 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 4647903 T12202 oasha.CollectionsHandler.handleDeleteShardAction Deleting Shard : action=DELETESHARD&collection=collection1&shard=shard1&wt=javabin&version=2
   [junit4]   2> 4647905 T12194 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 4647906 T12164 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/

[...truncated too long message...]

faultOperatorResource, QueryParsingTest, TestRangeQuery, DistributedQueryComponentCustomSortTest, TestStressUserVersions, TestReRankQParserPlugin, TestIntervalFaceting, ZkControllerTest, TestBlendedInfixSuggestions, AddSchemaFieldsUpdateProcessorFactoryTest, TestLeaderElectionZkExpiry, HighlighterConfigTest, ExternalCollectionsTest, AtomicUpdatesTest, TestCopyFieldCollectionResource, TestCharFilters, DOMUtilTest, TestElisionMultitermQuery, ChaosMonkeySafeLeaderTest, ReturnFieldsTest, TestPostingsSolrHighlighter, TestCollapseQParserPlugin, DeleteInactiveReplicaTest, TestSort, SolrIndexConfigTest, FieldFacetTest, TestShardHandlerFactory, TermsComponentTest, TestDistributedGrouping, TestWriterPerf, SimplePostToolTest, BlockCacheTest, TestExpandComponent, TestXIncludeConfig, DistributedIntervalFacetingTest, JsonLoaderTest, HdfsCollectionsAPIDistributedZkTest, SolrRequestParserTest, TestFunctionQuery, TestSolrCoreProperties, DisMaxRequestHandlerTest, HttpPartitionTest, TestSchemaResource, TestGroupingSearch, HighlighterMaxOffsetTest, TestIBSimilarityFactory, DistanceFunctionTest, FieldFacetExtrasTest, RangeFacetTest, ZkNodePropsTest, DirectSolrConnectionTest, ExpressionTest, ChaosMonkeyNothingIsSafeTest, DistribDocExpirationUpdateProcessorTest, TriLevelCompositeIdRoutingTest, SignatureUpdateProcessorFactoryTest, TestDefaultSearchFieldResource, TestCloudSchemaless, OutOfBoxZkACLAndCredentialsProvidersTest, BadCopyFieldTest, TestDistributedMissingSort, TestDistribDocBasedVersion, QueryElevationComponentTest, DistributedFacetPivotSmallTest, CurrencyFieldOpenExchangeTest, CopyFieldTest, TestHighFrequencyDictionaryFactory, TestLFUCache, TestStressReorder, SuggesterWFSTTest, TestManagedSchemaFieldTypeResource, TestFastLRUCache, FileUtilsTest, TestMiniSolrCloudCluster, TestSolr4Spatial, HdfsBasicDistributedZkTest, TestShortCircuitedRequests, PathHierarchyTokenizerFactoryTest, TestManagedSynonymFilterFactory, HdfsDirectoryTest, OverseerRolesTest, ParsingFieldUpdateProcessorsTest, TestJoin, TestMultiCoreConfBootstrap, DocumentAnalysisRequestHandlerTest, TestRandomMergePolicy, DistributedExpandComponentTest, LeaderElectionIntegrationTest, DistributedQueryComponentOptimizationTest, TestCursorMarkWithoutUniqueKey, AddBlockUpdateTest, VMParamsZkACLAndCredentialsProvidersTest, TestPivotHelperCode, AbstractAnalyticsFacetTest, WordBreakSolrSpellCheckerTest, MultiThreadedOCPTest, TestSearcherReuse, TestFuzzyAnalyzedSuggestions, CollectionsAPIDistributedZkTest, TestRemoteStreaming, HdfsUnloadDistributedZkTest, TestBM25SimilarityFactory, TestMaxScoreQueryParser, TestSolrJ, RollingRestartTest, TestSchemaVersionResource, DeleteReplicaTest, ConvertedLegacyTest, DistributedQueryElevationComponentTest, TestDynamicFieldCollectionResource, UpdateParamsTest, CollectionsAPIAsyncDistributedZkTest, DefaultValueUpdateProcessorTest, StressHdfsTest, QueryFacetTest, TestChildDocTransformer, IndexBasedSpellCheckerTest, TestMissingGroups, SpellCheckCollatorTest, PreAnalyzedUpdateProcessorTest, DistributedTermsComponentTest, TestSystemIdResolver, TestFaceting, TestOverriddenPrefixQueryForCustomFieldType, TestRandomDVFaceting, DateRangeFieldTest, CacheHeaderTest, IndexSchemaTest, SOLR749Test, SuggestComponentTest, TestQuerySenderListener, XsltUpdateRequestHandlerTest, CurrencyFieldXmlFileTest, TestReversedWildcardFilterFactory, TestStandardQParsers, TestArbitraryIndexDir, AlternateDirectoryTest, SolrTestCaseJ4Test, EnumFieldTest, DistributedSuggestComponentTest, TestSchemaSimilarityResource, TestIndexSearcher, TestExtendedDismaxParser, FullSolrCloudDistribCmdsTest, AsyncMigrateRouteKeyTest, TestUniqueKeyFieldResource, SolrPluginUtilsTest, TestIndexingPerformance, TestParamSet, TestUpdate, TestQueryTypes, CoreMergeIndexesAdminHandlerTest, ClusterStateTest, TestSimpleQParserPlugin, TestSurroundQueryParser, MergeStrategyTest, TestAnalyzedSuggestions, TestReplicationHandler, TestComplexPhraseQParserPlugin, JSONWriterTest, TestReplicationHandlerBackup, EchoParamsTest, TestRealTimeGet, BasicZkTest, DistributedFacetPivotLargeTest, HdfsLockFactoryTest, BasicDistributedZkTest, UpdateRequestProcessorFactoryTest, FieldAnalysisRequestHandlerTest, CursorMarkTest, TestCustomSort, TestRestManager, TestDFRSimilarityFactory, SuggesterTSTTest, SolrIndexSplitterTest, TestManagedResourceStorage, RecoveryZkTest, RemoteQueryErrorTest, URLClassifyProcessorTest, TestInfoStreamLogging, TestBadConfig, XmlUpdateRequestHandlerTest, AnalyticsQueryTest, HdfsChaosMonkeySafeLeaderTest, UniqFieldsUpdateProcessorFactoryTest, TestStressVersions, AssignTest, SpellingQueryConverterTest, AnalysisAfterCoreReloadTest, TestCollationFieldDocValues, TestFieldSortValues, TestReload, ResponseHeaderTest, SolrCmdDistributorTest, TestManagedResource, TestCSVLoader, QueryEqualityTest, ReplicationFactorTest, OpenExchangeRatesOrgProviderTest, CSVRequestHandlerTest, TimeZoneUtilsTest, TestClassNameShortening, TestCollationField, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, OverseerTest, ShardRoutingTest, ClusterStateUpdateTest, TestRandomFaceting, LeaderElectionTest, ZkSolrClientTest, TestZkChroot, ZkCLITest, TestDistributedSearch, AutoCommitTest, TestRecovery, TestHashPartitioner, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, HardAutoCommitTest, TestCoreContainer, SimpleFacetsTest, SolrCoreTest, StatsComponentTest, PeerSyncTest, BadIndexSchemaTest, TestFiltering, BasicFunctionalityTest, TestLazyCores, DirectUpdateHandlerTest, OverseerCollectionProcessorTest, HighlighterTest, SoftAutoCommitTest, ShowFileRequestHandlerTest, TestCoreDiscovery, SuggesterFSTTest, CoreAdminHandlerTest, TestFoldingMultitermQuery, DocValuesTest, SuggesterTest, TestStressLucene, TestTrie, SpatialFilterTest, PolyFieldTest, NoCacheHeaderTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, DirectUpdateHandlerOptimizeTest, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, DocValuesMultiTest, TestSolrDeletionPolicy1, SolrInfoMBeanTest, DebugComponentTest, LukeRequestHandlerTest, TestQueryUtils, StandardRequestHandlerTest, DirectSolrSpellCheckerTest, PrimitiveFieldTypeTest, TestOmitPositions, FileBasedSpellCheckerTest, RequestHandlersTest, DocumentBuilderTest, TestValueSourceCache, TermVectorComponentTest, MoreLikeThisHandlerTest, RequiredFieldsTest, TestSolrQueryParser, FastVectorHighlighterTest, IndexSchemaRuntimeFieldTest, LoggingHandlerTest, RegexBoostProcessorTest, TestJmxIntegration, TestCSVResponseWriter, MBeansHandlerTest, TestPartialUpdateDeduplication, TestPHPSerializedResponseWriter, TestComponentsName, SearchHandlerTest, CoreContainerCoreInitFailuresTest, TestBinaryResponseWriter, TestSolrIndexConfig, TestQuerySenderNoQuery, ResponseLogComponentTest, TestStressRecovery, TestSolrDeletionPolicy2, MultiTermTest, SampleTest, TestDocSet, TestBinaryField, TestSearchPerf, NumericFieldsTest, MinimalSchemaTest, TestConfig, OutputWriterTest, ExternalFileFieldSortTest, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestPhraseSuggestions, SpellPossibilityIteratorTest, TestCodecSupport, SynonymTokenizerTest, TestSweetSpotSimilarityFactory, TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestFastWriter, TestJmxMonitoredMap, ResourceLoaderTest, TestFastOutputStream, ScriptEngineTest, PluginInfoTest, DateMathParserTest, PreAnalyzedFieldTest, PrimUtilsTest, TestSuggestSpellingConverter, DateFieldTest, RAMDirectoryFactoryTest, TestLRUCache, TestUtils, TestDocumentBuilder, SliceStateTest, SystemInfoHandlerTest, UUIDFieldTest, CircularListTest, TestRTGBase, CursorPagingTest, TestHighlightDedupGrouping, TestTolerantSearch, AbstractAnalyticsStatsTest, NoFacetTest, FunctionTest, AliasIntegrationTest, ConnectionManagerTest, CustomCollectionTest, DeleteLastCustomShardedReplicaTest, DeleteShardTest, DistribCursorPagingTest, MigrateRouteKeyTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerStatusTest, SSLMigrationTest, ShardSplitTest, SharedFSAutoReplicaFailoverUtilsTest, SolrXmlInZkTest, TestCloudInspectUtil, TestCloudPivotFacet, TestCollectionAPI, TestModifyConfFiles, TestRequestStatusCollectionAPI, HdfsBasicDistributedZk2Test, HdfsRecoveryZkTest, HdfsSyncSliceTest, HdfsWriteToMultipleCollectionsTest, CachingDirectoryFactoryTest, QueryResultKeyTest, TestConfigSets, TestImplicitCoreProperties, TestNRTOpen, TestNonNRTOpen]
   [junit4] Completed in 0.87s, 2 tests, 1 failure <<< FAILURES!

[...truncated 152 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:491: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:471: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:60: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/extra-targets.xml:39: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:189: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:496: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1358: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:961: There were test failures: 438 suites, 1830 tests, 1 error, 1 failure, 50 ignored (22 assumptions)

Total time: 149 minutes 54 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any