[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3996 - Unstable!

[Policeman Jenkins Server <je...@thetaphi.de>]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3996/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

58 tests failed.
FAILED:  org.apache.solr.cloud.CdcrVersionReplicationTest.testCdcrDocVersions

Error Message:
The collection (tmp_collection) is still present - waited for 15 seconds

Stack Trace:
java.lang.AssertionError: The collection (tmp_collection) is still present - waited for 15 seconds
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:66646A1C9F855AFB]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForCollectionToDisappear(AbstractDistribZkTestBase.java:218)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForCollectionToDisappear(BaseCdcrDistributedZkTest.java:498)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.startServers(BaseCdcrDistributedZkTest.java:599)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.createSourceCollection(BaseCdcrDistributedZkTest.java:348)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.baseBefore(BaseCdcrDistributedZkTest.java:167)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:941)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionStateFormat2Test.testZkNodeLocation

Error Message:
Error from server at http://127.0.0.1:56767/solr: Could not fully remove collection: myExternColl

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:56767/solr: Could not fully remove collection: myExternColl
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:D15FEB1EC08C3B13]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.CollectionStateFormat2Test.testZkNodeLocation(CollectionStateFormat2Test.java:63)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionTooManyReplicasTest.testDownedShards

Error Message:
Error from server at http://127.0.0.1:55886/solr: Could not fully remove collection: TooManyReplicasWhenAddingShards

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:55886/solr: Could not fully remove collection: TooManyReplicasWhenAddingShards
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:FCDAC9EBE618F8A3]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:442)
	at org.apache.solr.cloud.CollectionTooManyReplicasTest.deleteCollections(CollectionTooManyReplicasTest.java:47)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:941)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionsAPIAsyncDistributedZkTest.testAsyncRequests

Error Message:
DeleteCollection did not complete expected same:<COMPLETED> was not:<FAILED>

Stack Trace:
java.lang.AssertionError: DeleteCollection did not complete expected same:<COMPLETED> was not:<FAILED>
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:7AB65D09CB4BFB38]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotSame(Assert.java:641)
	at org.junit.Assert.assertSame(Assert.java:580)
	at org.apache.solr.cloud.CollectionsAPIAsyncDistributedZkTest.testAsyncRequests(CollectionsAPIAsyncDistributedZkTest.java:204)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCreationAndDeletion

Error Message:
Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: acollectionafterbaddelete

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: acollectionafterbaddelete
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:ACF6C06E66F7E646]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:442)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.clearCluster(CollectionsAPIDistributedZkTest.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:941)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testMaxNodesPerShard

Error Message:
Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: nodeset_collection

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: nodeset_collection
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:7289EB7D3444A461]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:442)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.clearCluster(CollectionsAPIDistributedZkTest.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:941)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection

Error Message:
Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:B4720D5BDA39952F]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection(CollectionsAPIDistributedZkTest.java:172)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCoresAreDistributedAcrossNodes

Error Message:
Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:FAB2B4D2D62D3DF6]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:442)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.clearCluster(CollectionsAPIDistributedZkTest.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:941)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testTooManyReplicas

Error Message:
Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:6B53EC62E4AE30A7]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:442)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.clearCluster(CollectionsAPIDistributedZkTest.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:941)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testMissingNumShards

Error Message:
Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:96A2C81F26BD3581]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:442)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.clearCluster(CollectionsAPIDistributedZkTest.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:941)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.deleteCollectionRemovesStaleZkCollectionsNode

Error Message:
Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:1C5B71DF43B8897A]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:442)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.clearCluster(CollectionsAPIDistributedZkTest.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:941)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testBadActionNames

Error Message:
Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:251E38280060A1B0]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:442)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.clearCluster(CollectionsAPIDistributedZkTest.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:941)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.addReplicaTest

Error Message:
Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:D523698E3EBFE36]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:442)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.clearCluster(CollectionsAPIDistributedZkTest.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:941)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testNoConfigSetExist

Error Message:
Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:C2C0DA677EE57EBE]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:442)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.clearCluster(CollectionsAPIDistributedZkTest.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:941)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testZeroNumShards

Error Message:
Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:CD41CD492671412D]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:442)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.clearCluster(CollectionsAPIDistributedZkTest.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:941)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCreateShouldFailOnExistingCore

Error Message:
Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:7234CC0A967F0803]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:442)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.clearCluster(CollectionsAPIDistributedZkTest.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:941)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets

Error Message:
Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:B38C2EE49AC70FEB]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:442)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.clearCluster(CollectionsAPIDistributedZkTest.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:941)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testNoCollectionSpecified

Error Message:
Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:56046/solr: Could not fully remove collection: halfdeletedcollection
	at __randomizedtesting.SeedInfo.seed([9EF261BE6DE3B5E7:ED51E14ED98BE0C5]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:442)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.clearCluster(CollectionsAPIDistributedZkTest.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at

[...truncated too long message...]

ntCnxn$SendThread.startConnect(ClientCnxn.java:997)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
   [junit4]    >   29) Thread[id=2946, name=zkCallback-149-thread-3, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >   30) Thread[id=2153, name=zkCallback-840-thread-1, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >   31) Thread[id=2933, name=zkCallback-188-thread-4, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >   32) Thread[id=2937, name=zkCallback-149-thread-2, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >   33) Thread[id=2943, name=zkCallback-840-thread-2, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >   34) Thread[id=774, name=zkCallback-188-thread-1, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >   35) Thread[id=2941, name=zkCallback-845-thread-2, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >   36) Thread[id=2930, name=zkCallback-188-thread-2, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >   37) Thread[id=2939, name=zkCallback-172-thread-3, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >   38) Thread[id=772, name=TEST-StreamExpressionTest.testParallelDaemonCommitStream-seed#[A267C743CF9D9ABB]-SendThread(127.0.0.1:58431), state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A267C743CF9D9ABB]:0)
   [junit4] Completed [117/128 (2!)] on J1 in 472.16s, 68 tests, 14 errors <<< FAILURES!

[...truncated 42636 lines...]

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 4000 - Failure!

[Policeman Jenkins Server <je...@thetaphi.de>]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4000/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

All tests passed

Build Log:
[...truncated 4467 lines...]
   [junit4] JVM J1: stdout was not empty, see: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/backward-codecs/test/temp/junit4-J1-20170510_204058_0456242731457081395094.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] #
   [junit4] # A fatal error has been detected by the Java Runtime Environment:
   [junit4] #
   [junit4] #  SIGFPE (0x8) at pc=0x00007fff8a2ec143, pid=20546, tid=0x0000000000004003
   [junit4] #
   [junit4] # JRE version: Java(TM) SE Runtime Environment (8.0_131-b11) (build 1.8.0_131-b11)
   [junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.131-b11 mixed mode bsd-amd64 compressed oops)
   [junit4] # Problematic frame:
   [junit4] # C  [libsystem_kernel.dylib+0x11143]  __commpage_gettimeofday+0x43
   [junit4] #
   [junit4] # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
   [junit4] #
   [junit4] # An error report file with more information is saved as:
   [junit4] # /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/backward-codecs/test/J1/hs_err_pid20546.log
   [junit4] #
   [junit4] # If you would like to submit a bug report, please visit:
   [junit4] #   http://bugreport.java.com/bugreport/crash.jsp
   [junit4] #
   [junit4] <<< JVM J1: EOF ----

[...truncated 2 lines...]
   [junit4] ERROR: JVM J1 ended with an exception, command line: /Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre/bin/java -XX:+UseCompressedOops -XX:+UseSerialGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=C571A774EC9A9581 -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=7.0.0 -Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=1 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/backward-codecs/test/temp -Dcommon.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene -Dclover.db.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/clover/db -Djava.security.policy=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/tools/junit4/tests.policy -Dtests.LUCENE_VERSION=7.0.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.src.home=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX -Djunit4.childvm.cwd=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/backward-codecs/test/J1 -Djunit4.childvm.id=1 -Djunit4.childvm.count=2 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Dtests.disableHdfs=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Dfile.encoding=US-ASCII -classpath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/backward-codecs/classes/test:/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/test-framework/classes/java:/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/codecs/classes/java:/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java:/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/test-framework/lib/junit-4.10.jar:/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/test-framework/lib/randomizedtesting-runner-2.5.0.jar:/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/backward-codecs/classes/java:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/Users/jenkins/.ant/lib/ivy-2.3.0.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/lib/tools.jar:/Users/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.5.0.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/backward-codecs/test/temp/junit4-J1-20170510_204058_0455933829974317453865.events @/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/backward-codecs/test/temp/junit4-J1-20170510_204058_0456405981841473641645.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Forked process returned with error code: 134. Very likely a JVM crash.  See process stdout at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/backward-codecs/test/temp/junit4-J1-20170510_204058_0456242731457081395094.sysout
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:748)

BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:775: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:719: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:59: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build.xml:482: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/common-build.xml:2179: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/module-build.xml:58: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/common-build.xml:1455: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/common-build.xml:985: At least one slave process threw an exception, first: Forked process returned with error code: 134. Very likely a JVM crash.  See process stdout at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/backward-codecs/test/temp/junit4-J1-20170510_204058_0456242731457081395094.sysout

Total time: 17 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3999 - Still Unstable!

[Policeman Jenkins Server <je...@thetaphi.de>]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3999/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  org.apache.solr.cloud.CustomCollectionTest.testCustomCollectionsAPI

Error Message:
Error from server at http://127.0.0.1:65304/solr/implicitcoll: Async exception during distributed update: Error from server at http://127.0.0.1:65306/solr/implicitcoll_c_replica4: Server Error    request: http://127.0.0.1:65306/solr/implicitcoll_c_replica4/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A65304%2Fsolr%2Fimplicitcoll_b_replica4%2F&wt=javabin&version=2 Remote error message: java.io.FileNotFoundException: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node2/implicitcoll_c_replica4/data/tlog/tlog.0000000000000000001 (Too many open files)

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:65304/solr/implicitcoll: Async exception during distributed update: Error from server at http://127.0.0.1:65306/solr/implicitcoll_c_replica4: Server Error



request: http://127.0.0.1:65306/solr/implicitcoll_c_replica4/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A65304%2Fsolr%2Fimplicitcoll_b_replica4%2F&wt=javabin&version=2
Remote error message: java.io.FileNotFoundException: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node2/implicitcoll_c_replica4/data/tlog/tlog.0000000000000000001 (Too many open files)
	at __randomizedtesting.SeedInfo.seed([C3E823C8F8473366:A909ADA3C5DD851E]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:451)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:392)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:895)
	at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:858)
	at org.apache.solr.cloud.CustomCollectionTest.testCustomCollectionsAPI(CustomCollectionTest.java:86)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CustomCollectionTest.testRouteFieldForImplicitRouter

Error Message:
Collection not found: withShardField

Stack Trace:
org.apache.solr.common.SolrException: Collection not found: withShardField
	at __randomizedtesting.SeedInfo.seed([C3E823C8F8473366:96B8CB5A54BEFC96]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:1401)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1094)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
	at org.apache.solr.cloud.CustomCollectionTest.testRouteFieldForImplicitRouter(CustomCollectionTest.java:141)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12023 lines...]
   [junit4] Suite: org.apache.solr.cloud.CustomCollectionTest
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/init-core-data-001
   [junit4]   2> 2033425 WARN  (SUITE-CustomCollectionTest-seed#[C3E823C8F8473366]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
   [junit4]   2> 2033425 INFO  (SUITE-CustomCollectionTest-seed#[C3E823C8F8473366]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields
   [junit4]   2> 2033430 INFO  (SUITE-CustomCollectionTest-seed#[C3E823C8F8473366]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN) w/ MAC_OS_X supressed clientAuth
   [junit4]   2> 2033431 INFO  (SUITE-CustomCollectionTest-seed#[C3E823C8F8473366]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001
   [junit4]   2> 2033431 INFO  (SUITE-CustomCollectionTest-seed#[C3E823C8F8473366]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2033431 INFO  (Thread-3013) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2033431 INFO  (Thread-3013) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2033436 ERROR (Thread-3013) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2033541 INFO  (SUITE-CustomCollectionTest-seed#[C3E823C8F8473366]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:65300
   [junit4]   2> 2033580 INFO  (jetty-launcher-2128-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2033580 INFO  (jetty-launcher-2128-thread-3) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2033580 INFO  (jetty-launcher-2128-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2033581 INFO  (jetty-launcher-2128-thread-4) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2033583 INFO  (jetty-launcher-2128-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@46bbb500{/solr,null,AVAILABLE}
   [junit4]   2> 2033585 INFO  (jetty-launcher-2128-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@158a1221{HTTP/1.1,[http/1.1]}{127.0.0.1:65302}
   [junit4]   2> 2033585 INFO  (jetty-launcher-2128-thread-1) [    ] o.e.j.s.Server Started @2039648ms
   [junit4]   2> 2033585 INFO  (jetty-launcher-2128-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=65302}
   [junit4]   2> 2033586 ERROR (jetty-launcher-2128-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2033586 INFO  (jetty-launcher-2128-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@24a7481a{/solr,null,AVAILABLE}
   [junit4]   2> 2033586 INFO  (jetty-launcher-2128-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2033586 INFO  (jetty-launcher-2128-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2033586 INFO  (jetty-launcher-2128-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2033586 INFO  (jetty-launcher-2128-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-05-10T10:24:49.331Z
   [junit4]   2> 2033587 INFO  (jetty-launcher-2128-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1630ce21{/solr,null,AVAILABLE}
   [junit4]   2> 2033587 INFO  (jetty-launcher-2128-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4ebfbf23{/solr,null,AVAILABLE}
   [junit4]   2> 2033587 INFO  (jetty-launcher-2128-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@422cd289{HTTP/1.1,[http/1.1]}{127.0.0.1:65304}
   [junit4]   2> 2033587 INFO  (jetty-launcher-2128-thread-3) [    ] o.e.j.s.Server Started @2039650ms
   [junit4]   2> 2033587 INFO  (jetty-launcher-2128-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=65304}
   [junit4]   2> 2033587 ERROR (jetty-launcher-2128-thread-3) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2033587 INFO  (jetty-launcher-2128-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@76770a25{HTTP/1.1,[http/1.1]}{127.0.0.1:65306}
   [junit4]   2> 2033587 INFO  (jetty-launcher-2128-thread-2) [    ] o.e.j.s.Server Started @2039650ms
   [junit4]   2> 2033587 INFO  (jetty-launcher-2128-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=65306}
   [junit4]   2> 2033587 ERROR (jetty-launcher-2128-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2033587 INFO  (jetty-launcher-2128-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1a9df8{HTTP/1.1,[http/1.1]}{127.0.0.1:65303}
   [junit4]   2> 2033587 INFO  (jetty-launcher-2128-thread-4) [    ] o.e.j.s.Server Started @2039650ms
   [junit4]   2> 2033587 INFO  (jetty-launcher-2128-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=65303}
   [junit4]   2> 2033587 ERROR (jetty-launcher-2128-thread-4) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2033587 INFO  (jetty-launcher-2128-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2033587 INFO  (jetty-launcher-2128-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2033587 INFO  (jetty-launcher-2128-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2033587 INFO  (jetty-launcher-2128-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-05-10T10:24:49.332Z
   [junit4]   2> 2033593 INFO  (jetty-launcher-2128-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2033593 INFO  (jetty-launcher-2128-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2033593 INFO  (jetty-launcher-2128-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2033593 INFO  (jetty-launcher-2128-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-05-10T10:24:49.338Z
   [junit4]   2> 2033593 INFO  (jetty-launcher-2128-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 2033593 INFO  (jetty-launcher-2128-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2033593 INFO  (jetty-launcher-2128-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2033593 INFO  (jetty-launcher-2128-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-05-10T10:24:49.338Z
   [junit4]   2> 2033597 INFO  (jetty-launcher-2128-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2033599 INFO  (jetty-launcher-2128-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2033600 INFO  (jetty-launcher-2128-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2033601 INFO  (jetty-launcher-2128-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2033619 INFO  (jetty-launcher-2128-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:65300/solr
   [junit4]   2> 2033620 INFO  (jetty-launcher-2128-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:65300/solr
   [junit4]   2> 2033625 INFO  (jetty-launcher-2128-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:65300/solr
   [junit4]   2> 2033625 INFO  (jetty-launcher-2128-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:65300/solr
   [junit4]   2> 2033677 INFO  (jetty-launcher-2128-thread-2) [n:127.0.0.1:65306_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2033678 INFO  (jetty-launcher-2128-thread-3) [n:127.0.0.1:65304_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2033678 INFO  (jetty-launcher-2128-thread-1) [n:127.0.0.1:65302_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2033678 INFO  (jetty-launcher-2128-thread-4) [n:127.0.0.1:65303_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2033680 INFO  (jetty-launcher-2128-thread-2) [n:127.0.0.1:65306_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:65306_solr
   [junit4]   2> 2033681 INFO  (jetty-launcher-2128-thread-3) [n:127.0.0.1:65304_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:65304_solr
   [junit4]   2> 2033681 INFO  (jetty-launcher-2128-thread-4) [n:127.0.0.1:65303_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:65303_solr
   [junit4]   2> 2033682 INFO  (jetty-launcher-2128-thread-1) [n:127.0.0.1:65302_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:65302_solr
   [junit4]   2> 2033683 INFO  (jetty-launcher-2128-thread-2) [n:127.0.0.1:65306_solr    ] o.a.s.c.Overseer Overseer (id=97937777569300492-127.0.0.1:65306_solr-n_0000000000) starting
   [junit4]   2> 2033689 INFO  (zkCallback-2148-thread-1-processing-n:127.0.0.1:65303_solr) [n:127.0.0.1:65303_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 2033689 INFO  (zkCallback-2149-thread-1-processing-n:127.0.0.1:65304_solr) [n:127.0.0.1:65304_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 2033689 INFO  (zkCallback-2150-thread-1-processing-n:127.0.0.1:65306_solr) [n:127.0.0.1:65306_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 2033690 INFO  (zkCallback-2147-thread-1-processing-n:127.0.0.1:65302_solr) [n:127.0.0.1:65302_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 2033755 INFO  (jetty-launcher-2128-thread-2) [n:127.0.0.1:65306_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:65306_solr
   [junit4]   2> 2033758 INFO  (zkCallback-2149-thread-1-processing-n:127.0.0.1:65304_solr) [n:127.0.0.1:65304_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2033758 INFO  (zkCallback-2148-thread-1-processing-n:127.0.0.1:65303_solr) [n:127.0.0.1:65303_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2033758 INFO  (zkCallback-2147-thread-1-processing-n:127.0.0.1:65302_solr) [n:127.0.0.1:65302_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2033758 INFO  (zkCallback-2150-thread-1-processing-n:127.0.0.1:65306_solr) [n:127.0.0.1:65306_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2033836 INFO  (jetty-launcher-2128-thread-3) [n:127.0.0.1:65304_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65304.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2033837 INFO  (jetty-launcher-2128-thread-4) [n:127.0.0.1:65303_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65303.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2033840 INFO  (jetty-launcher-2128-thread-1) [n:127.0.0.1:65302_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65302.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2033843 INFO  (jetty-launcher-2128-thread-1) [n:127.0.0.1:65302_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65302.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2033843 INFO  (jetty-launcher-2128-thread-1) [n:127.0.0.1:65302_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65302.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2033845 INFO  (jetty-launcher-2128-thread-1) [n:127.0.0.1:65302_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node1/.
   [junit4]   2> 2033846 INFO  (jetty-launcher-2128-thread-4) [n:127.0.0.1:65303_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65303.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2033849 INFO  (jetty-launcher-2128-thread-4) [n:127.0.0.1:65303_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65303.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2033850 INFO  (jetty-launcher-2128-thread-4) [n:127.0.0.1:65303_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node4/.
   [junit4]   2> 2033859 INFO  (jetty-launcher-2128-thread-3) [n:127.0.0.1:65304_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65304.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2033859 INFO  (jetty-launcher-2128-thread-3) [n:127.0.0.1:65304_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65304.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2033860 INFO  (jetty-launcher-2128-thread-3) [n:127.0.0.1:65304_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node3/.
   [junit4]   2> 2033880 INFO  (jetty-launcher-2128-thread-2) [n:127.0.0.1:65306_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65306.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2033884 INFO  (jetty-launcher-2128-thread-2) [n:127.0.0.1:65306_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65306.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2033884 INFO  (jetty-launcher-2128-thread-2) [n:127.0.0.1:65306_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65306.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2033885 INFO  (jetty-launcher-2128-thread-2) [n:127.0.0.1:65306_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node2/.
   [junit4]   2> 2033983 INFO  (SUITE-CustomCollectionTest-seed#[C3E823C8F8473366]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 2033985 INFO  (SUITE-CustomCollectionTest-seed#[C3E823C8F8473366]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:65300/solr ready
   [junit4]   2> 2034021 INFO  (TEST-CustomCollectionTest.testRouteFieldForImplicitRouter-seed#[C3E823C8F8473366]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testRouteFieldForImplicitRouter
   [junit4]   2> 2034024 INFO  (TEST-CustomCollectionTest.testRouteFieldForImplicitRouter-seed#[C3E823C8F8473366]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 2034030 INFO  (qtp1524582488-13048) [n:127.0.0.1:65303_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params shards=a,b,c,d&replicationFactor=5&collection.configName=conf&maxShardsPerNode=6&router.field=shard_s&name=withShardField&router.name=implicit&action=CREATE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2034035 INFO  (OverseerThreadFactory-6149-thread-1-processing-n:127.0.0.1:65306_solr) [n:127.0.0.1:65306_solr    ] o.a.s.c.CreateCollectionCmd Create collection withShardField
   [junit4]   2> 2034035 WARN  (OverseerThreadFactory-6149-thread-1-processing-n:127.0.0.1:65306_solr) [n:127.0.0.1:65306_solr    ] o.a.s.c.CreateCollectionCmd Specified replicationFactor of 5 on collection withShardField is higher than or equal to the number of Solr instances currently live or live and part of your createNodeSet(4). It's unusual to run two replica of the same slice on the same Solr-instance.
   [junit4]   2> 2034153 INFO  (qtp1524582488-13049) [n:127.0.0.1:65303_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_a_replica4&action=CREATE&numShards=4&collection=withShardField&shard=a&wt=javabin&version=2
   [junit4]   2> 2034154 INFO  (qtp1524582488-13049) [n:127.0.0.1:65303_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2034159 INFO  (qtp1524582488-13050) [n:127.0.0.1:65303_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_b_replica3&action=CREATE&numShards=4&collection=withShardField&shard=b&wt=javabin&version=2
   [junit4]   2> 2034159 INFO  (qtp1524582488-13056) [n:127.0.0.1:65303_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_d_replica1&action=CREATE&numShards=4&collection=withShardField&shard=d&wt=javabin&version=2
   [junit4]   2> 2034159 INFO  (qtp1524582488-13058) [n:127.0.0.1:65303_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_c_replica2&action=CREATE&numShards=4&collection=withShardField&shard=c&wt=javabin&version=2
   [junit4]   2> 2034159 INFO  (qtp1524582488-13057) [n:127.0.0.1:65303_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_d_replica5&action=CREATE&numShards=4&collection=withShardField&shard=d&wt=javabin&version=2
   [junit4]   2> 2034162 INFO  (qtp1244449539-13038) [n:127.0.0.1:65306_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_b_replica5&action=CREATE&numShards=4&collection=withShardField&shard=b&wt=javabin&version=2
   [junit4]   2> 2034162 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_b_replica2&action=CREATE&numShards=4&collection=withShardField&shard=b&wt=javabin&version=2
   [junit4]   2> 2034162 INFO  (qtp1454893590-13028) [n:127.0.0.1:65302_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_d_replica2&action=CREATE&numShards=4&collection=withShardField&shard=d&wt=javabin&version=2
   [junit4]   2> 2034162 INFO  (qtp1244449539-13040) [n:127.0.0.1:65306_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_c_replica4&action=CREATE&numShards=4&collection=withShardField&shard=c&wt=javabin&version=2
   [junit4]   2> 2034163 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2034163 INFO  (qtp1244449539-13040) [n:127.0.0.1:65306_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2034163 INFO  (qtp1454893590-13029) [n:127.0.0.1:65302_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_b_replica4&action=CREATE&numShards=4&collection=withShardField&shard=b&wt=javabin&version=2
   [junit4]   2> 2034163 INFO  (qtp1454893590-13028) [n:127.0.0.1:65302_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2034163 INFO  (qtp1244449539-13041) [n:127.0.0.1:65306_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_d_replica3&action=CREATE&numShards=4&collection=withShardField&shard=d&wt=javabin&version=2
   [junit4]   2> 2034164 INFO  (qtp365726319-13055) [n:127.0.0.1:65304_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_c_replica5&action=CREATE&numShards=4&collection=withShardField&shard=c&wt=javabin&version=2
   [junit4]   2> 2034165 INFO  (qtp1454893590-13030) [n:127.0.0.1:65302_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_c_replica3&action=CREATE&numShards=4&collection=withShardField&shard=c&wt=javabin&version=2
   [junit4]   2> 2034162 INFO  (qtp365726319-13046) [n:127.0.0.1:65304_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_d_replica4&action=CREATE&numShards=4&collection=withShardField&shard=d&wt=javabin&version=2
   [junit4]   2> 2034165 INFO  (qtp1244449539-13042) [n:127.0.0.1:65306_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_a_replica2&action=CREATE&numShards=4&collection=withShardField&shard=a&wt=javabin&version=2
   [junit4]   2> 2034166 INFO  (qtp365726319-13060) [n:127.0.0.1:65304_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_a_replica3&action=CREATE&numShards=4&collection=withShardField&shard=a&wt=javabin&version=2
   [junit4]   2> 2034166 INFO  (qtp1454893590-13031) [n:127.0.0.1:65302_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_a_replica5&action=CREATE&numShards=4&collection=withShardField&shard=a&wt=javabin&version=2
   [junit4]   2> 2034166 INFO  (qtp1244449539-13043) [n:127.0.0.1:65306_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_b_replica1&action=CREATE&numShards=4&collection=withShardField&shard=b&wt=javabin&version=2
   [junit4]   2> 2034167 INFO  (qtp365726319-13051) [n:127.0.0.1:65304_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_c_replica1&action=CREATE&numShards=4&collection=withShardField&shard=c&wt=javabin&version=2
   [junit4]   2> 2034167 INFO  (qtp1454893590-13032) [n:127.0.0.1:65302_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_a_replica1&action=CREATE&numShards=4&collection=withShardField&shard=a&wt=javabin&version=2
   [junit4]   2> 2034337 INFO  (zkCallback-2150-thread-1-processing-n:127.0.0.1:65306_solr) [n:127.0.0.1:65306_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withShardField/state.json] for collection [withShardField] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2034337 INFO  (zkCallback-2147-thread-1-processing-n:127.0.0.1:65302_solr) [n:127.0.0.1:65302_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withShardField/state.json] for collection [withShardField] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2034337 INFO  (zkCallback-2148-thread-1-processing-n:127.0.0.1:65303_solr) [n:127.0.0.1:65303_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withShardField/state.json] for collection [withShardField] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2034337 INFO  (zkCallback-2149-thread-1-processing-n:127.0.0.1:65304_solr) [n:127.0.0.1:65304_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withShardField/state.json] for collection [withShardField] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2035230 INFO  (qtp1524582488-13057) [n:127.0.0.1:65303_solr c:withShardField s:d  x:withShardField_d_replica5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035232 INFO  (qtp365726319-13055) [n:127.0.0.1:65304_solr c:withShardField s:c  x:withShardField_c_replica5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035246 INFO  (qtp365726319-13046) [n:127.0.0.1:65304_solr c:withShardField s:d  x:withShardField_d_replica4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035247 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035254 INFO  (qtp1524582488-13058) [n:127.0.0.1:65303_solr c:withShardField s:c  x:withShardField_c_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035254 INFO  (qtp1524582488-13056) [n:127.0.0.1:65303_solr c:withShardField s:d  x:withShardField_d_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035264 INFO  (qtp1524582488-13049) [n:127.0.0.1:65303_solr c:withShardField s:a  x:withShardField_a_replica4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035267 INFO  (qtp1454893590-13031) [n:127.0.0.1:65302_solr c:withShardField s:a  x:withShardField_a_replica5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035269 INFO  (qtp1454893590-13030) [n:127.0.0.1:65302_solr c:withShardField s:c  x:withShardField_c_replica3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035276 INFO  (qtp365726319-13060) [n:127.0.0.1:65304_solr c:withShardField s:a  x:withShardField_a_replica3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035276 INFO  (qtp1244449539-13038) [n:127.0.0.1:65306_solr c:withShardField s:b  x:withShardField_b_replica5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035278 INFO  (qtp1454893590-13028) [n:127.0.0.1:65302_solr c:withShardField s:d  x:withShardField_d_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035297 INFO  (qtp1524582488-13050) [n:127.0.0.1:65303_solr c:withShardField s:b  x:withShardField_b_replica3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035319 INFO  (qtp365726319-13055) [n:127.0.0.1:65304_solr c:withShardField s:c  x:withShardField_c_replica5] o.a.s.s.IndexSchema [withShardField_c_replica5] Schema name=cloud-dynamic
   [junit4]   2> 2035319 INFO  (qtp1244449539-13042) [n:127.0.0.1:65306_solr c:withShardField s:a  x:withShardField_a_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035324 INFO  (qtp1454893590-13032) [n:127.0.0.1:65302_solr c:withShardField s:a  x:withShardField_a_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035330 INFO  (qtp1244449539-13041) [n:127.0.0.1:65306_solr c:withShardField s:d  x:withShardField_d_replica3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035330 INFO  (qtp1454893590-13029) [n:127.0.0.1:65302_solr c:withShardField s:b  x:withShardField_b_replica4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035347 INFO  (qtp1244449539-13040) [n:127.0.0.1:65306_solr c:withShardField s:c  x:withShardField_c_replica4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035350 INFO  (qtp365726319-13051) [n:127.0.0.1:65304_solr c:withShardField s:c  x:withShardField_c_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035356 INFO  (qtp1244449539-13043) [n:127.0.0.1:65306_solr c:withShardField s:b  x:withShardField_b_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2035374 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.s.IndexSchema [withShardField_b_replica2] Schema name=cloud-dynamic
   [junit4]   2> 2035379 INFO  (qtp365726319-13060) [n:127.0.0.1:65304_solr c:withShardField s:a  x:withShardField_a_replica3] o.a.s.s.IndexSchema [withShardField_a_replica3] Schema name=cloud-dynamic
   [junit4]   2> 2035435 INFO  (qtp365726319-13046) [n:127.0.0.1:65304_solr c:withShardField s:d  x:withShardField_d_replica4] o.a.s.s.IndexSchema [withShardField_d_replica4] Schema name=cloud-dynamic
   [junit4]   2> 2035445 INFO  (qtp1454893590-13029) [n:127.0.0.1:65302_solr c:withShardField s:b  x:withShardField_b_replica4] o.a.s.s.IndexSchema [withShardField_b_replica4] Schema name=cloud-dynamic
   [junit4]   2> 2035445 INFO  (qtp1454893590-13032) [n:127.0.0.1:65302_solr c:withShardField s:a  x:withShardField_a_replica1] o.a.s.s.IndexSchema [withShardField_a_replica1] Schema name=cloud-dynamic
   [junit4]   2> 2035446 INFO  (qtp1524582488-13050) [n:127.0.0.1:65303_solr c:withShardField s:b  x:withShardField_b_replica3] o.a.s.s.IndexSchema [withShardField_b_replica3] Schema name=cloud-dynamic
   [junit4]   2> 2035475 INFO  (qtp1244449539-13038) [n:127.0.0.1:65306_solr c:withShardField s:b  x:withShardField_b_replica5] o.a.s.s.IndexSchema [withShardField_b_replica5] Schema name=cloud-dynamic
   [junit4]   2> 2035476 INFO  (qtp1524582488-13057) [n:127.0.0.1:65303_solr c:withShardField s:d  x:withShardField_d_replica5] o.a.s.s.IndexSchema [withShardField_d_replica5] Schema name=cloud-dynamic
   [junit4]   2> 2035480 INFO  (qtp1524582488-13049) [n:127.0.0.1:65303_solr c:withShardField s:a  x:withShardField_a_replica4] o.a.s.s.IndexSchema [withShardField_a_replica4] Schema name=cloud-dynamic
   [junit4]   2> 2035486 INFO  (qtp1244449539-13043) [n:127.0.0.1:65306_solr c:withShardField s:b  x:withShardField_b_replica1] o.a.s.s.IndexSchema [withShardField_b_replica1] Schema name=cloud-dynamic
   [junit4]   2> 2035487 INFO  (qtp1454893590-13030) [n:127.0.0.1:65302_solr c:withShardField s:c  x:withShardField_c_replica3] o.a.s.s.IndexSchema [withShardField_c_replica3] Schema name=cloud-dynamic
   [junit4]   2> 2035501 INFO  (qtp365726319-13051) [n:127.0.0.1:65304_solr c:withShardField s:c  x:withShardField_c_replica1] o.a.s.s.IndexSchema [withShardField_c_replica1] Schema name=cloud-dynamic
   [junit4]   2> 2035509 INFO  (qtp1244449539-13041) [n:127.0.0.1:65306_solr c:withShardField s:d  x:withShardField_d_replica3] o.a.s.s.IndexSchema [withShardField_d_replica3] Schema name=cloud-dynamic
   [junit4]   2> 2035511 INFO  (qtp1244449539-13040) [n:127.0.0.1:65306_solr c:withShardField s:c  x:withShardField_c_replica4] o.a.s.s.IndexSchema [withShardField_c_replica4] Schema name=cloud-dynamic
   [junit4]   2> 2035625 INFO  (qtp1244449539-13042) [n:127.0.0.1:65306_solr c:withShardField s:a  x:withShardField_a_replica2] o.a.s.s.IndexSchema [withShardField_a_replica2] Schema name=cloud-dynamic
   [junit4]   2> 2035627 INFO  (qtp1454893590-13028) [n:127.0.0.1:65302_solr c:withShardField s:d  x:withShardField_d_replica2] o.a.s.s.IndexSchema [withShardField_d_replica2] Schema name=cloud-dynamic
   [junit4]   2> 2035630 INFO  (qtp1454893590-13031) [n:127.0.0.1:65302_solr c:withShardField s:a  x:withShardField_a_replica5] o.a.s.s.IndexSchema [withShardField_a_replica5] Schema name=cloud-dynamic
   [junit4]   2> 2035639 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035639 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_b_replica2' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035639 INFO  (qtp1524582488-13058) [n:127.0.0.1:65303_solr c:withShardField s:c  x:withShardField_c_replica2] o.a.s.s.IndexSchema [withShardField_c_replica2] Schema name=cloud-dynamic
   [junit4]   2> 2035639 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65304.solr.core.withShardField.b.replica2' (registry 'solr.core.withShardField.b.replica2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035639 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035639 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.c.SolrCore [[withShardField_b_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node3/withShardField_b_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node3/./withShardField_b_replica2/data/]
   [junit4]   2> 2035646 INFO  (qtp1524582488-13056) [n:127.0.0.1:65303_solr c:withShardField s:d  x:withShardField_d_replica1] o.a.s.s.IndexSchema [withShardField_d_replica1] Schema name=cloud-dynamic
   [junit4]   2> 2035702 INFO  (qtp1244449539-13038) [n:127.0.0.1:65306_solr c:withShardField s:b  x:withShardField_b_replica5] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035703 INFO  (qtp1244449539-13038) [n:127.0.0.1:65306_solr c:withShardField s:b  x:withShardField_b_replica5] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_b_replica5' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035703 INFO  (qtp1244449539-13038) [n:127.0.0.1:65306_solr c:withShardField s:b  x:withShardField_b_replica5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65306.solr.core.withShardField.b.replica5' (registry 'solr.core.withShardField.b.replica5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035703 INFO  (qtp1244449539-13038) [n:127.0.0.1:65306_solr c:withShardField s:b  x:withShardField_b_replica5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035704 INFO  (qtp1244449539-13038) [n:127.0.0.1:65306_solr c:withShardField s:b  x:withShardField_b_replica5] o.a.s.c.SolrCore [[withShardField_b_replica5] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node2/withShardField_b_replica5], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node2/./withShardField_b_replica5/data/]
   [junit4]   2> 2035704 INFO  (qtp1454893590-13029) [n:127.0.0.1:65302_solr c:withShardField s:b  x:withShardField_b_replica4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035704 INFO  (qtp1454893590-13029) [n:127.0.0.1:65302_solr c:withShardField s:b  x:withShardField_b_replica4] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_b_replica4' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035705 INFO  (qtp1454893590-13029) [n:127.0.0.1:65302_solr c:withShardField s:b  x:withShardField_b_replica4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65302.solr.core.withShardField.b.replica4' (registry 'solr.core.withShardField.b.replica4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035705 INFO  (qtp1454893590-13029) [n:127.0.0.1:65302_solr c:withShardField s:b  x:withShardField_b_replica4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035705 INFO  (qtp1454893590-13029) [n:127.0.0.1:65302_solr c:withShardField s:b  x:withShardField_b_replica4] o.a.s.c.SolrCore [[withShardField_b_replica4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node1/withShardField_b_replica4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node1/./withShardField_b_replica4/data/]
   [junit4]   2> 2035709 INFO  (qtp1454893590-13032) [n:127.0.0.1:65302_solr c:withShardField s:a  x:withShardField_a_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035709 INFO  (qtp1454893590-13032) [n:127.0.0.1:65302_solr c:withShardField s:a  x:withShardField_a_replica1] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_a_replica1' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035733 INFO  (qtp1454893590-13032) [n:127.0.0.1:65302_solr c:withShardField s:a  x:withShardField_a_replica1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65302.solr.core.withShardField.a.replica1' (registry 'solr.core.withShardField.a.replica1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035733 INFO  (qtp1454893590-13032) [n:127.0.0.1:65302_solr c:withShardField s:a  x:withShardField_a_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035733 INFO  (qtp1454893590-13032) [n:127.0.0.1:65302_solr c:withShardField s:a  x:withShardField_a_replica1] o.a.s.c.SolrCore [[withShardField_a_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node1/withShardField_a_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node1/./withShardField_a_replica1/data/]
   [junit4]   2> 2035736 INFO  (qtp1524582488-13057) [n:127.0.0.1:65303_solr c:withShardField s:d  x:withShardField_d_replica5] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035736 INFO  (qtp1524582488-13057) [n:127.0.0.1:65303_solr c:withShardField s:d  x:withShardField_d_replica5] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_d_replica5' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035737 INFO  (qtp1524582488-13057) [n:127.0.0.1:65303_solr c:withShardField s:d  x:withShardField_d_replica5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65303.solr.core.withShardField.d.replica5' (registry 'solr.core.withShardField.d.replica5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035738 INFO  (qtp1524582488-13057) [n:127.0.0.1:65303_solr c:withShardField s:d  x:withShardField_d_replica5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035738 INFO  (qtp1524582488-13057) [n:127.0.0.1:65303_solr c:withShardField s:d  x:withShardField_d_replica5] o.a.s.c.SolrCore [[withShardField_d_replica5] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node4/withShardField_d_replica5], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node4/./withShardField_d_replica5/data/]
   [junit4]   2> 2035739 INFO  (qtp1524582488-13050) [n:127.0.0.1:65303_solr c:withShardField s:b  x:withShardField_b_replica3] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035739 INFO  (qtp1524582488-13050) [n:127.0.0.1:65303_solr c:withShardField s:b  x:withShardField_b_replica3] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_b_replica3' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035739 INFO  (qtp1524582488-13050) [n:127.0.0.1:65303_solr c:withShardField s:b  x:withShardField_b_replica3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65303.solr.core.withShardField.b.replica3' (registry 'solr.core.withShardField.b.replica3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035740 INFO  (qtp1524582488-13050) [n:127.0.0.1:65303_solr c:withShardField s:b  x:withShardField_b_replica3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035740 INFO  (qtp1524582488-13050) [n:127.0.0.1:65303_solr c:withShardField s:b  x:withShardField_b_replica3] o.a.s.c.SolrCore [[withShardField_b_replica3] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node4/withShardField_b_replica3], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node4/./withShardField_b_replica3/data/]
   [junit4]   2> 2035742 INFO  (qtp1524582488-13049) [n:127.0.0.1:65303_solr c:withShardField s:a  x:withShardField_a_replica4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035742 INFO  (qtp1524582488-13049) [n:127.0.0.1:65303_solr c:withShardField s:a  x:withShardField_a_replica4] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_a_replica4' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035742 INFO  (qtp1524582488-13049) [n:127.0.0.1:65303_solr c:withShardField s:a  x:withShardField_a_replica4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65303.solr.core.withShardField.a.replica4' (registry 'solr.core.withShardField.a.replica4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035742 INFO  (qtp1524582488-13049) [n:127.0.0.1:65303_solr c:withShardField s:a  x:withShardField_a_replica4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035742 INFO  (qtp1524582488-13049) [n:127.0.0.1:65303_solr c:withShardField s:a  x:withShardField_a_replica4] o.a.s.c.SolrCore [[withShardField_a_replica4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node4/withShardField_a_replica4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node4/./withShardField_a_replica4/data/]
   [junit4]   2> 2035744 INFO  (qtp1244449539-13043) [n:127.0.0.1:65306_solr c:withShardField s:b  x:withShardField_b_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035744 INFO  (qtp1244449539-13043) [n:127.0.0.1:65306_solr c:withShardField s:b  x:withShardField_b_replica1] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_b_replica1' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035744 INFO  (qtp1244449539-13043) [n:127.0.0.1:65306_solr c:withShardField s:b  x:withShardField_b_replica1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65306.solr.core.withShardField.b.replica1' (registry 'solr.core.withShardField.b.replica1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035744 INFO  (qtp1244449539-13043) [n:127.0.0.1:65306_solr c:withShardField s:b  x:withShardField_b_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035744 INFO  (qtp1244449539-13043) [n:127.0.0.1:65306_solr c:withShardField s:b  x:withShardField_b_replica1] o.a.s.c.SolrCore [[withShardField_b_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node2/withShardField_b_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node2/./withShardField_b_replica1/data/]
   [junit4]   2> 2035745 INFO  (qtp365726319-13046) [n:127.0.0.1:65304_solr c:withShardField s:d  x:withShardField_d_replica4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035745 INFO  (qtp365726319-13046) [n:127.0.0.1:65304_solr c:withShardField s:d  x:withShardField_d_replica4] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_d_replica4' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035745 INFO  (qtp365726319-13046) [n:127.0.0.1:65304_solr c:withShardField s:d  x:withShardField_d_replica4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65304.solr.core.withShardField.d.replica4' (registry 'solr.core.withShardField.d.replica4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035746 INFO  (qtp365726319-13046) [n:127.0.0.1:65304_solr c:withShardField s:d  x:withShardField_d_replica4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035746 INFO  (qtp365726319-13046) [n:127.0.0.1:65304_solr c:withShardField s:d  x:withShardField_d_replica4] o.a.s.c.SolrCore [[withShardField_d_replica4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node3/withShardField_d_replica4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node3/./withShardField_d_replica4/data/]
   [junit4]   2> 2035746 INFO  (qtp365726319-13055) [n:127.0.0.1:65304_solr c:withShardField s:c  x:withShardField_c_replica5] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035746 INFO  (qtp365726319-13055) [n:127.0.0.1:65304_solr c:withShardField s:c  x:withShardField_c_replica5] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_c_replica5' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035747 INFO  (qtp365726319-13055) [n:127.0.0.1:65304_solr c:withShardField s:c  x:withShardField_c_replica5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65304.solr.core.withShardField.c.replica5' (registry 'solr.core.withShardField.c.replica5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035747 INFO  (qtp365726319-13055) [n:127.0.0.1:65304_solr c:withShardField s:c  x:withShardField_c_replica5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035747 INFO  (qtp365726319-13055) [n:127.0.0.1:65304_solr c:withShardField s:c  x:withShardField_c_replica5] o.a.s.c.SolrCore [[withShardField_c_replica5] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node3/withShardField_c_replica5], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node3/./withShardField_c_replica5/data/]
   [junit4]   2> 2035748 INFO  (qtp365726319-13060) [n:127.0.0.1:65304_solr c:withShardField s:a  x:withShardField_a_replica3] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035749 INFO  (qtp365726319-13060) [n:127.0.0.1:65304_solr c:withShardField s:a  x:withShardField_a_replica3] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_a_replica3' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035749 INFO  (qtp365726319-13060) [n:127.0.0.1:65304_solr c:withShardField s:a  x:withShardField_a_replica3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65304.solr.core.withShardField.a.replica3' (registry 'solr.core.withShardField.a.replica3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035749 INFO  (qtp365726319-13060) [n:127.0.0.1:65304_solr c:withShardField s:a  x:withShardField_a_replica3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035749 INFO  (qtp365726319-13060) [n:127.0.0.1:65304_solr c:withShardField s:a  x:withShardField_a_replica3] o.a.s.c.SolrCore [[withShardField_a_replica3] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node3/withShardField_a_replica3], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node3/./withShardField_a_replica3/data/]
   [junit4]   2> 2035767 INFO  (qtp1454893590-13030) [n:127.0.0.1:65302_solr c:withShardField s:c  x:withShardField_c_replica3] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035767 INFO  (qtp1454893590-13030) [n:127.0.0.1:65302_solr c:withShardField s:c  x:withShardField_c_replica3] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_c_replica3' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035767 INFO  (qtp1454893590-13030) [n:127.0.0.1:65302_solr c:withShardField s:c  x:withShardField_c_replica3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65302.solr.core.withShardField.c.replica3' (registry 'solr.core.withShardField.c.replica3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035768 INFO  (qtp1454893590-13030) [n:127.0.0.1:65302_solr c:withShardField s:c  x:withShardField_c_replica3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035768 INFO  (qtp1454893590-13030) [n:127.0.0.1:65302_solr c:withShardField s:c  x:withShardField_c_replica3] o.a.s.c.SolrCore [[withShardField_c_replica3] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node1/withShardField_c_replica3], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node1/./withShardField_c_replica3/data/]
   [junit4]   2> 2035769 INFO  (qtp1454893590-13031) [n:127.0.0.1:65302_solr c:withShardField s:a  x:withShardField_a_replica5] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035769 INFO  (qtp1454893590-13031) [n:127.0.0.1:65302_solr c:withShardField s:a  x:withShardField_a_replica5] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_a_replica5' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035770 INFO  (qtp1454893590-13031) [n:127.0.0.1:65302_solr c:withShardField s:a  x:withShardField_a_replica5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65302.solr.core.withShardField.a.replica5' (registry 'solr.core.withShardField.a.replica5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035770 INFO  (qtp1454893590-13031) [n:127.0.0.1:65302_solr c:withShardField s:a  x:withShardField_a_replica5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035770 INFO  (qtp1454893590-13031) [n:127.0.0.1:65302_solr c:withShardField s:a  x:withShardField_a_replica5] o.a.s.c.SolrCore [[withShardField_a_replica5] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node1/withShardField_a_replica5], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node1/./withShardField_a_replica5/data/]
   [junit4]   2> 2035770 INFO  (qtp365726319-13051) [n:127.0.0.1:65304_solr c:withShardField s:c  x:withShardField_c_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035770 INFO  (qtp365726319-13051) [n:127.0.0.1:65304_solr c:withShardField s:c  x:withShardField_c_replica1] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_c_replica1' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035771 INFO  (qtp365726319-13051) [n:127.0.0.1:65304_solr c:withShardField s:c  x:withShardField_c_replica1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65304.solr.core.withShardField.c.replica1' (registry 'solr.core.withShardField.c.replica1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035771 INFO  (qtp365726319-13051) [n:127.0.0.1:65304_solr c:withShardField s:c  x:withShardField_c_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035771 INFO  (qtp365726319-13051) [n:127.0.0.1:65304_solr c:withShardField s:c  x:withShardField_c_replica1] o.a.s.c.SolrCore [[withShardField_c_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node3/withShardField_c_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node3/./withShardField_c_replica1/data/]
   [junit4]   2> 2035781 INFO  (qtp1524582488-13056) [n:127.0.0.1:65303_solr c:withShardField s:d  x:withShardField_d_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035781 INFO  (qtp1524582488-13056) [n:127.0.0.1:65303_solr c:withShardField s:d  x:withShardField_d_replica1] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_d_replica1' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035781 INFO  (qtp1524582488-13056) [n:127.0.0.1:65303_solr c:withShardField s:d  x:withShardField_d_replica1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65303.solr.core.withShardField.d.replica1' (registry 'solr.core.withShardField.d.replica1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035781 INFO  (qtp1524582488-13056) [n:127.0.0.1:65303_solr c:withShardField s:d  x:withShardField_d_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035781 INFO  (qtp1524582488-13056) [n:127.0.0.1:65303_solr c:withShardField s:d  x:withShardField_d_replica1] o.a.s.c.SolrCore [[withShardField_d_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node4/withShardField_d_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node4/./withShardField_d_replica1/data/]
   [junit4]   2> 2035786 INFO  (qtp1524582488-13058) [n:127.0.0.1:65303_solr c:withShardField s:c  x:withShardField_c_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035786 INFO  (qtp1524582488-13058) [n:127.0.0.1:65303_solr c:withShardField s:c  x:withShardField_c_replica2] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_c_replica2' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035786 INFO  (qtp1524582488-13058) [n:127.0.0.1:65303_solr c:withShardField s:c  x:withShardField_c_replica2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65303.solr.core.withShardField.c.replica2' (registry 'solr.core.withShardField.c.replica2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035786 INFO  (qtp1524582488-13058) [n:127.0.0.1:65303_solr c:withShardField s:c  x:withShardField_c_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035787 INFO  (qtp1524582488-13058) [n:127.0.0.1:65303_solr c:withShardField s:c  x:withShardField_c_replica2] o.a.s.c.SolrCore [[withShardField_c_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node4/withShardField_c_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node4/./withShardField_c_replica2/data/]
   [junit4]   2> 2035789 INFO  (qtp1244449539-13040) [n:127.0.0.1:65306_solr c:withShardField s:c  x:withShardField_c_replica4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035790 INFO  (qtp1244449539-13040) [n:127.0.0.1:65306_solr c:withShardField s:c  x:withShardField_c_replica4] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_c_replica4' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035791 INFO  (qtp1244449539-13040) [n:127.0.0.1:65306_solr c:withShardField s:c  x:withShardField_c_replica4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65306.solr.core.withShardField.c.replica4' (registry 'solr.core.withShardField.c.replica4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035791 INFO  (qtp1244449539-13040) [n:127.0.0.1:65306_solr c:withShardField s:c  x:withShardField_c_replica4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035791 INFO  (qtp1244449539-13040) [n:127.0.0.1:65306_solr c:withShardField s:c  x:withShardField_c_replica4] o.a.s.c.SolrCore [[withShardField_c_replica4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node2/withShardField_c_replica4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node2/./withShardField_c_replica4/data/]
   [junit4]   2> 2035908 INFO  (qtp1454893590-13028) [n:127.0.0.1:65302_solr c:withShardField s:d  x:withShardField_d_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035908 INFO  (qtp1454893590-13028) [n:127.0.0.1:65302_solr c:withShardField s:d  x:withShardField_d_replica2] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_d_replica2' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035909 INFO  (qtp1454893590-13028) [n:127.0.0.1:65302_solr c:withShardField s:d  x:withShardField_d_replica2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65302.solr.core.withShardField.d.replica2' (registry 'solr.core.withShardField.d.replica2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035909 INFO  (qtp1454893590-13028) [n:127.0.0.1:65302_solr c:withShardField s:d  x:withShardField_d_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035909 INFO  (qtp1454893590-13028) [n:127.0.0.1:65302_solr c:withShardField s:d  x:withShardField_d_replica2] o.a.s.c.SolrCore [[withShardField_d_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node1/withShardField_d_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node1/./withShardField_d_replica2/data/]
   [junit4]   2> 2035913 INFO  (qtp1244449539-13041) [n:127.0.0.1:65306_solr c:withShardField s:d  x:withShardField_d_replica3] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035913 INFO  (qtp1244449539-13041) [n:127.0.0.1:65306_solr c:withShardField s:d  x:withShardField_d_replica3] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_d_replica3' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035914 INFO  (qtp1244449539-13041) [n:127.0.0.1:65306_solr c:withShardField s:d  x:withShardField_d_replica3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65306.solr.core.withShardField.d.replica3' (registry 'solr.core.withShardField.d.replica3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035914 INFO  (qtp1244449539-13041) [n:127.0.0.1:65306_solr c:withShardField s:d  x:withShardField_d_replica3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035914 INFO  (qtp1244449539-13041) [n:127.0.0.1:65306_solr c:withShardField s:d  x:withShardField_d_replica3] o.a.s.c.SolrCore [[withShardField_d_replica3] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node2/withShardField_d_replica3], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node2/./withShardField_d_replica3/data/]
   [junit4]   2> 2035919 INFO  (qtp1244449539-13042) [n:127.0.0.1:65306_solr c:withShardField s:a  x:withShardField_a_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 2035920 INFO  (qtp1244449539-13042) [n:127.0.0.1:65306_solr c:withShardField s:a  x:withShardField_a_replica2] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_a_replica2' using configuration from collection withShardField, trusted=true
   [junit4]   2> 2035920 INFO  (qtp1244449539-13042) [n:127.0.0.1:65306_solr c:withShardField s:a  x:withShardField_a_replica2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_65306.solr.core.withShardField.a.replica2' (registry 'solr.core.withShardField.a.replica2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@686a85f9
   [junit4]   2> 2035920 INFO  (qtp1244449539-13042) [n:127.0.0.1:65306_solr c:withShardField s:a  x:withShardField_a_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2035920 INFO  (qtp1244449539-13042) [n:127.0.0.1:65306_solr c:withShardField s:a  x:withShardField_a_replica2] o.a.s.c.SolrCore [[withShardField_a_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node2/withShardField_a_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001/tempDir-001/node2/./withShardField_a_replica2/data/]
   [junit4]   2> 2036005 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2036005 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2036007 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2036007 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2036049 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@32e4657[withShardField_b_replica2] main]
   [junit4]   2> 2036136 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2036171 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2036194 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2036195 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1567004444002877440
   [junit4]   2> 2036208 INFO  (qtp365726319-13054) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard b: total=5 found=1 timeoutin=9998ms
   [junit4]   2> 2036404 INFO  (zkCallback-2150-thread-1-processing-n:127.0.0.1:65306_solr) [n:127.0.0.1:65306_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withShardField/state.json] for collection [withShardField] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2036405 INFO  (zkCallback-2147-thread-1-processing-n:127.0.0.1:65302_solr) [n:127.0.0.1:65302_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withShardField/state.json] for collection [withShardField] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2036405 INFO  (zkCallback-2149-thread-1-processing-n:127.0.0.1:65304_solr) [n:127.0.0.1:65304_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withShardField/state.json] for collection [withShardField] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2036404 INFO  (zkCallback-2148-thread-1-processing-n:127.0.0.1:65303_solr) [n:127.0.0.1:65303_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withShardField/state.json] for collection [withShardField] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2036410 INFO  (searcherExecutor-6158-thread-1-processing-n:127.0.0.1:65304_solr x:withShardField_b_replica2 s:b c:withShardField) [n:127.0.0.1:65304_solr c:withShardField s:b  x:withShardField_b_replica2] o.a.s.c.SolrCore [withShardField_b_replica2] Registered new searcher Searcher@32e4657[withShardField_b_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2036454 ERROR (qtp1524582488-13058) [n:127.0.0.1:65303_solr c:withShardField s:c  x:withShardField_c_replica2] o.a.s.a.ApiBag Unable to register plugin:org.apache.solr.api.ApiBag$ReqHandlerToApiwith spec :{
   [junit4]   2>   "documentation":"https://cwiki.apache.org/confluence/display/solr/Schema+API",
   [junit4]   2>   "methods":["GET"],
   [junit4]   2>   "url":{
   [junit4]   2>     "paths":[
   [junit4]   2>       "/schema/dynamicfields",
   [junit4]   2>       "/schema/dynamicfields/{name}",
   [junit4]   2>       "/schema/fieldtypes",
   [junit4]   2>       "/schema/fieldtypes/{name}"],
   [junit4]   2>     "params":{"showDefaults":{
   [junit4]   2>         "type":"boolean",
   [junit4]   2>         "description":"If true, all default field properties from each field's field type will be included in the response (e.g.   tokenized  for   solr.TextField). If false, only explicitly specified field properties will be included."}}}}
   [junit4]   2> java.lang.RuntimeException: invalid API spec: apispec/core.SchemaRead.dynamicFields_fieldTypes.json
   [junit4]   2> 	at org.apache.solr.common.util.ValidatingJsonMap.parse(ValidatingJsonMap.java:318)
   [junit4]   2> 	at org.apache.solr.api.ApiBag.lambda$getSpec$0(ApiBag.java:229)
   [junit4]   2> 	at org.apache.solr.api.Api.getSpec(Api.java:64)
   [junit4]   2> 	at org.apache.solr.api.ApiBag.validateAndRegister(ApiBag.java:109)
   [junit4]   2> 	at org.apache.solr.api.ApiBag.register(ApiBag.java:70)
   [junit4]   2> 	at org.apache.solr.core.PluginBag.put(PluginBag.java:215)
   [junit4]   2> 	at org.apache.solr.core.PluginBag.init(PluginBag.java:272)
   [junit4]   2> 	at org.apache.solr.core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java:130)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:960)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:960)
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:895)
   [junit4]   2> 	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
   [junit4]   2> 	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
   [junit4]   2> 	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:388)
   [junit4]   2> 	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
   [junit4]   2> 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:178)
   [junit4]   2> 	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:747)
   [junit4]   2> 	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:728)
   [junit4]   2> 	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:509)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:372)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:316)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 2036465 INFO  (qtp1524582488-13058) [n:127.0.0.1:65303_solr c:withShardField s:c  x:withShardField_c_replica2] o.a.s.c.SolrCore [withShardField_c_replica2]  CLOSING SolrCore org.apache.solr.core.SolrCore@4df3627f
   [junit4]   2> 2036476 ERROR (qtp1244449539-13040) [n:127.0.0.1:65306_solr c:withShardField s:c  x:withShardField_c_replica4] o.a.s.a.ApiBag Unable to register plugin:org.apache.solr.api.ApiBag$ReqHandlerToApiwith spec :{
   [junit4]   2>   "documentation":"https://cwiki.apache.org/confluence/display/solr/Schema+API",
   [junit4]   2>   "description":"The Schema API provides viewing, editing, adding, and deleting elements of Solr's schema. This API can only be used if Managed Schema is enabled and the schema is defined as mutable. See https://cwiki.apache.org/confluence/display/solr/Schema+Factory+Definition+in+SolrConfig for more information about enabling Managed Schema.",
   [junit4]   2>   "methods":["POST"],
   [junit4]   2>   "url":{"paths":["$handlerName"]},
   [junit4]   2>   "commands":{
   [junit4]   2>     "add-field":{
   [junit4]   2>       "documentation":"https://cwiki.apache.org/confluence/display/solr/Schema+API",
   [junit4]   2>       "type":"object",
   [junit4]   2>       "properties":{
   [junit4]   2>         "name":{
   [junit4]   2>           "type":"string",
   [junit4]   2>           "description":"The name of the field. Names should be alphanumeric or underscore characters only, and not start with a digit. Names also cannot begin and end with an underscore, as such field names are reserved by the system."},
   [junit4]   2>         "type":{
   [junit4]   2>           "type":"string",
   [junit4]   2>           "description":"The name of the fieldType for this field."},
   [junit4]   2>         "defaultValue":{
   [junit4]   2>           "type":"string",
   [junit4]   2>           "description":"An optional default value that should be added automatically to any document that does not have a value for this field."},
   [junit4]   2>         "indexed":{
   [junit4]   2>           "type":"boolean",
   [junit4]   2>           "description":"If true, the field will be indexed and will be available for use in queries to retrieve matching documents. If this is not defined, it will inherit the value from the fieldType. If the fieldType does not define a value, it will default to true.",
   [junit4]   2>           "default":"true"},
   [junit4]   2>         "stored":{
   [junit4]   2>           "type":"boolean",
   [junit4]   2>           "description":"If true, the actual value of the field can be retrieved by queries and be displayed in results. If this is not defined, it will inherit the value from the fieldType. If the fieldType does not define a value, it will default to true.",
   [junit4]   2>           "default":"true"},
   [junit4]   2>         "omitNorms":{
   [junit4]   2>           "type":"boolean",
   [junit4]   2>           "description":"If true, length normalization and index-time boosting for a field are omitted from the index. If this is not defined, it will inherit the value from the fieldType. If the fieldType does not define a value, it will default to true for all primitive field types (such as dates, strings, boolean, and numeric fields), but will default to false for non-primitive field types."},
   [junit4]   2>         "omitTermFreqAndPositions":{
   [junit4]   2>           "type":"boolean",
   [junit4]   2>           "description":"If true, all term frequency, positions, and payloads will not be indexed. This means that phrase queries, proximity queries and similar queries that rely on analysis of the frequency of a query term or the position of a term to other terms will not be supported. If this is not defined, it will inherit the value from the fieldType. If the fieldType does not define a value, it will default to true for all field types that are not text fields."},
   [junit4]   2>         "termVectors":{
   [junit4]   2>           "type":"boolean",
   [junit4]   2>           "description":"If true, term vectors will be stored to be able to compute similarity between two documents. This is required to use More Like This. If this is not defined, it will inherit the value from the fieldType. If the fieldType does not define a value, it will default to false. Do not enable this if using the Post

[...truncated too long message...]

127.0.0.1:65304_solr c:implicitcoll s:b r:core_node1 x:implicitcoll_b_replica4] o.a.s.c.LeaderInitiatedRecoveryThread LeaderInitiatedRecoveryThread-implicitcoll_b_replica1 completed successfully after running for 5040.0ms
   [junit4]   2> 2088886 WARN  (updateExecutor-2141-thread-7-processing-x:implicitcoll_a_replica5 r:core_node7 http:////127.0.0.1:65303//solr//implicitcoll_a_replica3// n:127.0.0.1:65304_solr s:a c:implicitcoll) [n:127.0.0.1:65304_solr c:implicitcoll s:a r:core_node7 x:implicitcoll_a_replica5] o.a.s.c.LeaderInitiatedRecoveryThread Stop trying to send recovery command to downed replica core=implicitcoll_a_replica3 coreNodeName=core_node10 on 127.0.0.1:65303_solr because my core container is closed.
   [junit4]   2> 2088887 INFO  (updateExecutor-2141-thread-7-processing-x:implicitcoll_a_replica5 r:core_node7 http:////127.0.0.1:65303//solr//implicitcoll_a_replica3// n:127.0.0.1:65304_solr s:a c:implicitcoll) [n:127.0.0.1:65304_solr c:implicitcoll s:a r:core_node7 x:implicitcoll_a_replica5] o.a.s.c.LeaderInitiatedRecoveryThread LeaderInitiatedRecoveryThread-implicitcoll_a_replica3 completed successfully after running for 5062.0ms
   [junit4]   2> 2088886 WARN  (updateExecutor-2141-thread-8-processing-x:implicitcoll_a_replica5 r:core_node7 http:////127.0.0.1:65302//solr//implicitcoll_a_replica4// n:127.0.0.1:65304_solr s:a c:implicitcoll) [n:127.0.0.1:65304_solr c:implicitcoll s:a r:core_node7 x:implicitcoll_a_replica5] o.a.s.c.LeaderInitiatedRecoveryThread Stop trying to send recovery command to downed replica core=implicitcoll_a_replica4 coreNodeName=core_node4 on 127.0.0.1:65302_solr because my core container is closed.
   [junit4]   2> 2088887 INFO  (updateExecutor-2141-thread-8-processing-x:implicitcoll_a_replica5 r:core_node7 http:////127.0.0.1:65302//solr//implicitcoll_a_replica4// n:127.0.0.1:65304_solr s:a c:implicitcoll) [n:127.0.0.1:65304_solr c:implicitcoll s:a r:core_node7 x:implicitcoll_a_replica5] o.a.s.c.LeaderInitiatedRecoveryThread LeaderInitiatedRecoveryThread-implicitcoll_a_replica4 completed successfully after running for 5063.0ms
   [junit4]   2> 2088897 WARN  (updateExecutor-2141-thread-4-processing-x:implicitcoll_b_replica4 r:core_node1 http:////127.0.0.1:65306//solr//implicitcoll_b_replica5// n:127.0.0.1:65304_solr s:b c:implicitcoll) [n:127.0.0.1:65304_solr c:implicitcoll s:b r:core_node1 x:implicitcoll_b_replica4] o.a.s.c.LeaderInitiatedRecoveryThread Stop trying to send recovery command to downed replica core=implicitcoll_b_replica5 coreNodeName=core_node13 on 127.0.0.1:65306_solr because my core container is closed.
   [junit4]   2> 2088897 INFO  (updateExecutor-2141-thread-4-processing-x:implicitcoll_b_replica4 r:core_node1 http:////127.0.0.1:65306//solr//implicitcoll_b_replica5// n:127.0.0.1:65304_solr s:b c:implicitcoll) [n:127.0.0.1:65304_solr c:implicitcoll s:b r:core_node1 x:implicitcoll_b_replica4] o.a.s.c.LeaderInitiatedRecoveryThread LeaderInitiatedRecoveryThread-implicitcoll_b_replica5 completed successfully after running for 5050.0ms
   [junit4]   2> 2090409 WARN  (zkCallback-2149-thread-1-processing-n:127.0.0.1:65304_solr) [n:127.0.0.1:65304_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 2090410 INFO  (jetty-closer-2129-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1630ce21{/solr,null,UNAVAILABLE}
   [junit4]   2> 2090411 ERROR (SUITE-CustomCollectionTest-seed#[C3E823C8F8473366]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2090412 INFO  (SUITE-CustomCollectionTest-seed#[C3E823C8F8473366]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:65300 65300
   [junit4]   2> 2090424 INFO  (Thread-3013) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:65300 65300
   [junit4]   2> 2098203 WARN  (Thread-3013) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	15	/solr/configs/conf
   [junit4]   2> 	9	/solr/aliases.json
   [junit4]   2> 	9	/solr/clusterprops.json
   [junit4]   2> 	4	/solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	9	/solr/clusterstate.json
   [junit4]   2> 	4	/solr/collections/testCreateShardRepFactor/state.json
   [junit4]   2> 	4	/solr/collections/implicitcoll/state.json
   [junit4]   2> 	4	/solr/collections/routeFieldColl/state.json
   [junit4]   2> 	4	/solr/collections/withShardField/state.json
   [junit4]   2> 	3	/solr/collections/withShardField/leader_elect/b/election/97937777569300491-core_node2-n_0000000000
   [junit4]   2> 	2	/solr/collections/withShardField/leader_elect/b/election/97937777569300489-core_node4-n_0000000002
   [junit4]   2> 	2	/solr/collections/implicitcoll/leader_elect/c/election/97937777569300492-core_node15-n_0000000000
   [junit4]   2> 	2	/solr/collections/implicitcoll/leader_elect/a/election/97937777569300489-core_node10-n_0000000002
   [junit4]   2> 	2	/solr/collections/withShardField/leader_elect/a/election/97937777569300490-core_node16-n_0000000001
   [junit4]   2> 	2	/solr/overseer_elect/election/97937777569300491-127.0.0.1:65304_solr-n_0000000001
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	27	/solr/collections
   [junit4]   2> 	9	/solr/live_nodes
   [junit4]   2> 
   [junit4]   2> 2098204 INFO  (SUITE-CustomCollectionTest-seed#[C3E823C8F8473366]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.CustomCollectionTest_C3E823C8F8473366-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {shard_s=FST50, _version_=PostingsFormat(name=Memory), id=Lucene50(blocksize=128)}, docValues:{}, maxPointsInLeafNode=951, maxMBSortInHeap=7.772601024216588, sim=RandomSimilarity(queryNorm=true): {}, locale=he, timezone=Asia/Yerevan
   [junit4]   2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=3,threads=1,free=274802752,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [TestDistributedSearch, OutOfBoxZkACLAndCredentialsProvidersTest, BlobRepositoryCloudTest, VMParamsZkACLAndCredentialsProvidersTest, CoreAdminRequestStatusTest, TestDelegationWithHadoopAuth, DistribDocExpirationUpdateProcessorTest, TestCollapseQParserPlugin, TestManagedSchemaThreadSafety, QueryResultKeyTest, TestNumericTerms64, HttpPartitionTest, TestFieldSortValues, TestSolrCloudWithHadoopAuthPlugin, SharedFSAutoReplicaFailoverUtilsTest, PeerSyncWithIndexFingerprintCachingTest, TestInitQParser, TestRTimerTree, QueryElevationComponentTest, DistributedSuggestComponentTest, SSLMigrationTest, SolrCLIZkUtilsTest, TestSweetSpotSimilarityFactory, TestSurroundQueryParser, SolrCloudReportersTest, TestSuggestSpellingConverter, DisMaxRequestHandlerTest, TestShardHandlerFactory, JsonValidatorTest, OverseerRolesTest, DocValuesMultiTest, TestStressUserVersions, SolrRequestParserTest, TestMiniSolrCloudCluster, TestCollationField, TestBulkSchemaConcurrent, SolrPluginUtilsTest, FastVectorHighlighterTest, PathHierarchyTokenizerFactoryTest, TestManagedSchema, SyncSliceTest, SolrCoreMetricManagerTest, TestFastOutputStream, OpenExchangeRatesOrgProviderTest, CollectionsAPIAsyncDistributedZkTest, TestSort, SimpleMLTQParserTest, TestUniqueKeyFieldResource, CdcrUpdateLogTest, DeleteStatusTest, InfixSuggestersTest, TestTolerantUpdateProcessorCloud, BlockJoinFacetDistribTest, TestDefaultStatsCache, TestFieldCache, BinaryUpdateRequestHandlerTest, TestRawResponseWriter, AssignTest, AlternateDirectoryTest, DeleteLastCustomShardedReplicaTest, TestStressLucene, TestSSLRandomization, ConcurrentDeleteAndCreateCollectionTest, ExitableDirectoryReaderTest, TestCursorMarkWithoutUniqueKey, HdfsChaosMonkeySafeLeaderTest, TestStressReorder, TestReRankQParserPlugin, SegmentsInfoRequestHandlerTest, ShardSplitTest, CollectionStateFormat2Test, MetricUtilsTest, MBeansHandlerTest, LeaderInitiatedRecoveryOnCommitTest, TestReplicationHandlerBackup, CopyFieldTest, TestWordDelimiterFilterFactory, SearchHandlerTest, OverseerTest, TestSolr4Spatial2, TestPartialUpdateDeduplication, TestUpdate, DistributedFacetPivotSmallTest, TestExtendedDismaxParser, TestCloudJSONFacetJoinDomain, BlockCacheTest, TestMaxScoreQueryParser, TestEmbeddedSolrServerConstructors, DistributedFacetPivotLongTailTest, TestNRTOpen, TestAuthorizationFramework, BufferStoreTest, PeerSyncTest, ActionThrottleTest, SchemaVersionSpecificBehaviorTest, TestIntervalFaceting, PreAnalyzedFieldManagedSchemaCloudTest, BasicDistributedZkTest, SolrTestCaseJ4Test, SampleTest, AddBlockUpdateTest, TestBinaryField, SortSpecParsingTest, ZkCLITest, TestReplicationHandler, TestConfigSetsAPIZkFailure, TestQuerySenderNoQuery, TestTestInjection, TestSolrQueryResponse, CheckHdfsIndexTest, TestDistributedStatsComponentCardinality, RecoveryAfterSoftCommitTest, TestFaceting, TestRandomCollapseQParserPlugin, DocValuesTest, ClassificationUpdateProcessorFactoryTest, TestSolrQueryParserDefaultOperatorResource, TestCoreContainer, TestLazyCores, TestGraphTermsQParserPlugin, TestRebalanceLeaders, HardAutoCommitTest, CursorMarkTest, TestSolrDeletionPolicy1, SynonymTokenizerTest, EnumFieldTest, TestUseDocValuesAsStored, UniqFieldsUpdateProcessorFactoryTest, TestXmlQParser, TestConfigOverlay, InfoHandlerTest, TestCSVResponseWriter, PluginInfoTest, BigEndianAscendingWordDeserializerTest, TestBulkSchemaAPI, TestCloudPseudoReturnFields, UUIDUpdateProcessorFallbackTest, TestMacros, BooleanFieldTest, CustomCollectionTest]
   [junit4] Completed [320/710 (1!)] on J0 in 64.80s, 4 tests, 2 errors <<< FAILURES!

[...truncated 44649 lines...]

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3998 - Still Unstable!

[Policeman Jenkins Server <je...@thetaphi.de>]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3998/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

6 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud

Error Message:
Could not find collection:test_col

Stack Trace:
java.lang.AssertionError: Could not find collection:test_col
	at __randomizedtesting.SeedInfo.seed([5EBE9B0C1EFBE251]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:159)
	at org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud.createMiniSolrCloudCluster(TestTolerantUpdateProcessorRandomCloud.java:127)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:847)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud

Error Message:
ObjectTracker found 13 object(s) that were not released!!! [SolrZkClient, SolrZkClient, ZkController, InternalHttpClient, ZkController, InternalHttpClient, SolrZkClient, InternalHttpClient, ZkController, InternalHttpClient, Overseer, InternalHttpClient, InternalHttpClient] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185)  at org.apache.solr.cloud.ZkController.<init>(ZkController.java:300)  at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)  at org.apache.solr.core.CoreContainer.load(CoreContainer.java:504)  at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)  at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)  at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)  at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)  at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)  at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)  at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)  at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)  at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)  at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)  at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)  at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185)  at org.apache.solr.cloud.ZkController.<init>(ZkController.java:300)  at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)  at org.apache.solr.core.CoreContainer.load(CoreContainer.java:504)  at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)  at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)  at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)  at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)  at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)  at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)  at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)  at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)  at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)  at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)  at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)  at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.ZkController  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.cloud.ZkController.<init>(ZkController.java:426)  at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)  at org.apache.solr.core.CoreContainer.load(CoreContainer.java:504)  at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)  at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)  at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)  at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)  at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)  at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)  at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)  at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)  at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)  at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)  at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)  at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)  at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:94)  at org.apache.solr.core.CoreContainer.load(CoreContainer.java:495)  at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)  at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)  at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)  at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)  at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)  at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)  at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)  at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)  at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)  at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)  at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)  at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.ZkController  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.cloud.ZkController.<init>(ZkController.java:426)  at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)  at org.apache.solr.core.CoreContainer.load(CoreContainer.java:504)  at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)  at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)  at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)  at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)  at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)  at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)  at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)  at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)  at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)  at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)  at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)  at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)  at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:94)  at org.apache.solr.core.CoreContainer.load(CoreContainer.java:495)  at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)  at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)  at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)  at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)  at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)  at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)  at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)  at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)  at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)  at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)  at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)  at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185)  at org.apache.solr.cloud.ZkController.<init>(ZkController.java:300)  at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)  at org.apache.solr.core.CoreContainer.load(CoreContainer.java:504)  at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)  at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)  at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)  at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)  at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)  at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)  at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)  at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)  at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)  at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)  at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)  at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)  at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:94)  at org.apache.solr.core.CoreContainer.load(CoreContainer.java:495)  at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)  at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)  at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)  at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)  at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)  at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)  at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)  at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)  at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)  at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)  at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)  at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.ZkController  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.cloud.ZkController.<init>(ZkController.java:426)  at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)  at org.apache.solr.core.CoreContainer.load(CoreContainer.java:504)  at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)  at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)  at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)  at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)  at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)  at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)  at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)  at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)  at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)  at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)  at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)  at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)  at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:191)  at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47)  at org.apache.solr.core.CoreContainer.load(CoreContainer.java:489)  at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)  at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)  at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)  at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)  at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)  at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)  at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)  at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)  at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)  at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)  at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)  at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.Overseer  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.cloud.Overseer.start(Overseer.java:527)  at org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:771)  at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)  at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)  at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:56)  at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:348)  at org.apache.solr.common.cloud.SolrZkClient$3.lambda$process$0(SolrZkClient.java:269)  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)  at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:191)  at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47)  at org.apache.solr.core.CoreContainer.load(CoreContainer.java:489)  at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)  at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)  at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)  at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)  at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)  at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)  at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)  at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)  at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)  at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)  at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)  at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)  at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:191)  at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47)  at org.apache.solr.core.CoreContainer.load(CoreContainer.java:489)  at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)  at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)  at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)  at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)  at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)  at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)  at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)  at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)  at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)  at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)  at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)  at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)  at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)  at java.util.concurrent.FutureTask.run(FutureTask.java:266)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)  at java.lang.Thread.run(Thread.java:748)  

Stack Trace:
java.lang.AssertionError: ObjectTracker found 13 object(s) that were not released!!! [SolrZkClient, SolrZkClient, ZkController, InternalHttpClient, ZkController, InternalHttpClient, SolrZkClient, InternalHttpClient, ZkController, InternalHttpClient, Overseer, InternalHttpClient, InternalHttpClient]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185)
	at org.apache.solr.cloud.ZkController.<init>(ZkController.java:300)
	at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)
	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:504)
	at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)
	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)
	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
	at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)
	at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185)
	at org.apache.solr.cloud.ZkController.<init>(ZkController.java:300)
	at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)
	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:504)
	at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)
	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)
	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
	at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)
	at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.ZkController
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.cloud.ZkController.<init>(ZkController.java:426)
	at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)
	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:504)
	at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)
	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)
	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
	at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)
	at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)
	at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:94)
	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:495)
	at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)
	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)
	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
	at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)
	at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.ZkController
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.cloud.ZkController.<init>(ZkController.java:426)
	at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)
	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:504)
	at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)
	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)
	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
	at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)
	at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)
	at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:94)
	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:495)
	at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)
	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)
	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
	at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)
	at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185)
	at org.apache.solr.cloud.ZkController.<init>(ZkController.java:300)
	at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)
	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:504)
	at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)
	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)
	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
	at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)
	at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)
	at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:94)
	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:495)
	at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)
	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)
	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
	at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)
	at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.ZkController
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.cloud.ZkController.<init>(ZkController.java:426)
	at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)
	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:504)
	at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)
	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)
	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
	at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)
	at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)
	at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:191)
	at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47)
	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:489)
	at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)
	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)
	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
	at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)
	at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.Overseer
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.cloud.Overseer.start(Overseer.java:527)
	at org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:771)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:56)
	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:348)
	at org.apache.solr.common.cloud.SolrZkClient$3.lambda$process$0(SolrZkClient.java:269)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)
	at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:191)
	at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47)
	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:489)
	at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)
	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)
	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
	at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)
	at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:289)
	at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:191)
	at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47)
	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:489)
	at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:255)
	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:175)
	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
	at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)
	at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)


	at __randomizedtesting.SeedInfo.seed([5EBE9B0C1EFBE251]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:302)
	at sun.reflect.GeneratedMethodAccessor38.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud

Error Message:
27 threads leaked from SUITE scope at org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud:     1) Thread[id=21041, name=OverseerHdfsCoreFailoverThread-97935683286335518-127.0.0.1:52291_solr-n_0000000004, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139)         at java.lang.Thread.run(Thread.java:748)    2) Thread[id=21043, name=zkCallback-4140-thread-3, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:748)    3) Thread[id=21046, name=zkCallback-4152-thread-4, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:748)    4) Thread[id=21047, name=zkCallback-4140-thread-4, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:748)    5) Thread[id=21035, name=zkCallback-4140-thread-2, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:748)    6) Thread[id=21037, name=zkCallback-4148-thread-2, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:748)    7) Thread[id=20926, name=zkCallback-4152-thread-1, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:748)    8) Thread[id=20870, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)    9) Thread[id=20899, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   10) Thread[id=20893, name=jetty-launcher-4100-thread-6-EventThread, state=WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)   11) Thread[id=20877, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   12) Thread[id=20924, name=jetty-launcher-4100-thread-5-EventThread, state=WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)   13) Thread[id=20914, name=zkCallback-4148-thread-1, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:748)   14) Thread[id=20892, name=jetty-launcher-4100-thread-6-SendThread(127.0.0.1:52275), state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)   15) Thread[id=21045, name=zkCallback-4148-thread-4, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:748)   16) Thread[id=20910, name=jetty-launcher-4100-thread-9-EventThread, state=WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)   17) Thread[id=21038, name=zkCallback-4152-thread-2, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:748)   18) Thread[id=20879, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   19) Thread[id=20897, name=zkCallback-4140-thread-1, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:748)   20) Thread[id=20923, name=jetty-launcher-4100-thread-5-SendThread(127.0.0.1:52275), state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)   21) Thread[id=21040, name=OverseerCollectionConfigSetProcessor-97935683286335518-127.0.0.1:52291_solr-n_0000000004, state=TIMED_WAITING, group=Overseer collection creation process.]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)         at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)         at org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:387)         at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:193)         at java.lang.Thread.run(Thread.java:748)   22) Thread[id=21042, name=zkCallback-4152-thread-3, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:748)   23) Thread[id=20891, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   24) Thread[id=20909, name=jetty-launcher-4100-thread-9-SendThread(127.0.0.1:52275), state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)   25) Thread[id=20868, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   26) Thread[id=21044, name=zkCallback-4148-thread-3, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:748)   27) Thread[id=21039, name=OverseerStateUpdate-97935683286335518-127.0.0.1:52291_solr-n_0000000004, state=TIMED_WAITING, group=Overseer state updater.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:165)         at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:139)         at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:184)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 27 threads leaked from SUITE scope at org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud: 
   1) Thread[id=21041, name=OverseerHdfsCoreFailoverThread-97935683286335518-127.0.0.1:52291_solr-n_0000000004, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139)
        at java.lang.Thread.run(Thread.java:748)
   2) Thread[id=21043, name=zkCallback-4140-thread-3, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
   3) Thread[id=21046, name=zkCallback-4152-thread-4, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
   4) Thread[id=21047, name=zkCallback-4140-thread-4, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
   5) Thread[id=21035, name=zkCallback-4140-thread-2, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
   6) Thread[id=21037, name=zkCallback-4148-thread-2, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
   7) Thread[id=20926, name=zkCallback-4152-thread-1, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
   8) Thread[id=20870, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
   9) Thread[id=20899, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  10) Thread[id=20893, name=jetty-launcher-4100-thread-6-EventThread, state=WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
  11) Thread[id=20877, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  12) Thread[id=20924, name=jetty-launcher-4100-thread-5-EventThread, state=WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
  13) Thread[id=20914, name=zkCallback-4148-thread-1, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
  14) Thread[id=20892, name=jetty-launcher-4100-thread-6-SendThread(127.0.0.1:52275), state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
        at java.lang.Thread.sleep(Native Method)
        

[...truncated too long message...]

4]    >    6) Thread[id=21043, name=zkCallback-4140-thread-3, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    7) Thread[id=21046, name=zkCallback-4152-thread-4, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    8) Thread[id=21047, name=zkCallback-4140-thread-4, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    9) Thread[id=21035, name=zkCallback-4140-thread-2, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >   10) Thread[id=21042, name=zkCallback-4152-thread-3, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >   11) Thread[id=21037, name=zkCallback-4148-thread-2, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >   12) Thread[id=20926, name=zkCallback-4152-thread-1, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >   13) Thread[id=20909, name=jetty-launcher-4100-thread-9-SendThread(127.0.0.1:52275), state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
   [junit4]    >   14) Thread[id=21044, name=zkCallback-4148-thread-3, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >   15) Thread[id=20914, name=zkCallback-4148-thread-1, state=TIMED_WAITING, group=TGRP-TestTolerantUpdateProcessorRandomCloud]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([5EBE9B0C1EFBE251]:0)
   [junit4] Completed [541/710 (3!)] on J1 in 94.78s, 0 tests, 2 failures, 2 errors <<< FAILURES!

[...truncated 43953 lines...]

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3997 - Still Unstable!

[Policeman Jenkins Server <je...@thetaphi.de>]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3997/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

4 tests failed.
FAILED:  org.apache.solr.cloud.CustomCollectionTest.testCustomCollectionsAPI

Error Message:
Could not find collection : implicitcoll

Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : implicitcoll
	at __randomizedtesting.SeedInfo.seed([1D99669E9EB14292:7778E8F5A32BF4EA]:0)
	at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:194)
	at org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:245)
	at org.apache.solr.cloud.CustomCollectionTest.testCustomCollectionsAPI(CustomCollectionTest.java:68)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.handler.admin.MetricsHandlerTest.testPropertyFilter

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([1D99669E9EB14292:7FF498DF513F22AC]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.junit.Assert.assertNotNull(Assert.java:537)
	at org.apache.solr.handler.admin.MetricsHandlerTest.testPropertyFilter(MetricsHandlerTest.java:201)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CustomCollectionTest.testRouteFieldForImplicitRouter

Error Message:
Collection not found: withShardField

Stack Trace:
org.apache.solr.common.SolrException: Collection not found: withShardField
	at __randomizedtesting.SeedInfo.seed([1D99669E9EB14292:48C98E0C32488D62]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:1401)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1094)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
	at org.apache.solr.cloud.CustomCollectionTest.testRouteFieldForImplicitRouter(CustomCollectionTest.java:141)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test

Error Message:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO

Stack Trace:
org.apache.solr.common.SolrException: java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:171)
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:117)
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:112)
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:99)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.printLayout(AbstractDistribZkTestBase.java:322)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:668)
	at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
	at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1842)
	at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:447)
	at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)
	at org.apache.solr.common.cloud.SolrZooKeeper.<init>(SolrZooKeeper.java:43)
	at org.apache.solr.common.cloud.ZkClientConnectionStrategy.createSolrZooKeeper(ZkClientConnectionStrategy.java:105)
	at org.apache.solr.common.cloud.DefaultConnectionStrategy.connect(DefaultConnectionStrategy.java:37)
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:146)
	... 47 more
Caused by: java.io.IOException: Too many open files
	at sun.nio.ch.IOUtil.makePipe(Native Method)
	at sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:84)
	at sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:42)
	at java.nio.channels.Selector.open(Selector.java:227)
	at org.apache.zookeeper.ClientCnxnSocketNIO.<init>(ClientCnxnSocketNIO.java:43)
	at sun.reflect.GeneratedConstructorAccessor112.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.lang.Class.newInstance(Class.java:442)
	at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1839)
	... 53 more




Build Log:
[...truncated 17 lines...]
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: org.eclipse.jgit.api.errors.TransportException: Operation timed out (Read failed)
	at org.jenkinsci.plugins.gitclient.JGitAPIImpl$5.execute(JGitAPIImpl.java:1432)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
	at hudson.remoting.UserRequest.perform(UserRequest.java:153)
	at hudson.remoting.UserRequest.perform(UserRequest.java:50)
	at hudson.remoting.Request$2.run(Request.java:336)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
	at ......remote call to MacOSX VBOX(Native Method)
	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545)
	at hudson.remoting.UserResponse.retrieve(UserRequest.java:253)
	at hudson.remoting.Channel.call(Channel.java:830)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
	at com.sun.proxy.$Proxy100.execute(Unknown Source)
	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1067)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1107)
	at hudson.scm.SCM.checkout(SCM.java:495)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1281)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:604)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529)
	at hudson.model.Run.execute(Run.java:1735)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:405)
Caused by: org.eclipse.jgit.api.errors.TransportException: Operation timed out (Read failed)
	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:135)
	at org.jenkinsci.plugins.gitclient.JGitAPIImpl$5.execute(JGitAPIImpl.java:1424)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
	at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
	at hudson.remoting.UserRequest.perform(UserRequest.java:153)
	at hudson.remoting.UserRequest.perform(UserRequest.java:50)
	at hudson.remoting.Request$2.run(Request.java:336)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.eclipse.jgit.errors.TransportException: Operation timed out (Read failed)
	at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:377)
	at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:308)
	at org.eclipse.jgit.transport.BasePackFetchConnection.fetch(BasePackFetchConnection.java:298)
	at org.eclipse.jgit.transport.FetchProcess.fetchObjects(FetchProcess.java:245)
	at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:161)
	at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:122)
	at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1201)
	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:128)
	... 11 more
Caused by: java.net.SocketException: Operation timed out (Read failed)
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
	at org.eclipse.jgit.transport.SideBandInputStream.read(SideBandInputStream.java:139)
	at org.eclipse.jgit.transport.PackParser.fill(PackParser.java:1173)
	at org.eclipse.jgit.transport.PackParser$InflaterStream.read(PackParser.java:1706)
	at java.io.InputStream.read(InputStream.java:101)
	at org.eclipse.jgit.transport.PackParser.whole(PackParser.java:1019)
	at org.eclipse.jgit.transport.PackParser.indexOneObject(PackParser.java:952)
	at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:523)
	at org.eclipse.jgit.internal.storage.file.ObjectDirectoryPackParser.parse(ObjectDirectoryPackParser.java:195)
	at org.eclipse.jgit.transport.PackParser.parse(PackParser.java:484)
	at org.eclipse.jgit.transport.BasePackFetchConnection.receivePack(BasePackFetchConnection.java:773)
	at org.eclipse.jgit.transport.BasePackFetchConnection.doFetch(BasePackFetchConnection.java:370)
	... 18 more
ERROR: null
Retrying after 10 seconds
Fetching changes from the remote Git repository
Cleaning workspace
Checking out Revision 3217fd7c3cc7868648133950bc98d1b480c673b7 (refs/remotes/origin/master)
No emails were triggered.
[description-setter] Description set: Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC
[Lucene-Solr-master-MacOSX] $ /Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/bin/ant "-Dargs=-XX:+UseCompressedOops -XX:+UseParallelGC" jenkins-hourly
Buildfile: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml

jenkins-hourly:

-print-java-info:
[java-info] java version "1.8.0_131"
[java-info] Java(TM) SE Runtime Environment (1.8.0_131-b11, Oracle Corporation)
[java-info] Java HotSpot(TM) 64-Bit Server VM (25.131-b11, Oracle Corporation)
[java-info] Test args: [-XX:+UseCompressedOops -XX:+UseParallelGC]

clean:

clean:

clean:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: Apache Ivy 2.3.0 - 20130110142753 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve-groovy:
[ivy:cachepath] :: resolving dependencies :: org.codehaus.groovy#groovy-all-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.codehaus.groovy#groovy-all;2.4.8 in public
[ivy:cachepath] :: resolution report :: resolve 394ms :: artifacts dl 3ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   1   |   0   |   0   |   0   ||   1   |   0   |
	---------------------------------------------------------------------

-test-with-heapdumps-enabled:
     [echo] Java HotSpot(TM) 64-Bit Server VM: Enabling heap dumps on OutOfMemoryError to dir '/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/heapdumps'.
    [mkdir] Created dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/heapdumps

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve-groovy:

test:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

-clover.load:

resolve-groovy:

-init-totals:

test-core:

-clover.disable:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

-clover.load:

-clover.classpath:

-clover.setup:

clover:

-check-git-state:

-git-cleanroot:

-copy-git-state:
     [copy] Copying 1 file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build

git-autoclean:

resolve:

init:

compile-core:
    [mkdir] Created dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java
    [javac] Compiling 823 source files to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/util/graph/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/document/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/util/bkd/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/codecs/compressing/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene60/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/codecs/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/util/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/index/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/util/fst/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/analysis/tokenattributes/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/search/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene62/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/util/packed/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/util/mutable/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene70/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/codecs/perfield/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/analysis/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/analysis/standard/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/store/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/codecs/blocktree/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/search/spans/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/search/similarities/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/util/automaton/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene50/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java/org/apache/lucene/geo/package-info.class
     [copy] Copying 3 files to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/java

compile-test-framework:

-check-git-state:

-git-cleanroot:

-copy-git-state:

git-autoclean:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

init:

compile-lucene-core:

-check-git-state:

-git-cleanroot:

-copy-git-state:

git-autoclean:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

init:

-clover.disable:

-clover.load:

-clover.classpath:

-clover.setup:

clover:

compile-core:

compile-codecs:

-check-git-state:

-git-cleanroot:

-copy-git-state:

git-autoclean:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

common.init:

compile-lucene-core:

init:

-clover.disable:

-clover.load:

-clover.classpath:

-clover.setup:

clover:

compile-core:
    [mkdir] Created dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/codecs/classes/java
    [javac] Compiling 67 source files to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/codecs/classes/java
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/codecs/classes/java/org/apache/lucene/codecs/bloom/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/codecs/classes/java/org/apache/lucene/codecs/memory/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/codecs/classes/java/org/apache/lucene/codecs/simpletext/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/codecs/classes/java/org/apache/lucene/codecs/blockterms/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/codecs/classes/java/org/apache/lucene/codecs/blocktreeords/package-info.class
     [copy] Copying 3 files to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/codecs/classes/java

-clover.disable:

-clover.load:

-clover.classpath:

-clover.setup:

clover:

common.compile-core:
    [mkdir] Created dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/test-framework/classes/java
    [javac] Compiling 187 source files to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/test-framework/classes/java
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/asserting/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/ramonly/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/test-framework/classes/java/org/apache/lucene/mockfile/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/cheapbastard/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/mockrandom/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/cranky/package-info.class
    [javac] Creating empty /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/compressing/dummy/package-info.class
     [copy] Copying 4 files to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/test-framework/classes/java

compile-core:

compile-test:
    [mkdir] Created dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/test
    [javac] Compiling 458 source files to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/test
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
     [copy] Copying 3 files to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/classes/test

install-junit4-taskdef:

validate:

resolve-groovy:

-init-totals:

-test:
    [mkdir] Created dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/test
[loadresource] Do not set property tests.explicitclass as its length is 0.
    [mkdir] Created dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/core/test/temp
    [mkdir] Created dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/.caches/test-stats/core
   [junit4] <JUnit4> says salut! Master seed: 2767953249906B09
   [junit4] Executing 451 suites with 2 JVMs.
   [junit4] 
   [junit4] Started J1 PID(651@localhost).
   [junit4] Started J0 PID(652@localhost).
   [junit4] Suite: org.apache.lucene.util.TestIOUtils
   [junit4] Completed [1/451] on J0 in 1.32s, 18 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCodecs
   [junit4] Completed [2/451] on J0 in 1.60s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSegmentReader
   [junit4] Completed [3/451] on J0 in 1.88s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestBufferedChecksum
   [junit4] Completed [4/451] on J0 in 0.06s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestFieldCacheRewriteMethod
   [junit4] Completed [5/451] on J0 in 1.84s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIntBlockPool
   [junit4] Completed [6/451] on J0 in 0.42s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestRoaringDocIdSet
   [junit4] Completed [7/451] on J0 in 1.53s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNumericDocValuesUpdates
   [junit4] IGNOR/A 0.02s J1 | TestNumericDocValuesUpdates.testTonsOfUpdates
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [8/451] on J1 in 12.09s, 31 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestCachingTokenFilter
   [junit4] Completed [9/451] on J1 in 0.10s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestWildcard
   [junit4] Completed [10/451] on J1 in 0.80s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBlendedTermQuery
   [junit4] Completed [11/451] on J1 in 0.06s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterNRTIsCurrent
   [junit4] Completed [12/451] on J1 in 6.56s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSegmentTermDocs
   [junit4] Completed [13/451] on J1 in 1.29s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestNGramPhraseQuery
   [junit4] Completed [14/451] on J1 in 0.06s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestNRTCachingDirectory
   [junit4] IGNOR/A 0.01s J1 | TestNRTCachingDirectory.testPendingDeletions
   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory
   [junit4] IGNOR/A 0.00s J1 | TestNRTCachingDirectory.testFsyncDoesntCreateNewFiles
   [junit4]    > Assumption #1: test only works for FSDirectory subclasses
   [junit4] Completed [15/451] on J1 in 2.71s, 44 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanNearQuery
   [junit4] Completed [16/451] on J1 in 0.10s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestLongPostings
   [junit4] Completed [17/451] on J1 in 2.16s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestSleepingLockWrapper
   [junit4] Completed [18/451] on J0 in 18.84s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestCharFilter
   [junit4] Completed [19/451] on J0 in 0.02s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTermScorer
   [junit4] Completed [20/451] on J0 in 0.18s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50StoredFieldsFormat
   [junit4] IGNOR/A 0.01s J0 | TestLucene50StoredFieldsFormat.testBigDocuments
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [21/451] on J0 in 7.44s, 17 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat2
   [junit4] Completed [22/451] on J0 in 3.99s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestLevenshteinAutomata
   [junit4] Completed [23/451] on J0 in 1.97s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestRegexpQuery
   [junit4] Completed [24/451] on J0 in 0.39s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestAllFilesHaveCodecHeader
   [junit4] Completed [25/451] on J0 in 0.81s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestPriorityQueue
   [junit4] Completed [26/451] on J0 in 1.93s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestReaderWrapperDVTypeCheck
   [junit4] Completed [27/451] on J0 in 0.21s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMaxDocs
   [junit4] IGNOR/A 0.00s J1 | TestIndexWriterMaxDocs.testExactlyAtTrueLimit
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes over two hours))
   [junit4] Completed [28/451] on J1 in 18.62s, 21 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestAllFilesDetectTruncation
   [junit4] Completed [29/451] on J0 in 2.53s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestSloppyMath
   [junit4] Completed [30/451] on J0 in 3.45s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFilterLeafReader
   [junit4] Completed [31/451] on J0 in 0.12s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestAttributeSource
   [junit4] Completed [32/451] on J0 in 0.03s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMultiThreadTermVectors
   [junit4] Completed [33/451] on J0 in 0.95s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestNamedSPILoader
   [junit4] Completed [34/451] on J0 in 0.02s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMergePolicy
   [junit4] Completed [35/451] on J0 in 12.09s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestIntRangeFieldQueries
   [junit4] IGNOR/A 0.00s J0 | TestIntRangeFieldQueries.testRandomBig
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [36/451] on J0 in 3.40s, 5 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCodecHoldsOpenFiles
   [junit4] Completed [37/451] on J0 in 0.04s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestStringMSBRadixSorter
   [junit4] Completed [38/451] on J0 in 3.85s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene70.TestLucene70NormsFormat
   [junit4] IGNOR/A 0.00s J1 | TestLucene70NormsFormat.testNCommonBig
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] IGNOR/A 0.00s J1 | TestLucene70NormsFormat.testMergeStability
   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it
   [junit4] IGNOR/A 0.00s J1 | TestLucene70NormsFormat.testSparseNCommonBig
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [39/451] on J1 in 29.01s, 29 tests, 3 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestFilterSpans
   [junit4] Completed [40/451] on J1 in 0.05s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestFileSwitchDirectory
   [junit4] IGNOR/A 0.01s J1 | TestFileSwitchDirectory.testFsyncDoesntCreateNewFiles
   [junit4]    > Assumption #1: test only works for FSDirectory subclasses
   [junit4] IGNOR/A 0.01s J1 | TestFileSwitchDirectory.testPendingDeletions
   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory
   [junit4] Completed [41/451] on J1 in 2.71s, 43 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNRTReaderWithThreads
   [junit4] Completed [42/451] on J1 in 1.50s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPhraseQuery
   [junit4] Completed [43/451] on J1 in 1.13s, 21 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.document.TestFieldType
   [junit4] Completed [44/451] on J1 in 0.01s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestTimSorter
   [junit4] Completed [45/451] on J1 in 0.10s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestIntsRef
   [junit4] Completed [46/451] on J1 in 0.01s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestEarlyTerminatingSortingCollector
   [junit4] Completed [48/451] on J1 in 4.57s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMerging
   [junit4] Completed [49/451] on J0 in 13.01s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestCloseableThreadLocal
   [junit4] Completed [50/451] on J0 in 0.01s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestRecyclingIntBlockAllocator
   [junit4] Completed [51/451] on J0 in 0.02s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTermVectorsReader
   [junit4] Completed [52/451] on J1 in 0.30s, 13 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestRateLimiter
   [junit4] Completed [53/451] on J0 in 0.74s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocumentsWriterDeleteQueue
   [junit4] Completed [54/451] on J0 in 0.09s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOnVMError
   [junit4] IGNOR/A 0.00s J0 | TestIndexWriterOnVMError.testCheckpoint
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [55/451] on J0 in 1.39s, 3 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestParallelLeafReader
   [junit4] Completed [56/451] on J0 in 0.38s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestForceMergeForever
   [junit4] Completed [57/451] on J0 in 0.77s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestTimSorterWorstCase
   [junit4] Completed [58/451] on J0 in 23.18s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTopDocsMerge
   [junit4] Completed [59/451] on J0 in 2.06s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanBoostQuery
   [junit4] Completed [60/451] on J0 in 0.17s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTermsEnum
   [junit4] Completed [61/451] on J1 in 29.62s, 13 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterFromReader
   [junit4] Completed [62/451] on J0 in 0.79s, 12 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestLogMergePolicy
   [junit4] Completed [63/451] on J1 in 0.09s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestCharArrayMap
   [junit4] Completed [64/451] on J0 in 0.28s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestDelegatingAnalyzerWrapper
   [junit4] Completed [65/451] on J1 in 0.12s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMultiCollector
   [junit4] Completed [66/451] on J0 in 1.24s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBooleanOr
   [junit4] Completed [67/451] on J0 in 3.04s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSumDocFreq
   [junit4] Completed [68/451] on J0 in 0.41s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat
   [junit4] Completed [69/451] on J1 in 5.35s, 27 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestRollback
   [junit4] Completed [70/451] on J1 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestStringHelper
   [junit4] Completed [71/451] on J1 in 0.01s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTermRangeQuery
   [junit4] Completed [72/451] on J1 in 0.19s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPerSegmentDeletes
   [junit4] Completed [73/451] on J1 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNoMergeScheduler
   [junit4] Completed [74/451] on J1 in 0.03s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanExplanationsOfNonMatches
   [junit4] Completed [75/451] on J0 in 1.21s, 30 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanMultiTermQueryWrapper
   [junit4] Completed [76/451] on J1 in 0.33s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocsWithFieldSet
   [junit4] Completed [77/451] on J1 in 0.02s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSort
   [junit4] Completed [78/451] on J0 in 0.42s, 25 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestRadixSelector
   [junit4] Completed [79/451] on J1 in 1.47s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterUnicode
   [junit4] Completed [80/451] on J0 in 2.91s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFlushByRamOrCountsPolicy
   [junit4] Completed [81/451] on J0 in 7.07s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestMSBRadixSorter
   [junit4] Completed [82/451] on J0 in 3.13s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocsAndPositions
   [junit4] Completed [83/451] on J0 in 0.30s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterExceptions2
   [junit4] Completed [84/451] on J1 in 12.20s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSortRescorer
   [junit4] Completed [85/451] on J0 in 1.00s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDateSort
   [junit4] Completed [86/451] on J0 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBooleanQuery
   [junit4] Completed [87/451] on J0 in 0.92s, 22 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.geo.TestGeoEncodingUtils
   [junit4] Completed [88/451] on J0 in 0.60s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestReqExclBulkScorer
   [junit4] Completed [89/451] on J0 in 0.09s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestBytesRefArray
   [junit4] Completed [90/451] on J0 in 0.10s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterForceMerge
   [junit4] Completed [91/451] on J1 in 5.75s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSearcherManager
   [junit4] Completed [92/451] on J1 in 6.05s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BPostings
   [junit4] IGNOR/A 0.00s J1 | Test2BPostings.test
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [93/451] on J1 in 0.00s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestConstantScoreQuery
   [junit4] Completed [94/451] on J1 in 0.09s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSortRandom
   [junit4] Completed [95/451] on J1 in 0.29s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldPostingsFormat2
   [junit4] Completed [96/451] on J1 in 1.07s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexingSequenceNumbers
   [junit4] Completed [97/451] on J1 in 13.53s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestParallelTermEnum
   [junit4] Completed [98/451] on J1 in 0.04s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestBytesRefHash
   [junit4] Completed [99/451] on J1 in 1.46s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFieldReuse
   [junit4] Completed [100/451] on J1 in 0.06s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.similarities.TestBooleanSimilarity
   [junit4] Completed [101/451] on J1 in 0.10s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestLongBitSet
   [junit4] IGNOR/A 0.00s J1 | TestLongBitSet.testHugeCapacity
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [102/451] on J1 in 0.20s, 9 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestBytesRefAttImpl
   [junit4] Completed [103/451] on J1 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50CompoundFormat
   [junit4] Completed [104/451] on J1 in 62.38s, 26 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestTrackingDirectoryWrapper
   [junit4] IGNOR/A 0.00s J1 | TestTrackingDirectoryWrapper.testPendingDeletions
   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory
   [junit4] IGNOR/A 0.00s J1 | TestTrackingDirectoryWrapper.testFsyncDoesntCreateNewFiles
   [junit4]    > Assumption #1: test only works for FSDirectory subclasses
   [junit4] Completed [105/451] on J1 in 0.64s, 47 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPositiveScoresOnlyCollector
   [junit4] Completed [106/451] on J1 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestExceedMaxTermLength
   [junit4] Completed [107/451] on J1 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.document.TestDateTools
   [junit4] Completed [108/451] on J1 in 0.04s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBooleanScorer
   [junit4] Completed [109/451] on J1 in 0.91s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestSetOnce
   [junit4] Completed [110/451] on J1 in 0.03s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50StoredFieldsFormatHighCompression
   [junit4] IGNOR/A 0.00s J1 | TestLucene50StoredFieldsFormatHighCompression.testBigDocuments
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [111/451] on J1 in 3.59s, 19 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocumentWriter
   [junit4] Completed [112/451] on J1 in 0.20s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestQueryRescorer
   [junit4] Completed [113/451] on J1 in 0.22s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPostingsOffsets
   [junit4] Completed [114/451] on J1 in 0.35s, 13 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestLongValuesSource
   [junit4] Completed [115/451] on J1 in 0.72s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMultiPhraseQuery
   [junit4] Completed [116/451] on J1 in 0.24s, 16 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene70.TestLucene70DocValuesFormat
   [junit4] IGNOR/A 0.00s J0 | TestLucene70DocValuesFormat.testTermsEnumRandomMany
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] IGNOR/A 0.00s J0 | TestLucene70DocValuesFormat.testSortedSetVariableLengthManyVsStoredFields
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] IGNOR/A 0.00s J0 | TestLucene70DocValuesFormat.testSortedVariableLengthManyVsStoredFields
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [117/451] on J0 in 97.35s, 126 tests, 3 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestFilterWeight
   [junit4] Completed [118/451] on J0 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.TestCodecUtil
   [junit4] Completed [119/451] on J0 in 0.03s, 15 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestOperations
   [junit4] Completed [120/451] on J0 in 0.33s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestAutomaton
   [junit4] Completed [121/451] on J1 in 2.52s, 56 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestStressIndexing
   [junit4] Completed [122/451] on J1 in 2.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestNot
   [junit4] Completed [123/451] on J1 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestCollectionUtil
   [junit4] Completed [124/451] on J0 in 4.94s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestOmitTf
   [junit4] Completed [125/451] on J0 in 1.91s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestQueryBuilder
   [junit4] Completed [126/451] on J0 in 0.09s, 23 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestIntroSelector
   [junit4] Completed [127/451] on J0 in 0.62s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestConjunctions
   [junit4] Completed [128/451] on J0 in 0.13s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexableField
   [junit4] Completed [129/451] on J0 in 0.59s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestSearch
   [junit4] Completed [130/451] on J0 in 0.14s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestInfoStream
   [junit4] Completed [131/451] on J0 in 0.07s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTryDelete
   [junit4] Completed [132/451] on J0 in 0.02s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMatchAllDocsQuery
   [junit4] Completed [133/451] on J0 in 0.06s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.packed.TestPackedInts
   [junit4] IGNOR/A 0.00s J1 | TestPackedInts.testBlockReaderOverflow
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [134/451] on J1 in 5.58s, 28 tests, 3 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BBinaryDocValues
   [junit4] IGNOR/A 0.00s J1 | Test2BBinaryDocValues.testFixedBinary
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 6 hours if the heap is 5gb))
   [junit4] IGNOR/A 0.00s J1 | Test2BBinaryDocValues.testVariableBinary
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 6 hours if the heap is 5gb))
   [junit4] Completed [135/451] on J1 in 0.00s, 2 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.compressing.TestFastCompressionMode
   [junit4] Completed [136/451] on J1 in 0.67s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestDemo
   [junit4] Completed [137/451] on J1 in 0.04s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocValuesIndexing
   [junit4] Completed [138/451] on J0 in 1.41s, 34 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBoostQuery
   [junit4] Completed [139/451] on J0 in 0.01s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestCharTermAttributeImpl
   [junit4] Completed [140/451] on J1 in 0.34s, 12 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIsCurrent
   [junit4] Completed [141/451] on J1 in 0.02s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTermQuery
   [junit4] Completed [142/451] on J1 in 0.03s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestExternalCodecs
   [junit4] Completed [143/451] on J0 in 0.73s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocValues
   [junit4] Completed [144/451] on J0 in 0.48s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTopDocsCollector
   [junit4] Completed [145/451] on J0 in 0.26s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPayloadsOnVectors
   [junit4] Completed [146/451] on J0 in 0.04s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecsAtNight
   [junit4] IGNOR/A 0.00s J0 | TestDuelingCodecsAtNight.testEquals
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] IGNOR/A 0.00s J0 | TestDuelingCodecsAtNight.testCrazyReaderEquals
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] IGNOR/A 0.00s J0 | TestDuelingCodecsAtNight.testBigEquals
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [147/451] on J0 in 0.00s, 3 tests, 3 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTerm
   [junit4] Completed [148/451] on J0 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene70.TestLucene70SegmentInfoFormat
   [junit4] Completed [149/451] on J0 in 0.31s, 16 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFilterCodecReader
   [junit4] Completed [150/451] on J0 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestByteSlices
   [junit4] Completed [151/451] on J1 in 2.13s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat3
   [junit4] Completed [152/451] on J1 in 1.05s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanOrQuery
   [junit4] Completed [153/451] on J1 in 0.01s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPrefixQuery
   [junit4] Completed [154/451] on J0 in 1.80s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestEarlyTermination
   [junit4] Completed [155/451] on J0 in 0.07s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSimilarityProvider
   [junit4] Completed [156/451] on J0 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPrefixCodedTerms
   [junit4] Completed [157/451] on J1 in 0.21s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDocValuesScoring
   [junit4] Completed [158/451] on J1 in 0.13s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestNeedsScores
   [junit4] Completed [159/451] on J1 in 0.06s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTerms
   [junit4] Completed [160/451] on J0 in 0.11s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestConsistentFieldNumbers
   [junit4] Completed [161/451] on J0 in 0.75s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestBagOfPositions
   [junit4] Completed [162/451] on J0 in 2.51s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSloppyPhraseQuery
   [junit4] Completed [163/451] on J0 in 0.68s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSynonymQuery
   [junit4] Completed [164/451] on J0 in 0.04s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSnapshotDeletionPolicy
   [junit4] Completed [165/451] on J0 in 1.17s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSwappedIndexFiles
   [junit4] Completed [166/451] on J0 in 0.06s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexReaderClose
   [junit4] Completed [167/451] on J0 in 0.27s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestIndexOrDocValuesQuery
   [junit4] Completed [168/451] on J0 in 0.08s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestByteArrayDataInput
   [junit4] Completed [169/451] on J0 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDeletionPolicy
   [junit4] Completed [170/451] on J1 in 6.04s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOnJRECrash
   [junit4] IGNOR/A 0.00s J1 | TestIndexWriterOnJRECrash.testNRTThreads
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [171/451] on J1 in 0.00s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMultiPhraseEnum
   [junit4] Completed [172/451] on J1 in 0.02s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFilterDirectoryReader
   [junit4] Completed [173/451] on J1 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestAtomicUpdate
   [junit4] Completed [174/451] on J0 in 4.09s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.geo.TestPolygon2D
   [junit4] Completed [175/451] on J1 in 4.64s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BTerms
   [junit4] IGNOR/A 0.00s J1 | Test2BTerms.test2BTerms
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=very slow, use 5g minimum heap))
   [junit4] Completed [176/451] on J1 in 0.00s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSortedNumericSortField
   [junit4] Completed [177/451] on J1 in 0.07s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTransactions
   [junit4] Completed [178/451] on J1 in 0.77s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestRollingUpdates
   [junit4] Completed [179/451] on J1 in 1.11s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestDeterminism
   [junit4] Completed [180/451] on J1 in 0.28s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSubScorerFreqs
   [junit4] Completed [181/451] on J1 in 0.12s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSegmentInfos
   [junit4] Completed [182/451] on J1 in 0.04s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestMmapDirectory
   [junit4] Completed [183/451] on J1 in 6.24s, 43 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestComplexExplanationsOfNonMatches
   [junit4] Completed [184/451] on J1 in 0.09s, 21 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestFuzzyQuery
   [junit4] Completed [185/451] on J1 in 1.85s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestIntArrayDocIdSet
   [junit4] Completed [186/451] on J1 in 0.68s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.mutable.TestMutableValues
   [junit4] Completed [187/451] on J1 in 0.01s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSloppyPhraseQuery2
   [junit4] Completed [188/451] on J1 in 0.52s, 9 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPointQueries
   [junit4] IGNOR/A 0.00s J0 | TestPointQueries.testRandomLongsBig
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] IGNOR/A 0.00s J0 | TestPointQueries.testRandomBinaryBig
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [189/451] on J0 in 20.53s, 49 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.compressing.TestHighCompressionMode
   [junit4] Completed [190/451] on J0 in 0.68s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanSearchEquivalence
   [junit4] Completed [191/451] on J0 in 0.80s, 33 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestAssertions
   [junit4] Completed [192/451] on J0 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestForUtil
   [junit4] Completed [193/451] on J0 in 0.06s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestSparseFixedBitSet
   [junit4] Completed [194/451] on J0 in 0.98s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterWithThreads
   [junit4] IGNOR/A 0.01s J0 | TestIndexWriterWithThreads.testOpenTwoIndexWritersOnDifferentThreads
   [junit4]    > Assumption #1: aborting test: timeout obtaining lock
   [junit4] Completed [195/451] on J0 in 1.24s, 12 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexFileDeleter
   [junit4] Completed [196/451] on J0 in 0.50s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestFilterIterator
   [junit4] Completed [197/451] on J0 in 0.84s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSearchWithThreads
   [junit4] Completed [198/451] on J0 in 1.61s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.bkd.TestBKD
   [junit4] IGNOR/A 0.00s J1 | TestBKD.testRandomBinaryBig
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [199/451] on J1 in 21.57s, 19 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDocValuesQueries
   [junit4] Completed [200/451] on J1 in 2.38s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMaxTermFrequency
   [junit4] Completed [201/451] on J1 in 0.11s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.document.TestField
   [junit4] Completed [202/451] on J1 in 0.07s, 23 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMultiTermConstantScore
   [junit4] Completed [203/451] on J1 in 0.30s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestSimpleFSLockFactory
   [junit4] Completed [204/451] on J0 in 10.84s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldPostingsFormat
   [junit4] IGNOR/A 0.00s J1 | TestPerFieldPostingsFormat.testPostingsEnumReuse
   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it
   [junit4] IGNOR/A 0.00s J1 | TestPerFieldPostingsFormat.testMergeStability
   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it
   [junit4] Completed [205/451] on J1 in 1.77s, 25 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestAllFilesHaveChecksumFooter
   [junit4] Completed [206/451] on J1 in 0.68s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMultiTermsEnum
   [junit4] Completed [207/451] on J1 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDoubleValuesSource
   [junit4] Completed [208/451] on J0 in 2.08s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.document.TestBinaryDocument
   [junit4] Completed [209/451] on J0 in 0.01s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestScorerPerf
   [junit4] Completed [210/451] on J0 in 0.99s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSameTokenSamePosition
   [junit4] Completed [211/451] on J0 in 0.08s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.bkd.Test2BBKDPoints
   [junit4] IGNOR/A 0.00s J0 | Test2BBKDPoints.test1D
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))
   [junit4] IGNOR/A 0.00s J0 | Test2BBKDPoints.test2D
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))
   [junit4] Completed [212/451] on J0 in 0.00s, 2 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test4GBStoredFields
   [junit4] IGNOR/A 0.00s J0 | Test4GBStoredFields.test
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [213/451] on J0 in 0.00s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestMinimize
   [junit4] Completed [214/451] on J0 in 1.09s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestArrayUtil
   [junit4] Completed [215/451] on J0 in 2.63s, 14 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.geo.TestGeoUtils
   [junit4] Completed [216/451] on J1 in 7.03s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestVersion
   [junit4] Completed [217/451] on J1 in 0.03s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestFrequencyTrackingRingBuffer
   [junit4] Completed [218/451] on J1 in 0.14s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.Test2BDocs
   [junit4] IGNOR/A 0.00s J1 | Test2BDocs.test2BDocs
   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=Takes ~30min))
   [junit4] Completed [219/451] on J1 in 0.00s, 1 test, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSegmentMerger
   [junit4] Completed [220/451] on J1 in 0.64s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestRollingBuffer
   [junit4] Completed [221/451] on J1 in 0.15s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPayloads
   [junit4] Completed [222/451] on J1 in 0.24s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBooleanRewrites
   [junit4] Completed [223/451] on J1 in 0.26s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFlex
   [junit4] Completed [224/451] on J1 in 0.37s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestDirectory
   [junit4] IGNOR/A 0.00s J1 | TestDirectory.testListAll
   [junit4]    > Assumption #1: this test does not expect extra files
   [junit4] Completed [225/451] on J1 in 1.06s, 4 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestCharsRef
   [junit4] Completed [226/451] on J1 in 0.02s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriter
   [junit4] Completed [227/451] on J0 in 7.23s, 73 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterReader
   [junit4] IGNOR/A 0.00s J0 | TestIndexWriterReader.testDuringAddIndexes
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [228/451] on J0 in 4.21s, 23 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.fst.TestFSTs
   [junit4] IGNOR/A 0.00s J0 | TestFSTs.testBigSet
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [229/451] on J0 in 2.23s, 20 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.fst.TestBytesStore
   [junit4] Completed [230/451] on J0 in 0.82s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestGraphTokenizers
   [junit4] Completed [231/451] on J0 in 2.29s, 23 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestAddIndexes
   [junit4] Completed [232/451] on J0 in 3.94s, 25 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestShardSearching
   [junit4] Completed [233/451] on J0 in 4.31s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldDocValuesFormat
   [junit4] Completed [234/451] on J1 in 25.95s, 110 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterExceptions
   [junit4] IGNOR/A 0.00s J0 | TestIndexWriterExceptions.testTooManyTokens
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [235/451] on J0 in 10.60s, 35 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestMultiMMap
   [junit4] Completed [236/451] on J1 in 6.61s, 54 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestBasics
   [junit4] Completed [237/451] on J0 in 2.28s, 26 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterDelete
   [junit4] IGNOR/A 0.00s J1 | TestIndexWriterDelete.testApplyDeletesOnFlush
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4] Completed [238/451] on J1 in 4.19s, 27 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNRTThreads
   [junit4] Completed [239/451] on J0 in 4.47s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBoolean2
   [junit4] Completed [240/451] on J0 in 2.99s, 10 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestPersistentSnapshotDeletionPolicy
   [junit4] Completed [241/451] on J0 in 1.31s, 13 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecs
   [junit4] Completed [242/451] on J1 in 7.72s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestStressAdvance
   [junit4] Completed [243/451] on J0 in 1.36s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNorms
   [junit4] Completed [244/451] on J1 in 2.26s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterCommit
   [junit4] Completed [245/451] on J1 in 7.06s, 14 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDirectoryReaderReopen
   [junit4] Completed [246/451] on J1 in 1.57s, 14 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTimeLimitingCollector
   [junit4] Completed [247/451] on J1 in 1.82s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSearchAfter
   [junit4] Completed [248/451] on J1 in 0.15s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.standard.TestStandardAnalyzer
   [junit4] Completed [249/451] on J0 in 13.90s, 36 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDocumentsWriterStallControl
   [junit4] Completed [250/451] on J0 in 2.51s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestConcurrentMergeScheduler
   [junit4] Completed [251/451] on J1 in 3.59s, 16 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNeverDelete
   [junit4] Completed [252/451] on J0 in 1.23s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTieredMergePolicy
   [junit4] Completed [253/451] on J1 in 1.24s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestBufferedIndexInput
   [junit4] Completed [254/451] on J0 in 0.36s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanations
   [junit4] Completed [255/451] on J0 in 2.36s, 70 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestWeakIdentityMap
   [junit4] Completed [256/451] on J1 in 3.99s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDirectoryReader
   [junit4] Completed [257/451] on J0 in 1.33s, 24 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestLockFactory
   [junit4] Completed [258/451] on J1 in 0.01s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestOmitNorms
   [junit4] Completed [259/451] on J0 in 0.65s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestFixedBitSet
   [junit4] Completed [260/451] on J1 in 1.19s, 19 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestComplexExplanations
   [junit4] Completed [261/451] on J0 in 0.65s, 21 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestRegexpRandom2
   [junit4] Completed [262/451] on J1 in 0.73s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestParallelCompositeReader
   [junit4] Completed [263/451] on J1 in 0.93s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.codecs.compressing.TestFastDecompressionMode
   [junit4] Completed [264/451] on J0 in 1.58s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCrash
   [junit4] Completed [265/451] on J0 in 0.32s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMultiDocValues
   [junit4] Completed [266/451] on J0 in 0.13s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.analysis.TestToken
   [junit4] Completed [267/451] on J0 in 0.03s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestFieldsReader
   [junit4] Completed [268/451] on J0 in 0.67s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSimpleSearchEquivalence
   [junit4] Completed [269/451] on J0 in 0.21s, 17 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestPagedBytes
   [junit4] Completed [270/451] on J1 in 2.77s, 4 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestCustomSearcherSort
   [junit4] Completed [271/451] on J0 in 0.31s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestHugeRamFile
   [junit4] Completed [272/451] on J0 in 1.60s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.similarities.TestSimilarityBase
   [junit4] Completed [273/451] on J1 in 2.93s, 32 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBooleanMinShouldMatch
   [junit4] Completed [274/451] on J0 in 1.05s, 17 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTermsEnum2
   [junit4] Completed [275/451] on J1 in 0.71s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestDeterminizeLexicon
   [junit4] Completed [276/451] on J0 in 0.59s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOnDiskFull
   [junit4] IGNOR/A 0.00s J1 | TestIndexWriterOnDiskFull.testAddIndexOnDiskFull
   [junit4]    > Assumption #1: This test cannot run with Memory codec
   [junit4] Completed [277/451] on J1 in 0.24s, 4 tests, 1 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpans
   [junit4] Completed [278/451] on J0 in 0.41s, 26 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCustomNorms
   [junit4] Completed [279/451] on J1 in 0.50s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMixedCodecs
   [junit4] Completed [280/451] on J0 in 0.64s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTransactionRollback
   [junit4] Completed [281/451] on J1 in 0.30s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestUTF32ToUTF8
   [junit4] Completed [282/451] on J0 in 0.69s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestSpanExplanations
   [junit4] Completed [283/451] on J1 in 1.06s, 30 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestStressIndexing2
   [junit4] Completed [284/451] on J0 in 1.13s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.store.TestRAMDirectory
   [junit4] IGNOR/A 0.00s J1 | TestRAMDirectory.testPendingDeletions
   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory
   [junit4] IGNOR/A 0.00s J1 | TestRAMDirectory.testFsyncDoesntCreateNewFiles
   [junit4]    > Assumption #1: test only works for FSDirectory subclasses
   [junit4] Completed [285/451] on J1 in 0.60s, 46 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestLazyProxSkipping
   [junit4] Completed [286/451] on J0 in 0.29s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestMergeSchedulerExternal
   [junit4] Completed [287/451] on J1 in 0.58s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestTermVectorsWriter
   [junit4] Completed [288/451] on J1 in 0.20s, 15 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestBagOfPostings
   [junit4] Completed [289/451] on J0 in 0.82s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestOmitPositions
   [junit4] Completed [290/451] on J1 in 0.35s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestStressNRT
   [junit4] Completed [291/451] on J1 in 0.07s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestThreadedForceMerge
   [junit4] Completed [292/451] on J0 in 0.62s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexInput
   [junit4] Completed [293/451] on J1 in 0.38s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestDisjunctionMaxQuery
   [junit4] Completed [294/451] on J0 in 0.40s, 12 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestNearSpansOrdered
   [junit4] Completed [295/451] on J1 in 0.57s, 19 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSameScoresWithThreads
   [junit4] Completed [296/451] on J0 in 0.71s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCrashCausesCorruptIndex
   [junit4] Completed [297/451] on J0 in 0.18s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMultiFields
   [junit4] Completed [298/451] on J1 in 0.23s, 3 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestForTooMuchCloning
   [junit4] Completed [299/451] on J1 in 0.09s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestLiveFieldValues
   [junit4] Completed [300/451] on J0 in 0.35s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.similarities.TestSimilarity2
   [junit4] Completed [301/451] on J0 in 0.19s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.spans.TestFieldMaskingSpanQuery
   [junit4] IGNOR/A 0.00s J0 | TestFieldMaskingSpanQuery.testSpans2
   [junit4]    > Assumption #1: Broken scoring: LUCENE-3723
   [junit4] IGNOR/A 0.00s J0 | TestFieldMaskingSpanQuery.testSimple2
   [junit4]    > Assumption #1: Broken scoring: LUCENE-3723
   [junit4] Completed [302/451] on J0 in 0.13s, 11 tests, 2 skipped
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestNoDeletionPolicy
   [junit4] Completed [303/451] on J0 in 0.10s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestDoc
   [junit4] Completed [304/451] on J1 in 0.76s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanationsOfNonMatches
   [junit4] Completed [305/451] on J0 in 0.06s, 70 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSegmentTermEnum
   [junit4] Completed [306/451] on J0 in 0.16s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPrefixInBooleanQuery
   [junit4] Completed [307/451] on J1 in 0.47s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestUniqueTermCount
   [junit4] Completed [308/451] on J0 in 0.03s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestUnicodeUtil
   [junit4] Completed [309/451] on J1 in 0.29s, 5 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestParallelReaderEmptyIndex
   [junit4] Completed [310/451] on J1 in 0.05s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.document.TestDocument
   [junit4] Completed [311/451] on J1 in 0.09s, 12 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestTopFieldCollector
   [junit4] Completed [312/451] on J1 in 0.08s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestAutomatonQuery
   [junit4] Completed [313/451] on J0 in 1.21s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.TestSentinelIntSet
   [junit4] Completed [314/451] on J1 in 0.10s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestWildcardRandom
   [junit4] Completed [315/451] on J0 in 0.12s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestPrefixRandom
   [junit4] Completed [316/451] on J1 in 0.15s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestRegexpRandom
   [junit4] Completed [317/451] on J0 in 0.12s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestMultiLevelSkipList
   [junit4] Completed [318/451] on J1 in 0.20s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestBooleanQueryVisitSubscorers
   [junit4] Completed [319/451] on J1 in 0.31s, 7 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestCheckIndex
   [junit4] Completed [320/451] on J0 in 0.65s, 4 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.TestSearchForDuplicates
   [junit4] Completed [321/451] on J1 in 0.05s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestSizeBoundedForceMerge
   [junit4] Completed [322/451] on J0 in 0.05s, 11 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.util.automaton.TestCompiledAutomaton
   [junit4] Completed [323/451] on J1 in 0.04s, 6 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestElevationComparator
   [junit4] Completed [324/451] on J0 in 0.02s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestBinaryTerms
   [junit4] Completed [325/451] on J1 in 0.15s, 2 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterConfig
   [junit4] Completed [326/451] on J0 in 0.11s, 8 tests
   [junit4] 
   [junit4] Suite: org.apache.lucene.search.TestMultiTermQueryRewrites
   [junit4] Completed [327

[...truncated too long message...]

X/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

common.resolve:

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/null921386967
     [copy] Copying 35 files to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/null921386967
   [delete] Deleting directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/null921386967

resolve-example:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/null1335169641
     [copy] Copying 206 files to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/null1335169641
   [delete] Deleting directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/null1335169641

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] 	found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] 	found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] 	found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] 	found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] 	found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] 	found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 45ms :: artifacts dl 7ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
	---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 118 minutes 1 second
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/*_pid*.log". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
[WARNINGS] Computing warning deltas based on reference build #3996
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)